Commit e00cd4e
[SPARK-54206][CONNECT][FOLLOWUP] Use VARBINARY type and reasonable max length for BinaryType
### What changes were proposed in this pull request?
This PR improves the JDBC type mapping for BinaryType in the Spark Connect JDBC client
### Why are the changes needed?
- **Semantic correctness**: Types.VARBINARY (variable-length) better matches Spark's BinaryType semantics.
- **Industry alignment**:
SQL Server dialect already uses VARBINARY(MAX) for BinaryType .
Trino JDBC driver uses VARBINARY with a maximum of 1 GB.
MariaDB JDBC driver uses VARBINARY/LONGVARBINARY for blob types
### Does this PR introduce _any_ user-facing change?
Yes, but minimal impact.
Both BINARY and VARBINARY map to byte array types
The precision change is within reasonable bounds
### How was this patch tested?
Existing tests: All tests in `SparkConnectJdbcDataTypeSuite` pass.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53252 from vinodkc/br_SPARK-54206_followup_fix.
Authored-by: vinodkc <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 87a8b56)
Signed-off-by: Dongjoon Hyun <[email protected]>1 parent d50e9b7 commit e00cd4e
File tree
2 files changed
+6
-6
lines changed- sql/connect/client/jdbc/src
- main/scala/org/apache/spark/sql/connect/client/jdbc/util
- test/scala/org/apache/spark/sql/connect/client/jdbc
2 files changed
+6
-6
lines changedLines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
42 | | - | |
| 42 | + | |
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
| |||
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
86 | | - | |
| 86 | + | |
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
| |||
Lines changed: 4 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
219 | | - | |
| 219 | + | |
220 | 220 | | |
221 | | - | |
| 221 | + | |
222 | 222 | | |
223 | 223 | | |
224 | 224 | | |
| |||
389 | 389 | | |
390 | 390 | | |
391 | 391 | | |
392 | | - | |
| 392 | + | |
393 | 393 | | |
394 | 394 | | |
395 | 395 | | |
| |||
405 | 405 | | |
406 | 406 | | |
407 | 407 | | |
408 | | - | |
| 408 | + | |
409 | 409 | | |
410 | 410 | | |
411 | 411 | | |
| |||
0 commit comments