Skip to content

Commit 35596e8

Browse files
chenhao-dbdongjoon-hyun
authored andcommitted
[SPARK-52833][SQL] Fix VariantBuilder.appendFloat
### What changes were proposed in this pull request? There is a small bug that `appendFloat` appends 8 bytes, which should be 4 instead. This doesn't cause a correctness issue, because `writeLong` writes the least significant `numBytes` bytes. When int is extended to long, the lower 4 bytes don't change. This may cause an exception when the buffer doesn't have enough capacity. ### Why are the changes needed? Bug fix. ### Does this PR introduce _any_ user-facing change? Yes, as stated above. ### How was this patch tested? Unit test. It would fail (throw an exception) without the fix. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #51526 from chenhao-db/fix_float_to_variant. Authored-by: Chenhao Li <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent faae05a commit 35596e8

File tree

2 files changed

+4
-1
lines changed

2 files changed

+4
-1
lines changed

common/variant/src/main/java/org/apache/spark/types/variant/VariantBuilder.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -229,7 +229,7 @@ public void appendTimestampNtz(long microsSinceEpoch) {
229229
public void appendFloat(float f) {
230230
checkCapacity(1 + 4);
231231
writeBuffer[writePos++] = primitiveHeader(FLOAT);
232-
writeLong(writeBuffer, writePos, Float.floatToIntBits(f), 8);
232+
writeLong(writeBuffer, writePos, Float.floatToIntBits(f), 4);
233233
writePos += 4;
234234
}
235235

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionSuite.scala

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -908,6 +908,9 @@ class VariantExpressionSuite extends SparkFunSuite with ExpressionEvalHelper {
908908
check(Array[Byte](1, 2, 3), "\"AQID\"")
909909
check(Literal(0, DateType), "\"1970-01-01\"")
910910

911+
val floatArray = Array.tabulate(25) { i => i.toFloat }
912+
check(floatArray, floatArray.mkString("[", ",", "]"))
913+
911914
withSQLConf(SQLConf.SESSION_LOCAL_TIMEZONE.key -> "UTC") {
912915
check(Literal(0L, TimestampType), "\"1970-01-01 00:00:00+00:00\"")
913916
check(Literal(0L, TimestampNTZType), "\"1970-01-01 00:00:00\"")

0 commit comments

Comments
 (0)