Skip to content

Commit 99270f0

Browse files
vinodkcdongjoon-hyun
authored andcommitted
[SPARK-54551][CONNECT][TESTS] Fix random hangs in SparkConnectJdbcDataTypeSuite
### What changes were proposed in this pull request? Added explicit stdout/stderr redirection to Redirect.DISCARD for the child Spark Connect server process in non-debug mode in SparkConnectServerUtils. ### Why are the changes needed? SparkConnectJdbcDataTypeSuite randomly hangs because the child server process blocks on write() calls when stdout/stderr pipe buffers fill up. Without consuming the output, the buffers reach capacity and cause the process to block indefinitely. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually tested verified `SparkConnectJdbcDataTypeSuite` no longer randomly hangs and all tests pass consistently. ### Was this patch authored or co-authored using generative AI tooling? No Closes #53261 from vinodkc/br_handle_std_pipe. Authored-by: vinodkc <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 9e9358a commit 99270f0

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/test/RemoteSparkSession.scala

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,11 @@ object SparkConnectServerUtils {
9393
if (isDebug) {
9494
builder.redirectError(Redirect.INHERIT)
9595
builder.redirectOutput(Redirect.INHERIT)
96+
} else {
97+
// If output is not consumed, the stdout/stderr pipe buffers will fill up,
98+
// causing the server process to block on write() calls
99+
builder.redirectError(Redirect.DISCARD)
100+
builder.redirectOutput(Redirect.DISCARD)
96101
}
97102

98103
val process = builder.start()

0 commit comments

Comments
 (0)