Skip to content

Commit 8423d74

Browse files
committed
[SPARK-51232][PYTHON][DOCS] Remove PySpark 3.3 and older logic from binder/postBuild
### What changes were proposed in this pull request? This PR aims to remove PySpark 3.3 and older logic from `binder/postBuild` because they are the end of life already and didn't have `PySpark Connect` module. ### Why are the changes needed? After simplifying by removing legacy PySpark without `PySpark Connect` module, we are going to add additional logic to handle `PySpark Connect 4.0+` because they are different from `PySpark Connect 3.4 ~ 3.5`. ### Does this PR introduce _any_ user-facing change? No, this is a kind of documentation via Binder service. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #49973 from dongjoon-hyun/SPARK-51232. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 0002cdd commit 8423d74

File tree

1 file changed

+1
-5
lines changed

1 file changed

+1
-5
lines changed

binder/postBuild

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,7 @@ else
3737
SPECIFIER="<="
3838
fi
3939

40-
if [[ ! $VERSION < "3.4.0" ]]; then
41-
pip install plotly "pandas<2.0.0" "pyspark[sql,ml,mllib,pandas_on_spark,connect]$SPECIFIER$VERSION"
42-
else
43-
pip install plotly "pandas<2.0.0" "pyspark[sql,ml,mllib,pandas_on_spark]$SPECIFIER$VERSION"
44-
fi
40+
pip install plotly "pandas<2.0.0" "pyspark[sql,ml,mllib,pandas_on_spark,connect]$SPECIFIER$VERSION"
4541

4642
# Set 'PYARROW_IGNORE_TIMEZONE' to suppress warnings from PyArrow.
4743
echo "export PYARROW_IGNORE_TIMEZONE=1" >> ~/.profile

0 commit comments

Comments
 (0)