Skip to content

Optimize merging of partial case expression results #18152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
+445 −110
Open

Optimize merging of partial case expression results #18152

Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
e8698f7
Reduce record batch filtering in case_when_no_expr
pepijnve Oct 18, 2025
115b554
Reduce record batch filtering in case_when_with_expr
pepijnve Oct 18, 2025
9c641aa
Avoid unnecessary filtering in last iteration when no else expression…
pepijnve Oct 19, 2025
7539068
Use interleave to construct case result
pepijnve Oct 20, 2025
b4c1117
Add comments
pepijnve Oct 20, 2025
7eae3e4
Handle null where values correctly in `case_when_with_expr`
pepijnve Oct 21, 2025
f5448c9
Align `case_when_with_expr` and `case_when_no_expr`
pepijnve Oct 21, 2025
132757e
Exit early when all base values are null
pepijnve Oct 21, 2025
43c2fe2
Avoid calling `interleave` in simple cases
pepijnve Oct 21, 2025
f49d3ea
Formatting
pepijnve Oct 21, 2025
7e21a19
Merge branch 'main' into case_reduce_filtering
pepijnve Oct 21, 2025
9cb6496
Use a custom merge strategy that takes the case evaluation logic into…
pepijnve Oct 23, 2025
37f1334
Always optimize filters
pepijnve Oct 23, 2025
1b2942b
Remove accidental addition of pub
pepijnve Oct 23, 2025
f6a5734
Add comments regarding `RecordBatch::new_unchecked` usage
pepijnve Oct 23, 2025
3030557
Attempt to clarify merge logic
pepijnve Oct 23, 2025
cad318e
Rename arguments
pepijnve Oct 23, 2025
d57e7b6
Formatting
pepijnve Oct 23, 2025
7f60d68
More diagrams
pepijnve Oct 23, 2025
9cf46b2
More comments
pepijnve Oct 23, 2025
fc04bd5
Remove redundant comment
pepijnve Oct 23, 2025
7dba554
Calculate slice length once
pepijnve Oct 24, 2025
44b763c
Avoid filtering when all base values are null
pepijnve Oct 24, 2025
be578d4
Avoid filtering when branch matches all remaining rows
pepijnve Oct 24, 2025
53cb37b
Avoid expanding scalars in case_when_with_expr
pepijnve Oct 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading