Skip to content

feat: Add ignore_nulls to {list,arr}.{any,all}#27186

Open
coastalwhite wants to merge 5 commits intopola-rs:mainfrom
coastalwhite:feat/list-arr-any-all-ignore-nulls
Open

feat: Add ignore_nulls to {list,arr}.{any,all}#27186
coastalwhite wants to merge 5 commits intopola-rs:mainfrom
coastalwhite:feat/list-arr-any-all-ignore-nulls

Conversation

@coastalwhite
Copy link
Copy Markdown
Collaborator

Fixes #16764.

This PR dispatches {list,arr}.{any,all} to
{list,arr}.agg(pl.element().{any,all}() removing a bunch of code and giving us the ignore_nulls flag for free. This still should be efficient (although maybe a tiny bit more memory), as this will go through the group aggregation evaluation engine.

No AI was used.

Fixes pola-rs#16764.

This PR dispatches `{list,arr}.{any,all}` to
`{list,arr}.agg(pl.element().{any,all}()` removing a bunch of code and giving
us the `ignore_nulls` flag for free. This still should be efficient (although
maybe a tiny bit less) as this will go through the group aggregation evaluation
engine.
@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Apr 4, 2026
@github-actions github-actions bot added the changes-dsl Do not merge if this label is present and red. label Apr 4, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 4, 2026

Codecov Report

❌ Patch coverage is 88.23529% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.71%. Comparing base (7eb80ee) to head (32c556b).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-plan/src/dsl/expr/mod.rs 50.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #27186      +/-   ##
==========================================
+ Coverage   80.85%   81.71%   +0.85%     
==========================================
  Files        1816     1814       -2     
  Lines      250589   250439     -150     
  Branches     3147     3147              
==========================================
+ Hits       202618   204649    +2031     
+ Misses      47166    44985    -2181     
  Partials      805      805              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@ritchie46 ritchie46 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good to me. I'd rather improve the generic path performance indeed. We need to wait merging until we can break DSL.

@orlp
Copy link
Copy Markdown
Member

orlp commented Apr 7, 2026

Just for my sanity, the generic list aggregation path is marked as elementwise, right?

@coastalwhite
Copy link
Copy Markdown
Collaborator Author

Just for my sanity, the generic list aggregation path is marked as elementwise, right?

Indeed, {list,arr}.{eval,agg} are all always elementwise, regardless of their evaluation expression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changes-dsl Do not merge if this label is present and red. enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add the argument ignore_nulls in .arr.all(), .arr.any(), .list.all() and .list.any()

3 participants