Skip to content

refactor: Sink DSL and callback for Iceberg#27258

Open
nameexhaustion wants to merge 2 commits intomainfrom
nxs/iceberg-sink-dsl
Open

refactor: Sink DSL and callback for Iceberg#27258
nameexhaustion wants to merge 2 commits intomainfrom
nxs/iceberg-sink-dsl

Conversation

@nameexhaustion
Copy link
Copy Markdown
Collaborator

@nameexhaustion nameexhaustion commented Apr 10, 2026

  • Add a SinkType::Iceberg to the DSL, replacing the existing usage of ::PipeWithSchema
  • Add a SinkedPathsCallback::IcebergCommit variant, and refactors Iceberg commit to use this callback instead of a separate file-listing

@github-actions github-actions bot added A-io-iceberg Related to Apache Iceberg tables. internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars labels Apr 10, 2026
@nameexhaustion nameexhaustion changed the title refactor: Sink DSL for Iceberg refactor: Sink DSL and callback for Iceberg Apr 10, 2026
@github-actions github-actions bot added the changes-dsl Do not merge if this label is present and red. label Apr 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

The uncompressed lib size after this PR is 58.9041 MB.

@nameexhaustion nameexhaustion marked this pull request as ready for review April 10, 2026 11:01
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 10, 2026

Codecov Report

❌ Patch coverage is 86.30137% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.19%. Comparing base (efe654e) to head (788e7f1).
⚠️ Report is 32 commits behind head on main.

Files with missing lines Patch % Lines
py-polars/src/polars/io/iceberg/_sink.py 86.79% 6 Missing and 1 partial ⚠️
.../polars-plan/src/dsl/options/iceberg_sink_state.rs 81.81% 4 Missing ⚠️
crates/polars-utils/src/pl_str.rs 66.66% 3 Missing ⚠️
crates/polars-utils/src/python_function.rs 50.00% 3 Missing ⚠️
crates/polars-plan/src/dsl/options/sink.rs 90.47% 2 Missing ⚠️
crates/polars-plan/src/client/check.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #27258      +/-   ##
==========================================
- Coverage   81.80%   81.19%   -0.61%     
==========================================
  Files        1816     1821       +5     
  Lines      250592   251105     +513     
  Branches     3144     3144              
==========================================
- Hits       204985   203882    -1103     
- Misses      44803    46423    +1620     
+ Partials      804      800       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-io-iceberg Related to Apache Iceberg tables. changes-dsl Do not merge if this label is present and red. internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant