Skip to content

fix: Handle sliced list arrays correctly in Avro writer#27254

Open
ty1260 wants to merge 1 commit intopola-rs:mainfrom
ty1260:avro-write-sliced-list-corruption-27090
Open

fix: Handle sliced list arrays correctly in Avro writer#27254
ty1260 wants to merge 1 commit intopola-rs:mainfrom
ty1260:avro-write-sliced-list-corruption-27090

Conversation

@ty1260
Copy link
Copy Markdown

@ty1260 ty1260 commented Apr 10, 2026

Summary

  • skip child values before the first logical offset when serializing sliced list arrays to Avro
  • add a regression test covering sliced required and nullable list[str] columns in Avro roundtrip

Alternative to #27137.
Fixes #27090.

Test plan

  • cargo test -p polars-arrow --features io_avro --lib

@github-actions github-actions bot added A-io-avro Area: reading/writing Avro files fix Bug fix python Related to Python Polars rust Related to Rust Polars first-contribution First contribution by user title needs formatting labels Apr 10, 2026
@ty1260 ty1260 changed the title fix: handle sliced list arrays correctly in Avro writer fix: Handle sliced list arrays correctly in Avro writer Apr 10, 2026
@ty1260 ty1260 marked this pull request as ready for review April 10, 2026 10:36
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 10, 2026

Codecov Report

❌ Patch coverage is 0% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.83%. Comparing base (880651f) to head (0865c86).

Files with missing lines Patch % Lines
crates/polars-arrow/src/io/avro/write/serialize.rs 0.00% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #27254      +/-   ##
==========================================
+ Coverage   81.58%   81.83%   +0.24%     
==========================================
  Files        1820     1820              
  Lines      251036   251045       +9     
  Branches     3149     3149              
==========================================
+ Hits       204808   205436     +628     
+ Misses      45420    44801     -619     
  Partials      808      808              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-io-avro Area: reading/writing Avro files first-contribution First contribution by user fix Bug fix python Related to Python Polars rust Related to Rust Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

write_avro corrupts list[str] values when sliced DataFrames are written to multiple Avro files

1 participant