Skip to content

feat(writer): Add PositionDeleteFileWriter for position delete files #1898

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ethan-tyler wants to merge 1 commit into from
Closed

feat(writer): Add PositionDeleteFileWriter for position delete files #1898

ethan-tyler wants to merge 1 commit into from
+1,271 −0

Conversation

@ethan-tyler
Copy link

@ethan-tyler ethan-tyler commented Dec 3, 2025 •
edited
Loading

Which issue does this PR close?

Part of [EPIC] Transaction Support Issues and Pull Requests #1339
Contributes to #735 (Delete support) and #1104 (RowDeltaAction)

What changes are included in this PR?

Adds PositionDeleteFileWriter for writing position delete Parquet files per Iceberg spec:

Writer layer:

  • PositionDeleteFileWriter writes position delete files with correct Iceberg field IDs
  • Schema: file_path (string, field ID 2147483546), pos (i64, field ID 2147483545)
  • Tracks referenced_data_file for single-file delete optimization
  • Comprehensive schema validation (column count, types, non-nullable file_path)
  • Runtime validation for negative position values

Are these changes tested?

13 unit tests for PositionDeleteFileWriter:

  • Schema validation (wrong column count, wrong types, nullable file_path)
  • Runtime validation (negative positions)
  • Edge cases (multiple files, multiple batches, Unicode paths, i64::MAX positions)
  • Parquet field ID verification against Iceberg spec

@ethan-tyler ethan-tyler changed the title feat(transaction): Add DeleteAction for committing delete files feat(writer): Add PositionDeleteFileWriter for position delete files Dec 4, 2025
@ethan-tyler
Copy link
Author

ethan-tyler commented Dec 4, 2025

This PR is part of a 3-PR series to add delete file support:

  • This PR - PositionDeleteFileWriter (writer layer)
  • PR 2 - DeleteAction + SnapshotProducer changes (transaction layer)
  • PR 3 - Integration test with REST catalog

@ethan-tyler
feat(writer): Add PositionDeleteFileWriter for position delete files
a4c0726 
Add PositionDeleteFileWriter for writing position delete Parquet files
per Iceberg spec. Position deletes use a specific schema with file_path
(field ID 2147483546) and pos (field ID 2147483545) columns.

- Add PositionDeleteWriterConfig with Iceberg spec field IDs
- Implement PositionDeleteFileWriterBuilder with IcebergWriterBuilder trait
- Implement PositionDeleteFileWriter with IcebergWriter trait
- Track referenced_data_file for single-file delete optimization
- Add comprehensive schema validation with detailed error messages
- Add 13 tests covering edge cases, validation, and Parquet field IDs
@ethan-tyler ethan-tyler closed this Dec 4, 2025
@ethan-tyler ethan-tyler deleted the feat/delete-action branch December 4, 2025 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ethan-tyler