Add delete file index to pyiceberg and support equality delete reads

[geruh]

Closes #1210

Summary

This work was primarily done by @rutb327 while I provided guidance!

This PR adds equality delete read support to PyIceberg by implementing the delete file indexing system that matches delete files to data files, mimicking the behavior found in Iceberg Core. With this implementation we are able to index files and now read equality deletes during table scans.

Design details

Delete File Index

The new DeleteFileIndex class centralizes handling of all delete file types: positional deletes, equality deletes, and deletion vectors. It organizes deletes by type (equality vs. positional), partition (using PartitionMap for spec-aware grouping), and path (for path-specific positional deletes). This enables efficient lookup during table scans, reducing unnecessary delete file processing.

Equality Delete support

Equality delete files are loaded as PyArrow Tables with their respective equality ids for the schema and for each we are grouping tables with the same set equality id's to reduce anti join operations.

Testing

Added tests from the core iceberg DeleteFileIndex test suite and added some tests with dummy files. As well as some manual testing with a flink setup.

table_eq with only equality deletes on id=2, id=5
+---+-------+
| id|   data|
+---+-------+
|  1|  Alice|
|  3|Charlie|
|  4|  David|
|  6|  Frank|
+---+-------+

table_eq_pos with equality deletes and positional delete at position 3
+---+-----+
| id| data|
+---+-----+
|  1|Alice|
|  4|David|
|  6|Frank|
+---+-----+

Are there any user-facing changes?

Yes can read tables with equality deletes

[gabeiglio]

I noticed that this PR addresses the same issue/feature as the one I was working on in here. However, your implementation is more complete (by supporting reading equality deletes and deletion vectors), so I think it makes sense to move forward with this one instead. (cc: @sungwy, since you reviewed my PR)

[kevinjqliu]

oops, sorry @gabeiglio, I was searching for positional deletes in github search and i didnt see that you were already working on it in that PR. Looks like there are some parts of the PR that is still super useful to get merged, like the validates.

[gabeiglio]

Yea exactly, should have been more clear on my message, my implementation for DeleteFileIndex was a scope creep to achieve the validation. so now that PR can be only for the validation instead of partition maps, delete file index, etc. :) @kevinjqliu