This is currently blocked by apache/arrow-rs-object-store#106
We need:
- New API documentation from AWS to know what needs to be updated
- arrow-rs has to be updated
- arrow-rs needs to be released
- datafusion needs to update
- lance can then update
to test things out, we can certainly just fork arrow-rs and datafusion to create a new custom build.