Support file locking on writable mounts #1787
Description
Tell us more about this new feature.
Hello team! First, really thank you for this project and for maintaining it.
We use Mountpoint for Amazon S3 as a writable mounted filesystem for application data.
Operations on existing files that are commonly used for file locking fail with EPERM (Operation not permitted).
One concrete example is huggingface_hub.snapshot_download(local_dir=...), which downloads a repository into a local folder. In our case, this fails on the Mountpoint-backed path when the library performs file locking using .lock files under .cache/huggingface/....
I understand that file locking is not supported by Mountpoint’s current semantics, so this is likely expected behavior rather than a bug.
The difficulty is that the workaround requires local temp staging first. In our case, huggingface_hub snapshot_download(local_dir=...) cannot run directly on the Mountpoint-backed path, so the model must first be downloaded to a local POSIX filesystem and then copied into the mounted path. For large artifacts such as 200 GB models, provisioning enough local disk in Kubernetes just for staging is often impractical.
Environment
- EKS
- Mountpoint used through
aws-mountpoint-s3-csi-driver - Writable S3-backed volume
- Also tested with
allow-deleteandallow-overwrite