fix(storage): retry os.replace in atomic_write on Windows NTFS lock c… - #3880
Open
zhiwu215 wants to merge 1 commit into
Open
fix(storage): retry os.replace in atomic_write on Windows NTFS lock c…#3880zhiwu215 wants to merge 1 commit into
zhiwu215 wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes transient
PermissionError: [WinError 5] Access is deniedduringatomic_writeon native Windows (NTFS) environments.On Windows,
os.replace(MoveFileExWwithMOVEFILE_REPLACE_EXISTING) fails withPermissionErrorif the target file (such askv_store_doc_status.json) is momentarily held open by another async worker, an IDE file watcher, or Windows Defender. Previously,atomic_writelacked a retry mechanism, causing a single millisecond-level lock collision during concurrent document ingestion to crash the entire process withPipeline halted on internal storage error.This PR introduces a lightweight exponential backoff retry specifically for
PermissionErroronwin32, while preserving zero-overhead single-pass behavior on POSIX platforms.Related Issues
Fixed #3879
Changes Made
_replace_filehelper inlightrag/file_atomic.pywith exponential backoff retry (up to 10 attempts, starting at 50ms) onPermissionError.sys.platform == "win32"(max_attempts = 10), preserving single-attempt zero-overhead behavior on POSIX platforms (max_attempts = 1).os.replacecall inatomic_writewith_replace_file.stdlib only).Checklist
Additional Notes
Local Testing & Verification