Skip to content

fix(storage): retry os.replace in atomic_write on Windows NTFS lock c… - #3880

Open
zhiwu215 wants to merge 1 commit into
HKUDS:mainfrom
zhiwu215:fix/windows-atomic-write-retry
Open

fix(storage): retry os.replace in atomic_write on Windows NTFS lock c…#3880
zhiwu215 wants to merge 1 commit into
HKUDS:mainfrom
zhiwu215:fix/windows-atomic-write-retry

Conversation

@zhiwu215

@zhiwu215 zhiwu215 commented Sep 8, 2026

Copy link
Copy Markdown

Description

Fixes transient PermissionError: [WinError 5] Access is denied during atomic_write on native Windows (NTFS) environments.

On Windows, os.replace (MoveFileExW with MOVEFILE_REPLACE_EXISTING) fails with PermissionError if the target file (such as kv_store_doc_status.json) is momentarily held open by another async worker, an IDE file watcher, or Windows Defender. Previously, atomic_write lacked a retry mechanism, causing a single millisecond-level lock collision during concurrent document ingestion to crash the entire process with Pipeline halted on internal storage error.

This PR introduces a lightweight exponential backoff retry specifically for PermissionError on win32, while preserving zero-overhead single-pass behavior on POSIX platforms.

Related Issues

Fixed #3879

Changes Made

  • Added _replace_file helper in lightrag/file_atomic.py with exponential backoff retry (up to 10 attempts, starting at 50ms) on PermissionError.
  • Scoped multi-attempt retry specifically to sys.platform == "win32" (max_attempts = 10), preserving single-attempt zero-overhead behavior on POSIX platforms (max_attempts = 1).
  • Replaced direct os.replace call in atomic_write with _replace_file.
  • Maintained pure standard library dependencies (stdlib only).

Checklist

  • Changes tested locally
  • Code reviewed
  • Documentation updated (if necessary)
  • Unit tests added (if applicable)

Additional Notes

Local Testing & Verification

  • Environment: Native Windows (NTFS), Python 3.12, LightRAG.
  • Test Case: Batch ingestion and graph building for a 143-chapter novel (500k+ words).
  • Result: Prior to this change, the pipeline consistently halted around document 80~90 due to concurrent doc-status flush lock contention. With this retry logic applied, all 143 chapters completed smoothly with zero pipeline aborts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: PermissionError [WinError 5] Access is denied during atomic_write on Windows NTFS

1 participant