Skip to content

Add fsync() of _directories_ when renaming files to .committed #7030

@eddyashton

Description

@eddyashton

While reading about atomic rename semantics as part of #7029, we've surfaced something further that we may want to do to ensure the renames succeed, with the contents of the newly created file. Specifically, if we don't fsync() the parent directory, it is technically possible for the rename (especially around restarts) to access a version of the directory metadata that doesn't know the file exists. So we should, when dealing with .committed files:

  1. Before closeing a .committed file, fsync() that file
  2. After close(), call fsync() on the parent directory, to minimise loss window, and be sure it's known before an incoming rename() call
  3. After rename(), call fsync() on the destination directory, to minimise the window where the rename could be lost

This probably requires shifting to direct fopen/fwrite calls, so we have file-descriptor access for fsync(), and dropping the current use of std::filesystem/std::ofstream in many places.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions