GAUNT-NAMESPACE-DURABILITY-SHADOW-FS: prove directory-entry crash semantics, not just durable byte prefixes

[heyoub]

Why now

BatPak already has a serious crash-recovery gauntlet. The current SimFs/recovery stack proves a real Store over a faulting filesystem can crash, reopen, and satisfy prefix/no-undead/hash-chain legality. The recovery matrix also covers honest disk, lying fsync, and crash-before-fsync boundaries. That is not the gap.

The gap is narrower and nastier: namespace durability. Today the simulation is excellent at file-content durability, but the matrix explicitly models SimFs::sync_parent_dir as always durable: a crash truncates file contents, but never loses or resurrects directory entries. That leaves a SQLite-class power-loss surface under-modeled: create/rename/unlink/persist directory-entry survival after crashes.

This matters because BatPak's own StoreFs contract says sync_parent_dir must not return Ok before the freshly-created file's name survives a crash, and StagedFile::persist must be an atomic publish. If the simulator assumes that namespace truth for free, the gauntlet cannot falsify an embedder backend that loses the name, resurrects an old name, keeps both names, or leaves temp/final entries in an impossible combination.

No open duplicate found in the pre-flight searches: parent directory sync namespace durability, SimFs parent-dir fsync namespace rename remove persist temp crash, directory entry survives crash namespace rename unlink parent-dir sync, VFS shadow durability, GAUNT-NAMESPACE-DURABILITY.

Repo evidence from this scan

Current strength:

• StoreFs has the right durability contract: sync means durable for both file contents and parent directory names; staged publish must be old-or-new, never torn mixture; rename/remove are crash-sensitive compaction points.
  

  batpak/bpk-lib/crates/core/src/store/platform/fs.rs

  Lines 124 to 155 in 2afd2e4

  	}
  	pub(crate) fn open_file(path: &Path) -> io::Result<File> {
  	File::open(path)
  	}
  	pub(crate) fn read(path: &Path) -> io::Result<Vec<u8>> {
  	std::fs::read(path)
  	}
  	pub(crate) fn read_dir(path: &Path) -> io::Result<ReadDir> {
  	std::fs::read_dir(path)
  	}
  	pub(crate) fn create_dir_all(path: &Path) -> io::Result<()> {
  	std::fs::create_dir_all(path)
  	}
  	pub(crate) fn canonicalize(path: &Path) -> io::Result<PathBuf> {
  	std::fs::canonicalize(path)
  	}
  	pub(crate) fn metadata(path: &Path) -> io::Result<Metadata> {
  	std::fs::metadata(path)
  	}
  	pub(crate) fn symlink_metadata(path: &Path) -> io::Result<Metadata> {
  	std::fs::symlink_metadata(path)
  	}
  	pub(crate) fn remove_file(path: &Path) -> io::Result<()> {
  	std::fs::remove_file(path)

• persist_temp_with_parent_sync does the platform thing on Unix: sync temp file, persist/rename, then sync parent directory; non-Unix is explicitly a rename-only platform asymmetry.
  

  batpak/bpk-lib/crates/core/src/store/platform/sync.rs

  Lines 8 to 17 in 2afd2e4

  	/// The tempfile's contents must already be fsynced by the caller (this
  	/// function only handles the rename + directory-entry durability). On
  	/// unix the parent directory is opened and `sync_all`-ed after the
  	/// rename, so a crash immediately after this returns cannot lose the
  	/// directory entry that points at the new inode. On non-unix targets
  	/// `File::sync_all` on a directory is not meaningful, so the parent
  	/// fsync is skipped — the asymmetry is platform-level, not a shortcut:
  	/// windows POSIX semantics for directory fsync simply do not exist.
  	pub(crate) fn persist_temp_with_parent_sync(
  	named_temp: tempfile::NamedTempFile,

• SimFs already models dropped handle syncs, durable byte prefixes, crash truncation, op faults on rename/remove/persist, and positioned-read faults.
  

  batpak/bpk-lib/crates/core/src/store/sim/fs.rs

  Lines 11 to 23 in 2afd2e4

  	//! **honored**: the file's current inner length becomes its durable
  	//! length. Under the seed's schedule a sync is **dropped**: the call still
  	//! returns `Ok` to the store (a silently-lying disk), but the durable
  	//! length is NOT advanced, so the most recent bytes are lost on the next
  	//! crash.
  	//! * [`SimFs::crash`] truncates every tracked file to its last durable
  	//! length, discarding the write-but-unsynced (and sync-dropped) tail.
  	//! This models power loss losing the OS page-cache tail. (Truncation
  	//! reaches through the platform seam and therefore requires a
  	//! real-file-backed inner today; the fault schedules themselves are
  	//! backend-agnostic.)
  	//!
  	//! Reopening a real [`crate::store::Store`] over the same data directory after

• The B3 recovery matrix explicitly states the current terminal parent-dir model: parent-dir sync is treated as always durable; pure dropped parent-dir sync is not independently modeled.
  

  batpak/bpk-lib/crates/core/src/store/sim/recovery_matrix.rs

  Lines 40 to 45 in 2afd2e4

  	//! created file), so a pure "parent-dir sync dropped" mode is not independently
  	//! modeled. The [`Boundary::SegmentRotationCreate`] cell exercises the
  	//! new-segment-create + dir-sync window via the injector instead. Witnessed by
  	//! `recovery_matrix.rs::sim_parent_dir_sync_fail_closed_model_honest_deferral`.
  	use super::fs::SimFs;

• The gate registry already gives recovery-oracle and linearizability real blocking authority with red fixtures. So this issue should extend the model, not reinvent the existing recovery oracle.
  

  batpak/bpk-lib/tools/integrity/src/gate_registry.rs

  Lines 199 to 219 in 2afd2e4

  	// backend, driven through the real append/sync API, crashed at the
  	// durability boundary, and reopened; the oracle fails closed on any
  	// illegal recovered state (lost-after-sync commit, undead event, broken
  	// hash chain, non-canonical reopen) or nondeterminism. Under
  	// `gauntlet_red_fixture` the test asserts the (illegal) lost-after-sync
  	// outcome, so its red half fails — proving the oracle bites. ---
  	Gate {
  	slug: "dst-recovery",
  	red_fixture_test: Some(
  	"crates/core/tests/dst_recovery.rs::dst_recovery_is_legal_and_deterministic",
  	),
  	red_fixture_kind: Some(RedFixtureKind::ProductionFlip),
  	has_blocking_authority: true,
  	},
  	// --- Phase-B3 recovery-oracle matrix (blocking, qualified ProductionFlip).
  	// B2's dst-recovery ran the legality oracle under ONE fault profile
  	// (honest disk). B3 generalizes it across the FULL hostile-fs matrix SimFs
  	// can model over the real Store: honest-disk crash, lying-disk fsync-drop,
  	// and crash-before-fsync at each durability boundary (single-append frame,
  	// batch-commit marker, post-fsync-before-publish, segment-rotation create).
  	// Every cell must recover EXACTLY one of {CommittedPrefix | RolledBack |

  
  

  batpak/bpk-lib/tools/integrity/src/gate_registry.rs

  Lines 20 to 35 in 2afd2e4

  	//! Err branch); the registry additionally requires the test body to contain an
  	//! explicit failure-expecting assertion (`is_err`/`expect_err`/`Err(`/
  	//! `should_panic`/…), so a budget-less "consistent OR typed error" tautology
  	//! cannot qualify.
  	//! - [`RedFixtureKind::ProductionFlip`]: a test gated by
  	//! `#[cfg(gauntlet_red_fixture)]` — green on correct production, RED when the
  	//! cfg flips production (or the test's expectation) to the broken variant. Its
  	//! red half is PROVEN in automation by the `gauntlet-red-fixtures-bite` CI lane
  	//! (and `cargo xtask prove-gates-bite`), which builds with the cfg and asserts
  	//! the fixture FAILS. The registry requires the file to contain a
  	//! `gauntlet_red_fixture` branch.
  	//!
  	//! A gate that genuinely blocks today but has no qualified RED fixture yet is
  	//! recorded with `has_blocking_authority: false` and listed in
  	//! [`UNQUALIFIED_BLOCKING_GATES`] as an explicit finding — we do NOT fabricate a
  	//! fixture to launder authority it has not earned.

• The roadmap already calls out perf investigation cliffs and 1.0 exit criteria requiring heavy matrix, fuzz, chaos, loom, and Windows to be green. This is the right moment to harden durability semantics before the API freezes.
  

  batpak/ROADMAP.md

  Lines 111 to 138 in 2afd2e4

  	Surfaced by a full local Criterion soak of the neutral surface on weak hardware
  	(i5-1035G1, 15 W, thermal-limited). Absolutes are a floor; the ratios/shapes are
  	the signal. The cloud `perf.yml` reference baseline is the apples-to-apples
  	comparison and should be captured alongside these. None are 0.10.0 blockers.
  	- [ ] **`projection_run` 20× layout cliff.** `evidence/projection_run` on the
  	same operation: `entity-local` ≈ 46 µs and `all` ≈ 42 µs, but `aos` ≈ 857 µs,
  	`scan` ≈ 847 µs, `tiled` ≈ 902 µs — a ~20× gap. `read_walk/query_with_report`
  	is uniform (~600 µs) across the *same* layouts, so this is projection-path
  	specific. Investigate whether `aos`/`scan`/`tiled` do redundant work in the
  	projection run path or `entity-local`/`all` hit a short-circuit — likely a
  	large free win for the slow layouts. (`benches/evidence_reports.rs`,
  	`src/store/projection/`, `src/store/index/columnar/`.)
  	- [ ] **Frontier wake superlinear at 512 spread waiters.**
  	`frontier_waiter_wake_all` scales fine to 128, then `spread-targets/512` =
  	~626 ms (durable) / ~378 ms (visible) vs `same-target/512` = ~53 ms — a ~12×
  	cliff when many waiters are spread across many distinct targets. Fine for
  	realistic fan-out; a wall for 500+ waiters on distinct targets. Confirm the
  	wake set isn't doing an O(waiters × targets) sweep. (`benches/frontier_waiters.rs`,
  	`src/store/write/` frontier/waiter wake path.)
  	- [ ] **Non-monotonic 100k cold-start dip.** `cold_start/reopen_open_only`
  	per-event throughput: ~220 K/s @10k, **~67 K/s @100k**, ~222 K/s @1M — the
  	100k corpus is per-event slower than *both* neighbors, breaking an otherwise
  	clean linear curve. Confirm it's a corpus-gen artifact vs. a real
  	segment-count / cache threshold. (`benches/cold_start.rs`,
  	`src/store/cold_start/`.)
  	## 3. 1.0.0 track — API stabilization & ergonomics

  
  

  batpak/ROADMAP.md

  Lines 171 to 175 in 2afd2e4

  	published crates; heavy matrix (24-shard mutation, fuzz, chaos, loom,
  	Windows) green on the release candidate; docs current (per-change +
  	repo-wide audit); CHANGELOG migration notes for every break since 0.9.0.
  	## 3b. CI cost policy (durable — the $422 lesson, 2026-07-02)

External prior art / receipts

• SQLite's testing doctrine is the closest model for “small, fast, reliable” storage: independent harnesses, OOM tests, I/O error tests, crash/power-loss tests, fuzzing, malformed database tests, branch coverage, mutation testing, and release checklists. Especially relevant: SQLite crash tests use a special VFS and simulate power loss by reordering/corrupting unsynchronized writes, then reopening and running integrity checks.
  https://www.sqlite.org/testing.html
• FoundationDB leans hard on deterministic simulation to find failures before the real world; it models machines, disks, networks, failures, degraded performance, reboots, and runs daily performance comparisons. The key transferable idea is not “build FDB”; it is: make failures replayable and make performance evidence comparable across time.
  https://apple.github.io/foundationdb/testing.html
• TigerBeetle's VOPR split safety and liveness modes. For BatPak, the analogue is content-durability safety vs namespace-durability safety vs progress/liveness under partially failed storage seams.
  https://tigerbeetle.com/blog/2023-07-06-simulation-testing-for-liveness/
• Jepsen's testing niche is opaque-box/generative tests under failure modes, with histories checked against a model. BatPak already has model-checked histories; this issue pushes the storage model closer to what a real filesystem can do under strain.
  https://jepsen.io/analyses
• Elle shows the value of deriving dependency/anomaly explanations from observed histories rather than relying on local assertions alone. BatPak's recovery classifications should get the same “explain the anomaly” ergonomics for namespace failures.
  https://arxiv.org/abs/2003.10554
• Criterion's own FAQ warns that cloud CI introduces enough benchmark noise that wall-clock results can look changed even when the code is not; it suggests instruction/memory-access style measurements for virtualized environments. This supports using counts/receipts as PR gates and wall-time as trend evidence, not gospel.
  https://bheisler.github.io/criterion.rs/book/faq.html

Proposal

Add a namespace-aware shadow filesystem model behind or alongside SimFs.

Working name:

GAUNT-NAMESPACE-DURABILITY-SHADOW-FS

The model should track two orthogonal truths:

1. content truth: bytes written, bytes synced, durable byte prefix, torn/short reads;
2. namespace truth: directory entries created, renamed, removed, parent-dir synced, and which entries survive a crash.

That means modeling the filesystem as a small logical tree, not just a map of path -> durable_len.

Suggested states per file path:

Absent
TempOnly { temp_path, content_durable_len, name_durable: bool }
FinalOld { old_inode }
FinalNewUnsyncedName { new_inode }
FinalNewDurableName { new_inode }
BothImpossibleTrap { old_inode, new_inode }        // must never be observable after legal recovery
LostNameButContentExists { inode }                // backend bug shape
ResurrectedName { inode }                         // backend bug shape

Do not overfit the enum names. The important thing is that the oracle can express impossible namespace outcomes instead of silently treating names as durable.

Failure families to model

Add fault schedules for namespace edges, independent from content sync drops:

• parent-dir sync dropped after create_new_file;
• parent-dir sync dropped after StagedFile::persist;
• parent-dir sync dropped after rename;
• parent-dir sync dropped after remove_file;
• crash after temp-file content sync but before final name durability;
• crash after final name appears but before parent directory sync;
• crash during compaction swap/rollback with old/new segment names;
• crash during metadata artifact publish: mmap-index, checkpoint, idempotency sidecar, visibility ranges, keyset;
• virtual backend mismatch: MemFs-style backend must not get a stronger namespace contract than RealFs unless it proves it.

New legality oracle

Extend the recovery classification from:

CommittedPrefix | RolledBack | CanonicalRefusal

to include namespace-aware legality checks:

CommittedPrefix { namespace: LegalNamespace }
RolledBack { namespace: LegalNamespace }
CanonicalRefusal { error: typed }
IllegalNamespace { explanation, path, op, seed }

Every reopened state must satisfy:

• no invented/undead user events;
• no lost acknowledged-durable commit on honest disk;
• intact hash chain;
• no impossible directory-entry state;
• no stale temp/final pair that changes cold-start meaning;
• no silent rebuild from a namespace state that should be a typed refusal;
• deterministic replay by (seed, op_plan, namespace_fault_schedule).

Bench/perf side quest: make the non-physics-bounded benches scream honestly

Use this same shadow-fs work to improve perf receipts, not just correctness.

Add an optional BenchEvidenceTape emitted by targeted benches and recovery/perf sentinels:

bench: projection_run/aos
seed: 123
input_shape:
  events: 100000
  layout: aos
storage_work:
  file_opens: 12
  reads: 100003
  bytes_read: 8388608
  writes: 3
  bytes_written: 4096
  fsyncs: 1
  parent_dir_syncs: 0
  renames: 0
  removes: 0
cpu_work:
  allocations: 41
  optional_perf_stat:
    instructions: ...
    cycles: ...
    cache_misses: ...
classification:
  physics_bounded: false
  suspected_extra_work: true

Rules of thumb:

• PR gates should prefer work-unit budgets over cloud wall-clock budgets: allocations, file ops, bytes, fsync count, parent-dir sync count, instruction/cache counts where available.
• Criterion wall time remains useful for local/dev/cloud trend evidence, but cloud CI should not make final claims from wall time alone.
• For the roadmap's projection_run 20× layout cliff, require an evidence tape explaining whether the slow layouts do extra projection/index work or whether the fast layouts short-circuit.
• For the frontier 512 spread-target cliff, require a work-count proof of whether the wake path is accidentally O(waiters × targets).
• For the 100k cold-start dip, require a corpus-shape receipt: segment count, footer count, mmap/checkpoint/scan choice, bytes scanned, file opens, allocations, and recovered event count.

This is the “cure the lower non-physics-bounded benches” rule: do not merely optimize until the stopwatch smiles. First prove where the work went, then cut the extra work. Scalpel before chainsaw.

Acceptance criteria

• A new namespace-aware model exists either as:
  • a new ShadowFs/NamespaceSimFs layer, or
  • an extension of SimFs with explicit namespace durability state.
• The model can replay a failing seed deterministically using BATPAK_SEED or equivalent seed input.
• Parent-dir sync is no longer globally assumed always-durable in the B3/B4 durability proof path; where it remains simplified, the simplification is explicitly scoped and guarded by a test that proves the stronger namespace model covers the omitted behavior elsewhere.
• New recovery tests cover at least:
  • dropped parent-dir sync after staged publish;
  • dropped parent-dir sync after segment create;
  • dropped parent-dir sync after compaction rename;
  • dropped parent-dir sync after remove/reclaim;
  • stale temp plus final artifact after crash;
  • virtual backend namespace mismatch;
  • typed refusal instead of silent rebuild when namespace state is unprovable.
• The new gate has a gate_registry.rs row with anti-vacuous red fixture coverage. Red fixture must plant at least one impossible namespace outcome and prove the checker rejects it.
• Existing recovery oracle remains intact; this issue adds namespace evidence rather than weakening content-durability evidence.
• BenchEvidenceTape or equivalent receipt exists for at least one roadmap perf cliff, preferably projection_run first because it has the biggest cheap-win smell.
• CI placement is cost-aware:
  • small namespace seed set in ci-fast or default PR path if cheap enough;
  • larger seed/fault matrix behind chaos/heavy profile;
  • no new always-on whale job.

Non-goals

• Do not replace the existing recovery oracle.
• Do not claim cross-platform POSIX directory fsync semantics on Windows/non-Unix; model the asymmetry honestly.
• Do not turn benchmark gates into fragile wall-clock astrology.
• Do not hand-wave “SQLite grade” as vibes. The standard here is replayable failure evidence, anti-vacuous gates, typed refusals, and receipts that explain both correctness and performance.

BatPak flavor

Current SimFs proves the bytes. This issue makes the names testify too. The crime scene is not just the corpse length; it is also the door locks, footprints, and whether the supposedly dead temp file is still smoking in the alley.

[heyoub]

Enhancement addendum: compound-failure recovery

The namespace model is already the right cut. Add one further failure family: a second failure while recovering from the first.

SQLite's test doctrine explicitly exercises compound failures, including I/O and allocation failures during recovery from an earlier crash. The transferable standard is not its implementation, but this rule: recovery code does not receive a magical failure-free universe.

Prior art:
https://www.sqlite.org/testing.html

Compound schedules to cover

• I/O error while reopening after a dropped parent-directory sync;
• allocation failure while classifying an uncertain namespace state;
• second crash during compaction swap rollback;
• failure while persisting the artifact/report that records recovery;
• repeated failure after the first injected error instead of one-shot injection;
• short read or checksum failure while inspecting a stale temp/final pair;
• failure while converting an unprovable namespace into the required typed refusal.

Determinism requirement

A compound witness must be replayable from something equivalent to:

(commit, initial_seed, primary_fault_point, recovery_fault_point, operation_plan)

The failure report must distinguish:

• the original storage fault;
• the recovery-path fault;
• the final legality verdict;
• the last point at which durable truth was still provable.

Acceptance sharpening

• at least one RED fixture must plant a recovery routine that incorrectly assumes the second operation cannot fail;
• compound failure may end in a typed canonical refusal, but never invented events, lost acknowledged truth on an honest durability path, or an unexplained namespace state;
• a second crash during recovery must reopen to the same legal classification as an equivalent uninterrupted recovery or a stricter typed refusal;
• keep the small compound seed set cheap, with the wider cross-product in chaos/heavy lanes.

This makes the gauntlet test the firefighter while the building is still on fire, not after the smoke has politely cleared.

[heyoub]

Architecture-convergence addendum

Namespace durability must be modeled against the concentrated artifact roles from #195 and #207, not as isolated file operations. Extend the crash matrix so rename/unlink/parent-sync schedules cover whole transitions involving:

• segment set;
• pending compaction marker;
• idempotency authority;
• visibility authority;
• keyset generation;
• checkpoint/mmap caches;
• future v2 event/clock-order artifacts from GAUNT-EVENT-COMMITMENT-V2: separate payload identity from event identity and prove exact chain topology #191/GAUNT-CLOCK-ORDER-TAXONOMY: separate time source, journal commit point, stream position, and causal time #208.

The reference result must classify each surviving file as SourceTruth, MutableAuthority, DerivedCache, SecretAuthority, or diagnostic material. A namespace schedule is legal only when the surviving set represents one old generation, one new generation, or a typed refusal/recovery state. A cache may disappear and rebuild; an authority may not disappear and become a default.

Coordinate the StoreFs conformance corpus with #179 so external backends prove the same parent-directory, rename, replacement, unlink, and crash rules. Also feed namespace-operation facts into the unified Repo IR under #211 so production filesystem contact and proof ownership have one inventory rather than parallel build/integrity lists.

[heyoub]

Concrete current-main defect: namespace observation collapses I/O failure into absence

Why now

The #189/#226 curing batch correctly promoted compaction recovery into an old-or-new namespace transaction, and current main now classifies a pending transaction from the marker, store.meta, the authority image, and the surviving segment names.

That new recovery path also exposes a narrower fail-open hole: several authority-bearing path observations treat every metadata error as “the file is absent.”

At current main (6c4a46f), MarkerLayout::probe records existence with fs.metadata(...).is_ok():

• final_exists, src_exists, and staged_exists

The committed roll-forward path then builds its retirement set by filtering each non-merged source through another fs.metadata(path).is_ok(), removes only the paths that survived that filter, syncs the parent, clears the pending marker, and returns RolledForward:

• existing_non_merged_sources
• repair_pending_compaction

That is safe only when metadata has exactly two outcomes: present or NotFound. StoreFs does not promise that. A custom backend, a remote/virtual store, or the host filesystem can return PermissionDenied, Interrupted, transport failure, timeout-shaped Other, or another non-absence error.

Rust’s own API names this distinction explicitly: Path::exists coerces errors to false, while Path::try_exists returns an error when presence cannot be confirmed. The standard-library docs also warn that existence checks do not eliminate TOCTOU races:

• https://doc.rust-lang.org/std/path/struct.Path.html#method.try_exists

SQLite’s VFS makes the same separation at the storage boundary: xAccess returns an I/O status separately from the resulting existence/access fact. A failed observation is not a negative fact:

• https://www.sqlite.org/c3ref/vfs.html

Concrete failure sequence

1. A compaction transaction is durably committed: marker and store.meta agree on the compaction token and merged segment.
2. One old non-merged source still exists and must be retired before the marker disappears.
3. During recovery, StoreFs::metadata(source) returns a non-NotFound error, for example an injected backend timeout or PermissionDenied.
4. existing_non_merged_sources silently omits that path because .is_ok() is false.
5. Recovery removes the other visible sources, syncs, clears the pending marker, and returns RolledForward.
6. The transient observation fault disappears. The stale source is visible again beside the committed replacement, but the transaction marker that explained and authorized cleanup is gone.

The result is neither a proven NEW generation nor a typed refusal. It is a stale namespace with its recovery receipt burned behind it.

The same semantic collapse exists in the production symlink guard:

match std::fs::symlink_metadata(path) {
    Ok(meta) if meta.file_type().is_symlink() => Err(...),
    Ok(_) | Err(_) => Ok(()),
}

• reject_symlink_leaf

Here a permission or lookup failure is treated as proof that the destination is safe. That contradicts the trait’s stated fail-closed symlink contract.

BatPak already implements the correct law nearby. remove_file_if_present maps only NotFound to absence and propagates every other error, with a planted PermissionDenied test proving that a failed removal cannot report “already absent”:

• StoreFs::remove_file_if_present
• remove_file_if_present_propagates_non_not_found_errors

Downstream dogfood trigger

This is no longer a POSIX-only thought experiment.

• batpak-saas now supplies a Durable Object SQL-backed StoreFs, so filesystem facts cross a JS/SQL/wasm boundary with error classes that do not naturally collapse into POSIX NotFound: PR #10.
• Its two SQL filesystem implementations independently demonstrated that one incorrectly handled path edge can destroy source data while the broader adapter appears healthy: PR #9, PR #18.
• Its repository judge also had to replace lexical path assumptions with canonical containment because an “exists” result was weaker than the claim being certified: PR #17.
• Its DR and journal-core work now depend directly on BatPak’s authority and restart semantics: PR #16, PR #14.

The transferable lesson is simple: unknown is not absent, and observation failure must never advance or erase recovery authority.

Proposal: one tri-state path-fact primitive

Centralize the distinction instead of fixing each .is_ok() by hand.

Illustrative shape:

enum PathFact<T> {
    Present(T),
    Absent,
}

fn observe_metadata(
    fs: &dyn StoreFs,
    path: &Path,
) -> io::Result<PathFact<FileStat>> {
    match fs.metadata(path) {
        Ok(stat) => Ok(PathFact::Present(stat)),
        Err(error) if error.kind() == io::ErrorKind::NotFound => Ok(PathFact::Absent),
        Err(error) => Err(error),
    }
}

Exact spelling is maintainer choice. The contract is not:

• Present means positively observed;
• Absent means positively observed NotFound;
• every other result remains an error/unknown and must propagate.

Use that one source for authority/recovery paths, including:

• compaction MarkerLayout probing;
• source-retirement discovery;
• symlink-leaf admission;
• any future artifact/authority generation inspection.

MarkerLayout::probe and existing_non_merged_sources should become fallible. A pending marker must remain intact whenever a required namespace fact is unknown.

This does not solve the separate TOCTOU class by itself. For security-sensitive RealFs writes, dirfd-relative/no-follow mechanisms such as Linux openat2(..., RESOLVE_NO_SYMLINKS) may later be appropriate, but the immediate correctness fix is smaller: stop converting observation failure into evidence.

Acceptance criteria

• Only io::ErrorKind::NotFound may become Absent in recovery or authority-bearing path inspection.
• MarkerLayout::probe returns a typed/fallible result; no boolean is synthesized from an arbitrary metadata error.
• existing_non_merged_sources returns a typed/fallible result; a failed observation cannot omit a retirement obligation.
• A committed recovery encountering an unknown source-path fact returns a typed error/refusal and leaves the pending marker present.
• Reopening after the observation fault clears resumes the same transaction and reaches the canonical NEW generation.
• reject_symlink_leaf treats NotFound as an acceptable not-yet-created leaf, but propagates PermissionDenied, Interrupted, and backend errors.
• The public StoreFs contract and conformance report distinguish Absent from ObservationFailed; an unsupported/faulting backend never receives a pass.
• No new public API is required unless an external backend genuinely needs the tri-state helper; a private/default helper is preferable to another method every implementation must copy.

Suggested RED fixtures and validation gates

Deterministic recovery cases

• committed_recovery_metadata_permission_denied_keeps_marker
• committed_recovery_metadata_interrupted_keeps_marker
• transient_metadata_fault_then_reopen_finishes_same_transaction
• unknown_staged_path_fact_never_becomes_replacement_missing
• symlink_guard_metadata_failure_is_not_safe

Use ShadowFs/hostile controls to inject the error on one exact source path after the committed marker/meta state exists. The oracle must prove:

• marker retained after the failed inspection;
• no source retired based on an unknown fact;
• no replacement/source overlap accepted as canonical;
• a later clean reopen converges or returns a stricter typed refusal.

State-machine property

Add the law:

Unknown(path) != Absent(path)

marker_absent
  => every retirement obligation was either
       (a) successfully removed and parent-synced, or
       (b) positively observed NotFound

Generate source counts, faulted path position, metadata error kind, and first/second recovery attempts. Include compound schedules from the existing #177 addendum.

Mutation kills

Kill mutants that:

• replace tri-state observation with .is_ok() or .is_err();
• map every metadata error to Absent;
• clear the marker after an unknown path result;
• drop error propagation from MarkerLayout::probe;
• restore Ok(_) | Err(_) => Ok(()) in the symlink guard.

Structural guard

Add an AST/semantic lint scoped to crash, recovery, authority, and admission modules that rejects:

• metadata(...).is_ok() / .is_err() as an existence decision;
• wildcard error arms that produce Absent, false, or success;
• Path::exists() in authority-bearing code.

Allow explicit test assertions and diagnostic-only code through anchored exceptions, not filename folklore.

Conformance corpus

Add an addressable external-backend case requiring a hostile backend to return a non-NotFound metadata error during recovery inspection. The report must name the exact case and must never classify unsupported injection as a pass.

Benchmark methodology

This cure should not add a syscall. It classifies the result already being obtained.

For the recovery/compaction bench lane, record work rather than gating on cloud wall time:

• metadata probes;
• source paths classified;
• removes;
• renames;
• parent-directory syncs;
• retries/reopens;
• allocations.

Require linear probe count in the number of source paths and no accidental duplicate probing. Keep wall time as trend evidence only. The desired performance result is pleasantly boring: identical physical work on the green path, honest refusal on the foggy path.

Scope decision

Keep this under #177 rather than opening another issue. #177 owns namespace truth and compound recovery; this is a concrete current-main implementation arm of that law. #179 should consume the external-backend conformance case, and #211 may inventory the structural rule, but neither should become a second semantic owner.