Skip to content

manifest-shape only validates manifest→tree; a new template file missing from the manifest passes CI silently #81

Description

@psaboia

Summary

The manifest-shape unit fixture enforces the manifest's internal invariants and the manifest→tree direction (every listed path exists in the tree, no double-ownership, every SUBSTITUTE entry carries {{REPO_NAME}}). It does not enforce tree→manifest: a new file added under a template-owned directory but never listed in scripts/lib/template-manifest.sh passes CI, while being silently invisible to every derived project — update-from-template.sh and adopt.sh both assemble their file sets from the manifest.

Evidence this bites in practice — three occurrences, each caught by a human

  1. PR Discipline gates + Verification Gate + PostToolUse advisory hook #1 shipped wiki/agents/discipline-gates.md and verification-gate.md without updating the (then two parallel) sync lists; caught in human review. Documented as Lesson-Parallel-File-Drift.
  2. Issue adopt.sh: template-manifest.sh is missing from its own manifest — adopted hosts get broken sync tooling #74 (fixed in PR manifest: ship template-manifest.sh itself and Edge-Types.md.template (fixes #74, #75) #76): the manifest didn't list itself, so adopted hosts received the three sync scripts without the library they source — broken sync tooling in the field, requiring the legacy self-heal that PR manifest: ship template-manifest.sh itself and Edge-Types.md.template (fixes #74, #75) #76 ships. The strongest instance of the class: not even the manifest's own file was covered by any check. (adopt.sh: wiki/Edge-Types.md.template not shipped — adopted wikis get a SCHEMA full of dead Edge-Types links #75, the missing wiki/Edge-Types.md.template, rode the same gap.)
  3. PR Add /ask slash command for agent-comms cross-agent consultation #80 ships .claude/commands/ask.md without a TEMPLATE_OVERLAY_CLAUDE entry; CI green, flagged in review (comment).

The Test-Harness wiki page has carried exactly this smoke-test addition as a "filed as a follow-up; not yet implemented" note since before the manifest consolidation.

Proposed check

Extend manifest-shape (or add a sibling smoke assertion) with an allowlist-based sweep: every file in the template tree under the template-owned surfaces — .claude/commands/, .claude/skills/, .cursor/rules/, wiki/agents/, scripts/lib/, scripts/wiki-write-protocol/ — must appear in exactly one of: the manifest's sync arrays, TEMPLATE_HOST_OWNED, TEMPLATE_ONE_SHOT, or an explicit documented-exclusions list.

Why an explicit exclusions list (see #62)

Not every template file should be in the manifest — but today "deliberately excluded" and "forgotten" are indistinguishable, both reading as mere absence. Issue #62 is the live example: scripts/kg/ exists in the template, is hinted at by CLAUDE.md, and is absent from the manifest by decision (planned as an opt-in feature) — yet nothing records that decision where a check could honor it. The exclusions list makes silence mean "covered": every deliberately-unsynced path gets named in one greppable place, and the next scripts/kg/-style decision is recorded rather than implied. Resolving this issue would also give #62 its natural home (an EXCLUDED entry with a reason, or promotion to a feature).

Red-first validation: add the check, confirm it fails on a synthetic unlisted file (and would have failed on the PR #80 tree without the manifest line), then confirm it passes once the entry exists. A check that has never been red proves nothing — per .claude/rules/observe-the-failure.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions