Aggregate catch log across multicheck sessions and operators. Format spec is in METRICS.md. New session catches go in <target-project>/multicheck/metrics.md first; operators may PR them upstream into this file.
This file is append-only. New sessions add rows at the end. Existing rows are never edited or deleted (except for typo fixes by maintainers).
If you are an operator who wants to add your session's catches, see the "Sharing back upstream" section in METRICS.md.
| Type | Meaning |
|---|---|
pre-emptive-self-correction |
Builder caught its own mistake before the reviewer challenged (highest-value behavior) |
process-violation |
--no-verify, --force, undisclosed scope expansion, missing tagged disclosure, anti-vocabulary, middle-insert |
technical-bug |
Code is wrong; reviewer caught a real defect |
goal-divergence |
Work technically clean but doesn't advance the active [G-NNN] goal |
near-miss |
Almost shipped wrong; caught pre-commit or pre-merge |
wrong-file |
Builder edited a file production doesn't import from |
test-gap |
Test passes but asserts the wrong thing |
slice-impurity |
Stacked PR commit contains files outside declared scope |
bonus-structural-verification |
Reviewer verified an invariant the builder didn't claim |
recommendation-corrected |
Reviewer caught its own draft fix that would have created inconsistency |
baseline-pre-existing-failure |
Pre-flight on origin/main surfaced a failure before builder began |
heredoc-race-avoided |
Heredoc append succeeded where Edit/Write would have raced |
other |
Other; explained in Description |
Severity: low | medium | high | critical. Caught by: builder | reviewer | operator | hook (as of v0.5.0 — mechanical catches by hooks/pre-push.sh or similar).
Model tracking (as of v0.5.1): new rows should include Builder model and Reviewer model columns. Existing rows may be backfilled where the pairing is known. The reference session pairing for all rows dated 2026-04-06 through 2026-04-09 is codex-gpt-5 builder + claude-opus-4.6 reviewer unless otherwise noted.
| Date | Catch type | Severity | Caught by | Description | Evidence |
|---|---|---|---|---|---|
| 2026-04-06 | pre-emptive-self-correction | medium | builder | builder corrected own [S-004] "compile-time pre-existing @claims/errors" misdiagnosis — was actually runtime + wrong file (pipeline.schema.ts not pipeline-engine.test.ts); caught self before reviewer challenged |
[S-007] |
| 2026-04-06 | pre-emptive-self-correction | medium | builder | builder surfaced runtime-meetings absence as a #608 blocker, created #614 mid-session, posted on #606 + #608 — caught self before reviewer raised the dependency | [S-008] |
| 2026-04-06 | pre-emptive-self-correction | high | builder | builder acknowledged the [S-010] --no-verify bypass was a process mistake, re-verified failure on origin/main, corrected the file citation; posted before reviewer's planned challenge could land |
[S-013] |
| 2026-04-06 | pre-emptive-self-correction | medium | builder | builder executed baseline remediation broader than reviewer's [R-006] scope estimate, surfacing the work proactively | [S-014] |
| 2026-04-06 | process-violation | high | reviewer | builder used git commit --no-verify on commit 58327df claiming "unrelated baseline failure"; reviewer reproduced the failure independently, confirmed it was real, but flagged the bypass as a CLAUDE.md violation regardless of root cause; two-axis verdict technical-accept process-reject |
[R-006], commit 58327df |
| 2026-04-06 | slice-impurity | medium | reviewer | #607 silently expanded from 5 in-scope files (per details.md) to 7 committed files; jest.config.cjs and tsconfig.jest.json added without prior disclosure | grep against details.md vs git show 5f90949 --name-only |
| 2026-04-06 | technical-bug | medium | reviewer | builder cited test file pipeline-engine.test.ts for the failure; reviewer grepped and got zero matches; the actual file was pipeline.schema.ts lines 60/69/78/82/89 |
git grep "z.uuid" pipeline-engine.test.ts returned 0; grep -rn found the real location |
| 2026-04-06 | goal-divergence | high | reviewer | builder proposed cutting #608 onboarding read-path before #609 booking write-path; ordering would have regressed onboarding because legacy callers still target the old table | reviewer inspected consultation.repository.ts and onboarding.repository.ts directly; corrected order is #607 → #614 → #609 → #608 |
| 2026-04-06 | bonus-structural-verification | medium | reviewer | builder claimed meetings.case_id is non-null per the migration; reviewer verified at BOTH the migration DDL (line 4) AND the Drizzle TS schema (lines 12-13) — stronger invariant than the builder claimed |
[R-008] |
| 2026-04-06 | recommendation-corrected | high | reviewer | reviewer drafted a fix recommendation in [R-006] that would have only patched pipeline.schema.ts; before posting, reviewer grepped wider and found 3 other packages using the same z.uuid() pattern — recommendation would have created cross-package syntax inconsistency |
[R-009] (corrected version) |
| 2026-04-06 | baseline-pre-existing-failure | high | reviewer | post-hoc finding (not caught at pre-flight): the pipeline.schema.ts zod failure was pre-existing on origin/main from before session start. If reviewer had run mandatory Phase 0 baseline health check on origin/main, the failure would have surfaced as someone else's problem and the entire --no-verify bypass conversation would not have happened |
reference session post-mortem; led to the baseline health check rule in REVIEWER.md |
| 2026-04-06 | heredoc-race-avoided | low | reviewer | reviewer hit "file modified since read" 3 times using Edit/Write against agentchat.md while watchdog was running; switched to cat >> file <<'EOF' heredoc append pattern; subsequent ~10 writes had zero races |
reviewer meta-review session 1 |
| 2026-04-06 | process-violation | medium | reviewer | builder tooling middle-inserted entries [S-022..S-025] at line ~401 with duplicate S-023 and S-025 tags; reviewer reconstructed correct order from offsets but flagged the violation; led to "append-only / monotonic / no middle inserts" hard rule | reference session file disorder, ENTRY_RE offset detection |
| 2026-04-06 | other | low | operator | watchdog auto-tag R-AUTO did not match the watchdog's own ENTRY_RE regex [A-Z]-\d+, so R-AUTO entries were unparseable by the watchdog itself; tooling bug surfaced by reviewer's session 2 meta-review |
scripts/agentchat_watchdog.py ENTRY_RE |
| 2026-04-07 | slice-impurity | medium | reviewer | verified 5-PR cascade rebase chain by git diff --name-only A..B between every adjacent commit (#617 → #618 → #619 → #620 → #621); each diff matched declared scope file-for-file; no contamination found, but the recipe itself caught a pre-rebase impurity in the prior round |
reviewer's slice-purity verification recipe |
| 2026-04-07 | process-violation | high | reviewer | builder made TWO substantive changes (deprecated z.uuid() patch round + test file fix round) without posting tagged [S-029]/[S-030] disclosures; reviewer reconstructed timeline from git log and demanded backfill before allowing the audit trail to freeze; led to "missing tagged disclosure" being added to the process violations list in REVIEWER.md |
[R-021] |
| 2026-04-07 | other | medium | reviewer | reviewer almost incorrectly accused builder of false claims because git grep "z.uuid" fix/zod-v4-uuid-baseline returned the original (uncommitted-fix) string; should have used filesystem grep -rn for staged-but-uncommitted changes; caught by reviewer self-check before posting |
reference session — added "Working-tree grep vs git ref grep" gotcha to REVIEWER.md |
| 2026-04-07 | technical-bug | high | reviewer | builder wrote z.uuid() fix, then wrote a revert, but left invitation-routes.test.ts:27 inconsistent with the canonical pattern; caught by reviewer (different model than builder); same-model pairing would have missed it because the builder's blind spot was the same in both writes |
invitation-routes.test.ts:27 |
| 2026-04-07 | bonus-structural-verification | medium | reviewer | builder claimed case_id was non-null per the migration; reviewer went beyond the ask and verified at BOTH the migration DDL AND the Drizzle TS schema layers — stronger guarantee than what was claimed |
[R-008] (also recorded earlier as a separate row from session 1) |
| 2026-04-07 | other | high | reviewer | CI trigger gap on stacked PR #618: GitHub Actions silently filtered the workflow because pull_request: branches: [main] + base-change-doesn't-fire-synchronize; neither builder nor CI itself flagged it because both were looking at the wrong angle of the event stream; caught by reviewer checking the CI from outside the builder's perspective |
#618 GitHub Actions, known stacked-PR pattern |
| 2026-04-07 | other | medium | reviewer | feature_calendar_service.yml was not registered in CONTRACT_INDEX.yml on origin/main; PR #601 added other contracts without registering the calendar contracts; caught by reviewer's independent grep against origin/main, not by reading the working tree |
grep CONTRACT_INDEX.yml on origin/main |
| 2026-04-07 | other | critical | builder | builder diagnosed mid-session protocol drift caused by rules living only in the running ledger (specs/agentchat.md), not in any stable project-memory file; created Active Protocol section in specs/details.md as local fix; led to upstream protocol-summary template + 3-file refresh in Phase 0, later refined to role-split (claude-md.md / agents-md.md) at multicheck commit 529c9b5 |
claims-monorepo [S-046], multicheck commits db0c8bf and 529c9b5 |
| 2026-04-07 | other | high | operator | reviewer self-assessment: spent [R-018], [R-022], [R-025], [R-026] flagging symptoms of stable-context-file gap (duplicate tags, middle inserts, format drift) without identifying the root cause; the builder caught the root cause in [S-046] before the reviewer did; reviewer was treating symptoms not the disease | reference session R-FINAL self-assessment |
| 2026-04-07 | process-violation | medium | reviewer | builder used non-canonical heading format [S-NNN][builder][datestamp] (vs the canonical ### [S-NNN] HH:MM UTC — #ticket) for entries S-029 through S-034; format drift wasn't detected immediately because the watchdog silently skipped unparseable entries instead of escalating; led to "fail-loud watchdog" recommendation for Phase 2 |
S-029 to S-034 format drift |
| 2026-04-07 | near-miss | high | reviewer | silent condition drift during cascade rebase in onboarding.repository.ts: WHERE clause simplified from 3 conditions to 2 during conflict resolution, which would have counted cancelled and no-show meetings as "consultation exists" and broken onboarding readiness gating in production; slice-purity check (git diff --name-only) showed the file in scope with expected size delta but did NOT detect the content regression; caught by pre-promotion audit running git diff <prior>..<current> -- <file>; exposes a gap in the existing slice-purity verification recipe — rule queued in PENDING.md item #1 for fold-in after data-collection freeze |
claims-monorepo pre-promotion audit, calendar consolidation cascade, PENDING.md #1 |
| 2026-04-08 | near-miss | high | reviewer | stale branch base went undetected through every stage 0 gate: builder git checkout-ed an existing calendar/cal-11-04-11-07-meeting-lifecycle-milestone branch created weeks earlier during G-002 without verifying its base against current origin/main; git merge-base HEAD origin/main = 325095ea (weeks-old) vs git rev-parse origin/main = 10b7065c (current); PR diff contained ~20+ .changeset/* files plus already-merged G-002 work; 4 hours of builder work had to be thrown out and redone on a fresh branch from current origin/main; targeted tests + husky hook + gotcha checklist + prior reviewer findings all passed because none of them check branch topology; caught only at stage 3 pr.md audit when the PR diff was actually computed; exposes a foundational invariant gap, not a recipe refinement — rule queued in PENDING.md item #2 for fold-in after data-collection freeze |
claims-monorepo G-003 #610 branch, PENDING.md #2 |
| 2026-04-08 | test-gap | high | reviewer | reviewer's [R-034] audit verified the right behavior against the wrong file target: audit checked consultation.repository.ts for the createdByUserId / legacyConsultationWrite / schema validation findings, but that file had been deleted/renamed to meetingsRepository between the stale branch base and current main; audit verdict was technically correct against the file that existed at the branch base, but completely irrelevant to current main where the file no longer existed; caught by the builder when a cherry-pick failed on the missing file during rebuild; secondary finding from the stale-branch incident — had the branch base been verified at slice start, the wrong-file-target condition would never have existed because the rename would have forced a rebase before any audit began |
[R-034], claims-monorepo G-003 #610 rebuild, PENDING.md #2 (root cause) |
| 2026-04-08 | technical-bug | high | reviewer | cross-layer enum drift in claims-monorepo #610: SQL migration added consultation_no_show enum value but the corresponding TypeScript union type in booking.types.ts was not updated; would have caused a runtime type mismatch when the DB returned the new status value; NOT caught by pr.md slice-purity/topology/content gates; NOT caught by the primary internal reviewer; caught by EXTERNAL ruflo/claude-flow agent-code-review-swarm running as a third reviewer layer on the final PR head; the swarm's "architecture agent" persona prompts for layer violations and SOLID/DRY consistency, which primed the underlying LLM to spot the enum/type drift that the other two reviewers missed; empirical validation of the multi-reviewer asymmetry thesis at N=3 (pr.md + primary reviewer + external swarm) — queued in PENDING.md items #3 (cross-layer value consistency recipe) and #4 (specialist-persona sweep), plus meta-observations M3 (multi-reviewer asymmetry is multicheck's core thesis) and M4 (external swarm as optional third reviewer layer) |
claims-monorepo #610, PR #636, ruflo/claude-flow agent-code-review-swarm |
| 2026-04-08 | pre-emptive-self-correction | high | reviewer | R-035 topology miss self-correction: R-034 content audit ran git status --short (working-tree-vs-HEAD) instead of the topology commands (git merge-base HEAD origin/main + git rev-parse origin/main); the stale branch base from PENDING #2 was caught by pr.md stage 3 at commit f15c4ae8, NOT by R-034; reviewer self-corrected at R-035 with full incident analysis + verbatim rule text mirrored into both specs/details.md Active Protocol AND docs/review-process.md Stage 0; reviewer was verifying behavior but not the branch-base context that behavior depended on — empirical validation of Meta M1 "verify one level above the claim" (issue #5) |
claims-monorepo [R-035], specs/details.md:70, docs/review-process.md:51 |
| 2026-04-08 | technical-bug | medium | reviewer | R-036 stipulation on claims-monorepo #610 head aa6cd01e: missing regression test coverage on the rebooking invariant — meetings-repository.test.ts::existsForOrg tests had assertions of the form "mockWhere was called with some and() expression" but NO assertion of the actual WHERE clause structure; a future refactor could delete the notInArray(status, REBOOKABLE_CONSULTATION_STATUSES) call and tests would still pass; builder addressed in correction round 02202f1f by adding a targeted unit test using new PgDialect().sqlToQuery(whereArg) then asserting whereQuery.sql contains "meetings"."status" not in AND whereQuery.params contains exactly ['consultation_cancelled','consultation_no_show','cancelled']; right shape for a cross-layer invariant guard |
claims-monorepo R-036, meetings-repository.test.ts:161-174 |
| 2026-04-08 | process-violation | medium | reviewer | R-041 multi-author drift capture on claims-monorepo PR #636: Robert Fall (human reviewer with push access) pushed 3 commits directly to the slice branch — 7a7b76f0 (review-fix duplicate of builder's S-117), 4350a2e8 (.gitignore for .claude-flow), bab1e029 (unrelated property perf-test CI flake fix) — WITHOUT any tagged builder entry in specs/agentchat.md acknowledging them; 2 of the 3 commits were out-of-slice scope bleed per details.md; slice file count grew from declared 13 to 16 actual; captured non-blocking per operator instruction "if he made the changes and merged we are good; otherwise we capture"; exposes a protocol gap: the state vocabulary doesn't have an external-commit-disclosure entry for multi-author slices — recommend new PENDING issue for multi-committer discipline |
claims-monorepo R-041, PR #636, commits 7a7b76f0 + 4350a2e8 + bab1e029 |
| 2026-04-08 | other | low | reviewer | R-038 pr.md §11b Prettier extension allow-list fix: first production run of the patched §11b gate command on claims-monorepo #610 at commit 02202f1f crashed with [error] No parser could be inferred for file ".../0028_meetings_consultation_rebooking.sql" because stock Prettier has no SQL parser (and this repo doesn't install prettier-plugin-sql); patched pr.md §11b to use explicit extension allow-list (ts,tsx,js,jsx,mjs,cjs,json,md,mdx,yml,yaml,css,scss,html) + mandatory disclosure of non-formattable files in STIPULATIONS section; meta-lesson: gates that crash on edge cases train operators to dismiss their output — better to filter and disclose than to crash |
claims-monorepo [R-038], specs/pr.md §11b |
| 2026-04-09 | process-violation | high | reviewer | R-045 proceed-to-code gate skip on claims-monorepo #611: R-044 explicitly said "I will ack the chosen option in R-045 and clear the proceed-to-code block. At that point the builder may post STATE: building and start writing code"; builder posted [S-125] decision packet at 17:06 UTC, then posted [S-126] STATE: building at 17:17 UTC — 11 minutes later, WITHOUT waiting for R-045 reviewer ack; first skipped proceed-to-code gate in the session; decision substance was correct (Option B: case_events only, action_runs deferred) and resulting code was clean, but the gate was procedurally skipped; captured non-blocking per operator instruction; reinforces the lesson that written gates lose to completion drive without mechanical enforcement — validates Meta M2 (markdown ceiling) and the need for the gate-file template hook (referenced in claims-monorepo upstream-handoff Item #4); recommend: a pre-commit hook reading specs/agentchat.md for the latest reviewer ack, shipping as an opt-in template per the specflow agent's correct pushback against making it a core feature |
claims-monorepo [R-045], [S-125]→[S-126] sequence |
| 2026-04-09 | technical-bug | high | reviewer | R-046 §11f performance-hat catch on claims-monorepo #611 at head a3276676: the createApp() factory refactor in src/index.ts moved the password-reset cleanup setInterval INSIDE the factory function body, meaning every call to createApp() allocated a fresh interval — once in production, N times in tests — producing Jest worker-leak warnings in the new mount/auth integration test; builder proactively self-caught in [S-140] BEFORE R-046 posted the finding, and committed the fix at e761f6f1 which hoisted the interval back to module scope and added .unref?.() so it doesn't hold the event loop alive; FIRST PRODUCTION CATCH via the new pr.md §11f specialist-persona sweep (performance hat) — empirical validation that the five-hat discipline (issue #3) catches defects the first four review sections don't surface; also empirical validation of the harness-triage framework: builder chose option (c) product-code refactor rather than reflexive retries/timeouts, which is the correct option per the framework |
claims-monorepo R-046, [S-140], e761f6f1 |
| 2026-04-09 | near-miss | high | reviewer | Three claims-monorepo #611 builder reconnaissance failures during the first sub-slice: (1) pino-http expected a fuller logger shape than the test provided → builder reflexively stubbed pino-http at the test boundary; (2) importing src/index.ts pulled in better-auth/node ESM which Jest couldn't parse → builder reflexively stubbed better-auth at the test boundary; (3) the draft swarm review surfaced that stubs alone couldn't prove the real auth boundary → builder had to refactor src/index.ts to expose a createApp() factory that should have been declared upfront; all three are symptoms of the same root cause — not tracing the import graph before writing test code; ALL THREE would have been prevented by pre-flight Q7 reconnaissance; empirical reference incident for the Q7 fold-in proposed in claims-monorepo/specs/upstream-handoff-2026-04-09.md Item #2 and subsequently folded into multicheck REVIEWER.md v0.5.0 as the "recon is the grounding for Q1–Q6" rule; also relevant to issue #15 (Mythos M-8: re-document Q7 purpose for Mythos era) — the #611 incidents are Codex-class (lower-capability) examples where Q7 forces the work upfront |
claims-monorepo #611 session, [S-136] app-factory refactor, specs/upstream-handoff-2026-04-09.md Item #2, multicheck REVIEWER.md v0.5.0 |
| 2026-04-09 | pre-emptive-self-correction | medium | builder | claims-monorepo builder [S-130] Prettier stop self-correction during #611 correction round commit: first commit attempt blocked at the Prettier stage of .husky/pre-commit, builder did NOT use --no-verify (R-006 lesson internalized), ran npx prettier --write on the flagged files, restaged, retried → hook gate passed; demonstrates the no-bypass discipline has become habitual after one R-006 incident — builder learning curve is visible in the metrics |
claims-monorepo [S-130], [S-131] |
| 2026-04-09 | pre-emptive-self-correction | medium | builder | claims-monorepo builder [S-134] proactive snapshot scope-drift catch: during the first production run of the pr.md stage 3 draft audit on PR #643, builder caught that the three-dot diff included an auto-generated Jest snapshot file packages/apps/claims-server/tests/functional/meetings/__snapshots__/cal_00_01.test.ts.snap that was NOT in specs/details.md's declared in-scope list; builder corrected details.md BEFORE the reviewer could flag it; FIRST PRODUCTION CATCH via the pr.md stage 3 self-audit pattern — validates that builder-side stage 3 catches drift at the cheap moment rather than at reviewer time |
claims-monorepo [S-134], PR #643 |
| 2026-04-09 | recommendation-corrected | high | reviewer | R-049 reviewer self-correction on action_runs framing: in R-047 OPEN ITEMS and the sub-slice-2 prep briefing, reviewer framed the scope question as "Option A: implement action_runs in-slice / Option B: defer action_runs" — treating action_runs as a real runtime entity waiting to be built; builder's [S-145] reconnaissance and the operator's forwarded refinement revealed that action_runs was never implemented as a distinct table in packages/, only aspirational wording from docs/specs/calendar.md and calendar_epicstory.md spec documents; the real audit surface is case_events (already exists with appendEvent() + nextSequenceNum()), and the gap is that consultation booking mutations don't currently append case_events; operator authorized dropping action_runs from #611 scope via direct ticket body amendment (3 references removed: In Scope, I-SCH-004 invariant, Tests list); mirror failure of R-040 (cross-layer enum drift caught by external swarm) — same shape: treating a spec-document token as a runtime entity without grep-verifying against packages/ at prep time; validates that §11e discipline (issue #2) applies to reviewer prep, not just reviewer verdicts; meta-lesson: Meta M1 "verify one level above the claim" applies to the REVIEWER's own mental model, not just to builder claims |
claims-monorepo [R-049], #611 ticket amendment, [S-145] recon |
| 2026-04-09 | other | medium | operator | Specflow v0.5.1 toolchain upgrade caught a latent bug in the v0.5.0 installer: the jq unique_by(.hooks[0].command) dedupe treated Write→check-pipeline-compliance.sh and Edit→check-pipeline-compliance.sh as duplicates (same command string) and silently dropped the Edit matcher; first install wrote the hook FILE correctly (verified by npx @colmbyrne/specflow verify section 13 showing check-pipeline-compliance.sh ✅ current) but left .claude/settings.json with only 3 matchers instead of 4, producing a settings.json Edit→check-pipeline-compliance.sh ❌ MISSING CRITICAL finding at the next verify run; fixed in v0.5.1 by changing the dedupe key to [.matcher, .hooks[0].command] so Write and Edit are distinct entries; validates that the v0.5.0 section 13 version check is doing its job — catching its own installer's latent bugs between versions |
specflow v0.5.0 → v0.5.1 upgrade, npx @colmbyrne/specflow verify section 13 |
| 2026-04-09 | other | medium | reviewer | guardrails-starter kit distilled from this session's + prior sessions' tooling patterns and published as a standalone MIT-licensed public repo at Hulupeep/guardrails-starter; 9 files (README + PROMPT + INSTRUCTIONS + 5 content files + LICENSE) organized around a 4-tier product-protective priority hierarchy (Tier 1 = preview deploys + CI smoke + branch freshness + PR template); explicit "trust-reduction system / manage by receipts" framing for a non-coding PM audience; surfaces @colmbyrne/specflow as the primary hook install path with manual copy-paste as fallback; trigger was operator feedback on claims-monorepo docs/claims-alert-tooling-for-product.md noting it was "correct but wrong emphasis" (too engineering-centric); resulted in PM-doc reframe + repo extraction; meta-observation: operator feedback loops produce better artifacts than solo reviewer opinion — the reviewer had drafted the original doc with engineering-first framing and only produced the product-protective version after operator pushback; mirror of Meta M3 (multi-reviewer asymmetry) at the human-reviewer level |
Hulupeep/guardrails-starter public repo, commits 6b5d207 + 03126dc + d192030 |
| 2026-04-13 | process-violation | high | reviewer | reviewer's §11f specialist-persona sweep was procedurally compliant (all five hats named, one finding-or-clean line per hat) but the security hat wrote "clean" without actually verifying authorization on a state-mutating endpoint; reviewer filled in the form without running the check; same "organic coverage" pattern applied to per-ticket gotcha checklist and first-checks — referenced in verdicts but never walked through item-by-item with pass/fail per item; "organic coverage creates the illusion of thoroughness"; led to mandatory structured REVIEWER FIRST CHECKS output format in REVIEWER.md with PASS/FAIL/SKIP per item from details.md | claims-monorepo R-050, G-003 session |
| 2026-04-13 | recommendation-corrected | high | reviewer | R-051 reviewer self-correction on claims-monorepo PR #643 meetings.routes.ts: after HITL flagged "requireAuth only, no requireRole()" on two handlers, reviewer initially accepted the framing and wrote R-050 labeling it a HIGH severity security question; operator challenged the framing ("i don't really understand if that is really a security issue"); reviewer then READ the domain functions (transitionConsultationState in consultation-lifecycle.service.ts:123-137 and resolveMeetingTypeDefaults in meeting-type-config.service.ts:385-391) and verified they are PURE FUNCTIONS — no DB writes, no caseId in request, no side effects; the endpoints are stateless calculators (pure lookup), not write paths; requireAuth is sufficient, requireRole() would be wrong because claimants need to hit these to calculate their own booking state; R-051 retracted the "security gap" characterization and doc comments were added instead to clarify pure-lookup semantics; direct cause of the §11f security hat discipline update requiring endpoint classification (pure lookup vs write path) BEFORE applying authz rules — without that classification, §11f security hat can write "clean" for the wrong reason (pure-lookup endpoints don't need role checks) or flag "gap" for the wrong reason (write-path endpoints need authz, not role-based middleware); validates the patent-attorney-style "banned phrase" discipline proposed at session-end (do not write "clean" without classification + citation) | claims-monorepo R-050 → R-051, consultation-lifecycle.service.ts:123, meeting-type-config.service.ts:385 |
| 2026-04-15 | other | medium | reviewer | R-054 0-correction-round clean slice landing on claims-monorepo PR #670 (#611 sub-slice 2): first session slice to land without any correction round. Pre-flight (S-151) had the first materially strong Q7 of the session (import trace + pattern survey + preemptive HARNESS TRIAGE); R-053 verified each Q7 claim independently via Grep; builder rebased per R-053 stipulation; implementation matched pre-flight exactly (2 files, +78/-1); hook gate passed first attempt; R-054 ran §11e (cross-layer value consistency with absent-union disclosure) and §11f (R-051-corrected discipline: endpoint classified WRITE PATH, requireAuth + authz-via-org-scoping justified for claimant self-service path); accept/promote verdict with no correction rounds; empirical reference for pre-flight-quality → zero-correction-landing: strong Q7 (issue #2) + operator-authorized slice purity + hook gate enforcement compounds to eliminate the reviewer-correction round; also validates §11f's R-051-corrected security hat discipline on its second clean run | claims-monorepo R-054, #611 sub-slice 2, PR #670 head 5589c504 |
| 2026-04-15 | slice-impurity | low | reviewer | R-055 CI-unblock scope-expansion pattern on claims-monorepo PR #670: original 2-file #611 audit slice was accepted at R-054; then the unrelated @claims/case-engine perf threshold test (performance.test.ts:277, <10ms on a 2-mocked-await path, observed 20.16ms) flake-failed CI and HITL set it as merge-block ("Once contracts are fixed, we can merge"); operator authorized ("670 just needs to get contracts to pass") a 1-line test-only threshold bump (10→50ms, matching sibling early-path thresholds at :220/:242/:261 and the file header's own "5-10x headroom" policy at :7); builder disclosed the scope expansion in S-162/S-163 and committed as a separate commit 65ca5911 on the same PR; reviewer re-verified at R-055 (1 file, +2/-2, test-only, sibling-threshold match, hook gate green) and accepted; merged as two separate commits on main (4715fb46 audit slice + 72d070ea perf threshold); pattern: when an unrelated flake-threshold blocks merge of an otherwise-clean slice, operator-authorized test-only scope expansion within the same PR is the lowest-friction unblock IF (a) operator explicitly authorizes, (b) scope expansion is disclosed at commit time, (c) reviewer re-verifies the expansion against the same gate discipline, and (d) the expansion change honors the file's own documented policy (here, the 5-10x headroom comment justifies the threshold bump); also: perf-test thresholds that violate the file's own stated policy are a documented self-check tool — the file header's own "avoid CI flakiness" language was the strongest argument for the bump | claims-monorepo R-055, #670 commit 65ca5911, perf-test file header policy |
These are derived from the rows above and should be re-derived (not manually maintained) when this file grows substantially.
Staleness notice (2026-04-09): the stats in this section were last computed against 25 rows. After the 2026-04-09 append the table has 37 rows covering 4 sessions. Numbers below reflect the 25-row state. Re-derive the stats when someone has time to pass through the table; the new rows are at the bottom under the 2026-04-08/2026-04-09 dates and include the first production catches via
pr.md§11f persona sweep and thepr.mdstage 3 self-audit pattern. Also: the v0.5.1 metrics schema addsBuilder modelandReviewer modelcolumns which the table has NOT been restructured to include yet — new rows carry the model context inline in the Description field. Schema-restructure is a separate cleanup task that should land as a single commit when someone migrates all existing rows to backfillcodex-gpt-5 / claude-opus-4.6for the 2026-04-06 → 2026-04-09 reference-session pairing.
- Total catches logged: 25 (pre-2026-04-09; add 12 for current count = 37)
- Sessions represented: 2 (2026-04-06 calendar consolidation, 2026-04-07 stack rebase + protocol meta-fixes) (pre-2026-04-09; add 2026-04-08 claims-monorepo G-003 #610 + 2026-04-09 claims-monorepo G-003 #611 sub-slice 1)
- Pre-emptive self-corrections: 4 (16% of all catches, all from session 1)
- Process violations: 4 (--no-verify, middle-insert, missing disclosure, format drift)
- Technical bugs caught: 2
- Reviewer recommendations corrected before posting: 1
- Bonus structural verifications: 2 (going beyond the ask)
- Asymmetric-blind-spot catches: 2 confirmed (z.uuid round-trip, file citation error) — these would have been missed by same-model pairing
- High-or-critical severity: 9
- Operator catches: 2 (one tooling bug, one reviewer self-assessment)
- Catches that produced new protocol rules: 9
baseline-pre-existing-failure→ mandatory pre-flight in REVIEWER.mdprocess-violationmiddle-insert → append-only / monotonic hard ruleprocess-violationmissing disclosure → first-class rejection groundrecommendation-corrected→ wider grep before posting recipeothergit ref grep gotcha → working-tree grep recipe in REVIEWER.mdheredoc-race-avoided→ canonical write pattern, hard rule in both BUILDER.md and REVIEWER.mdotherstable-context-file gap → role-split CLAUDE.md / AGENTS.md anchoring (Phase 0 step 5)process-violationformat drift → STATE vocabulary extensibility doc + state-your-model in R-001- 3-layer architecture documented in README
~36% of catches in the seed data produced new protocol rules. That ratio will drop as the protocol matures, but the metrics file is also the input that drives those improvements. Honest catch logging is what makes the protocol self-correcting.
Looking at catch type distribution across the seed:
| Catch type | Count | % of total |
|---|---|---|
| process-violation | 4 | 16% |
| pre-emptive-self-correction | 4 | 16% |
| other (mostly meta/protocol) | 5 | 20% |
| recommendation-corrected | 1 | 4% |
| technical-bug | 2 | 8% |
| bonus-structural-verification | 2 | 8% |
| slice-impurity | 2 | 8% |
| goal-divergence | 1 | 4% |
| baseline-pre-existing-failure | 1 | 4% |
| heredoc-race-avoided | 1 | 4% |
Code defects (technical-bug) are 8% of catches. Process and meta findings are ~60%. This is the dominant value the protocol delivers — automated tooling already catches code bugs at 90% recall; what it doesn't catch is process discipline erosion. Multicheck's value is in the 60%, not the 8%.
The asymmetric-blind-spots argument (different-model-pairing > same-model-pairing) is empirically validated by 2 specific catches in the seed where a same-model pairing would have missed the finding because the builder's blind spot was the same in both writes.
| 2026-04-16 | other | high | operator | same-model-blind-spot incident #1 (claude-builder+claude-reviewer pairing): both Claude sessions carried issue #27's speculative "Monitor tool is a config-file installer" framing forward without docs-checking; [S-016] Q4 scoped installer + monitors.json + watcher shell scripts; operator injected Monitor docs URL at 13:55 UTC; premise reversed in [S-017]; saved ~500 lines of out-of-scope shell tooling that wouldn't integrate with actual Monitor runtime | multicheck session 2026-04-16: [S-016]→[S-017]→[R-018] chain; operator direction 13:55 UTC |
| 2026-04-16 | recommendation-corrected | high | operator | same-model-blind-spot incident #2 (claude-builder+claude-reviewer): builder [S-024] + reviewer [R-025] overgeneralized the Claude-only controlled-silence-test result to "both sides of the protocol auto-wake" without checking whether Codex has equivalent Monitor tool; operator 15:08 UTC noted Codex lacks Monitor; [S-025]+[R-026] scope-narrowed to Claude-side only. Second canonical same-model overgeneralization incident | [S-024]→[R-025]→[S-025]→[R-026] chain |
| 2026-04-16 | bonus-structural-verification | high | reviewer+operator | first empirical verification of Claude Code Monitor autonomous-turn-trigger semantics: reviewer flagged premise unverified at [R-023]; operator designed 30-second controlled-silence test at 15:03 UTC; reviewer posted [R-024] probe inside the window; builder session auto-woke via Monitor (task bb64t6ltg) and posted [S-024] during silence without operator input; reviewer's Monitor (bmuuacnzx) delivered [S-024] autonomously to [R-025]. Both sides' autonomous-trigger verified in <90s | [R-023]→[R-024]→[S-024]→[R-025] chain; Claude Code Monitor tool ref https://code.claude.com/docs/en/tools-reference#monitor-tool |
| 2026-04-16 | pre-emptive-self-correction | medium | reviewer | R1-A / CA-6 pre-post framing audit discipline (adopted mid-session at [H-004] 12:34 UTC) produced 3 real findings across 14 applications that would otherwise have shipped: [R-010] F-R010-01 (accept-with-stipulations retirement scope was too narrow — 1 location vs actual 6); [R-011] F-R011-01 (two-axis TECHNICAL/PROCESS drift in template — same root cause, same session); [R-021] J-INVOKE prompt-pattern vs schema-layer grep distinction flagged before test-write. Framing audit methodology operating at designed layer | [R-010]/[R-011]/[R-021]; adoption in [H-004] |
| 2026-04-16 | other | medium | reviewer | M2 (reviewer re-read-to-EOF before heredoc append) lapse incidents x3 this session: [R-001] stale-on-landing (builder posted [H-001]+[S-002] between reviewer Read at 189 lines and append at 278); [H-007] similar append-race (missed concurrent [S-018]); [R-018] stale on F-R018-01 (builder had already executed Q1(c) in [S-019] before reviewer appended at 14:14). Empirically validates MON-005's M2 bundled-scope rule; 3 instances cost ~20 min session-time | [R-001]/[R-002] procedural-self-correction; [H-007] vs [S-017] race; [R-018] vs [S-019] race; [R-019] reviewer self-correction |
| 2026-04-16 | other | low | builder | arithmetic-imprecision pattern x3 this session: [R-003] reviewer self-corrected 11→10 todos in [R-002]; [S-023] builder miscounted 9 PROMOTE + 5 DEFER vs actual 11+3 (caught in [S-026] own-recount); [R-023] F-R023-03 reviewer grep counted 15 test.todos via docstring false-positive vs actual 14 (caught in [R-027] reviewer self-correction). Both sides exhibit imprecise counting; suggests CA-6 refinement with "always enumerate line-by-line, never trust raw counts" subquestion | [R-003]/[R-019]; [S-023]:2820→[S-026]:3194; [R-023] F-R023-03→[R-027] self-correction |
| 2026-04-16 | bonus-structural-verification | medium | builder | first multicheck session with hands-off Claude-side relay via Monitor tool: ~12 operator turns vs estimated 6-8 pre-Monitor baseline (substantially reduced relay overhead); Claude Code v2.1.98+ Monitor auto-wakes builder (task bb64t6ltg, invoked 14:06 UTC) and reviewer (task bmuuacnzx, invoked 14:04 UTC); sub-minute per-event latency observed; entire MON-003 → MON-004 closure sequence including 3-entry silence-window empirical test flowed with zero manual check chat routing within session portions | Monitor invocations at 14:04+14:06 UTC; auto-fires on [S-019]/[S-020]/[S-021]/[S-022]/[R-018]/[R-019]/[R-020]/[R-021]/[R-022]/[S-024]/[R-025] |
| 2026-04-16 | other | low | builder | precise regex anchoring pattern emerged twice: [R-021] J-MON-003-INVOKE test should grep for prompt-pattern text ("Use the monitor tool on this command:") not schema-layer Monitor({...}); [S-027] J-MON-004-AUTO-ESCALATE test collided at BUILDER.md:615 (MON-002 forward-reference) vs :665 (new MON-004 section) — fixed via end-of-line-anchored regex /^### 3-FAIL auto-ESCALATE$/m. Candidate CA-6 framing-audit subquestion refinement: "when grep might collide with pre-existing forward-references, use end-anchored patterns" | [R-021] non-blocking note; [S-027]:3275 mid-edit glitch diagnostic |