You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -21,7 +21,7 @@ walks the two traces, applies per-tool semantic-equivalence rules,
21
21
and emits a falsifiable parity score plus a closed-enum drift
22
22
category for any mismatch.
23
23
24
-
**Status (2026-05-15)**: M0–M188 all SHIPPED. **OUTCOME PARITY = 1.0000 (5/5 BOTH_PASS)** on MultiPL-E-Rust HumanEval/0..4 *(measured at M150 against the original 5-fixture POC corpus; **M168 extended the available corpus to 21 fixtures (HumanEval/0..20) — next operator dispatch of `scripts/phase-3-bench.sh` will produce the recalibrated curve)*** — **gated by FALSIFY-CCPA-016 (M152, threshold 0.5)** — both real `claude` and real `apr code` (Qwen2.5-Coder-1.5B) generated working Rust on all 5 measured fixtures. **STRUCTURAL EQUIVALENCE = 0.5201** (M153 P3.3 line-set Jaccard — both pass the test oracle but generate structurally divergent code, range 0.33–0.83 per fixture, zero byte-identical pairs). **TEST-SURVIVAL = 1.0000** (M154 P3.3 sub-metric — 10/10 cross-swaps pass; every test from either system runs correctly against either implementation, proving the structural divergence is STYLISTIC not semantic). Evidence: `evidence/phase-3/multipl-e-rust-scores.json` + `evidence/phase-3/cross-output-equivalence.json` + `evidence/phase-3/test-survival.json`. M148 procedural parity (0.3333 OS-event Jaccard) stays as diagnostic. **Phase 1 Machinery COMPLETE at M141; Phase 2 Execution opens at M142.** See [docs/specifications/phase-2-execution-plan.md](docs/specifications/phase-2-execution-plan.md) for the P2.1-P2.5 sub-deliverables targeting first runtime evidence-based parity measurement. **Completeness caveat**: the `1.0 on 30/30 fixtures` parity score is against **AUTHORED canonical references**, not live Claude Code recordings (M2.3 rescoped Phase 1 RECORD via HTTPS proxy OOS). Honest 3-axis breakdown lives at [docs/specifications/claude-code-parity-apr-poc.md § Completeness assessment](docs/specifications/claude-code-parity-apr-poc.md): Axis 1 (harness machinery) ~95%, **Axis 2 (real differential test vs Claude Code) ~30%**, Axis 3 (production-ready `apr code` validation) ~70%. Contract at v1.27.0
24
+
**Status (2026-05-15)**: M0–M190 all SHIPPED. **OUTCOME PARITY = 1.0000 (5/5 BOTH_PASS)** on MultiPL-E-Rust HumanEval/0..4 *(measured at M150 against the original 5-fixture POC corpus; **M168 extended the available corpus to 21 fixtures (HumanEval/0..20) — next operator dispatch of `scripts/phase-3-bench.sh` will produce the recalibrated curve)*** — **gated by FALSIFY-CCPA-016 (M152, threshold 0.5)** — both real `claude` and real `apr code` (Qwen2.5-Coder-1.5B) generated working Rust on all 5 measured fixtures. **STRUCTURAL EQUIVALENCE = 0.5201** (M153 P3.3 line-set Jaccard — both pass the test oracle but generate structurally divergent code, range 0.33–0.83 per fixture, zero byte-identical pairs). **TEST-SURVIVAL = 1.0000** (M154 P3.3 sub-metric — 10/10 cross-swaps pass; every test from either system runs correctly against either implementation, proving the structural divergence is STYLISTIC not semantic). Evidence: `evidence/phase-3/multipl-e-rust-scores.json` + `evidence/phase-3/cross-output-equivalence.json` + `evidence/phase-3/test-survival.json`. M148 procedural parity (0.3333 OS-event Jaccard) stays as diagnostic. **Phase 1 Machinery COMPLETE at M141; Phase 2 Execution opens at M142.** See [docs/specifications/phase-2-execution-plan.md](docs/specifications/phase-2-execution-plan.md) for the P2.1-P2.5 sub-deliverables targeting first runtime evidence-based parity measurement. **Completeness caveat**: the `1.0 on 30/30 fixtures` parity score is against **AUTHORED canonical references**, not live Claude Code recordings (M2.3 rescoped Phase 1 RECORD via HTTPS proxy OOS). Honest 3-axis breakdown lives at [docs/specifications/claude-code-parity-apr-poc.md § Completeness assessment](docs/specifications/claude-code-parity-apr-poc.md): Axis 1 (harness machinery) ~95%, **Axis 2 (real differential test vs Claude Code) ~85% (post-Phase 4 P4.1-P4.4)**, Axis 3 (production-ready `apr code` validation) ~70%. Contract at v1.28.0 (adds CCPA-017 project_scale_parity_bound at status: PROPOSED; M188+M190 Phase 4 P4.4-P4.5)
25
25
ACTIVE_RUNTIME. Corpus complete at the spec-prescribed 30 fixtures
0 commit comments