Prove runtime-created world eval path

Author: YSCJRH

README.md

@@ -426,6 +426,7 @@ For a guided walkthrough of the canonical demo flow, see [docs/demo/fog-harbor-w
 - Phase 67 closeout shorthand: `#507` `Phase 67 exit gate` closed by PR `#514`; `#508` closed by PR `#510`; `#509` closed by PR `#512`; `#511` closed by PR `#513`.
 - Post-Phase-67 successor intake audit keeps the queue paused; `docs/plans/post-phase-67-successor-intake-audit-2026-06-08.md` records the successor-intake decision. No Phase 68 successor queue is opened by this audit. Do not open Phase 68 as an execution queue until this intake audit identifies a new source-backed minimum-loop gap or protected-core contract blocker. The intake remains tied to `corpus -> chunks -> graph -> personas -> scenarios -> deterministic runs -> report/claims -> eval`.
 - Post-Phase-67 outcome/report/eval generalization audit keeps the queue paused; `docs/plans/post-phase-67-outcome-report-eval-generalization-audit-2026-06-08.md` records that selected-world proof is current for `fog-harbor-east-gate`, `museum-night`, and `library-rain`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this audit.
+- Post-Phase-67 runtime-created world eval proof keeps the queue paused; `docs/plans/post-phase-67-runtime-created-world-eval-proof-2026-06-08.md` records that a runtime-created bounded world can pass `eval-world`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this proof.
 - Phase 48 is closed after PR `#382`, issue `#375`, and milestone `Phase 48 - Successor Intake and Boundary Contract Triage`.
 - Phase 49 is closed after PR `#395`, issue `#383`, and milestone `Phase 49 - Kernel, Perturbation, and Runtime Contract Hardening`; completed work items were `#384`, `#386`, `#388`, `#390`, `#392`, and `#394`.
 - Phase 50 is closed after PR `#402`, issue `#396`, and milestone `Phase 50 - Runtime Orchestration Measurement and Product Boundary`; completed work items were `#397`, `#398`, and `#401`. Phase 50 measured before any `task_id` or worker contract is introduced and kept the private-beta launch hub planning-only for now.

backend/app/evals/service.py

@@ -28,6 +28,13 @@
 DEFAULT_TRANSFER_WORLD_IDS = [CANONICAL_DEMO_WORLD_ID, "museum-night", "library-rain"]
 
 
+def _default_redlines_path(repo_root: Path) -> Path:
+    runtime_path = repo_root / "evals" / "assertions" / "redlines.yaml"
+    if runtime_path.exists():
+        return runtime_path
+    return Path(__file__).resolve().parents[3] / "evals" / "assertions" / "redlines.yaml"
+
+
 def _compare(op: str, left: Any, right: Any) -> bool:
     if op == "gt":
         return left > right
@@ -559,7 +566,7 @@ def record(name: str, passed: bool, details: str) -> None:
     )
 
     redline_failures = _evaluate_redlines_texts(
-        world_paths.repo_root / "evals" / "assertions" / "redlines.yaml",
+        _default_redlines_path(world_paths.repo_root),
         _transfer_redline_texts(artifacts_root),
     )
     record("redlines_pass", not redline_failures, "; ".join(redline_failures) if redline_failures else "ok")

backend/app/world_templates/service.py

@@ -173,6 +173,24 @@ def _baseline_title(spec: CreateWorldTemplateInput) -> str:
     return f"Baseline {spec.world_name}"
 
 
+def _default_report_scenario_title(spec: CreateWorldTemplateInput) -> str:
+    if _is_chinese(spec):
+        return f"{spec.evidence_document_name}延迟"
+    return f"{spec.evidence_document_name} Delay"
+
+
+def _default_report_scenario_description(spec: CreateWorldTemplateInput) -> str:
+    if _is_chinese(spec):
+        return (
+            f"{spec.evidence_document_name}比基线晚两个回合进入决策链，"
+            f"以验证{spec.public_event_name}与{spec.response_location_name}响应是否随证据延迟而变化。"
+        )
+    return (
+        f"{spec.evidence_document_name} reaches the decision loop two turns later than baseline, "
+        f"testing whether {spec.public_event_name} and the response around {spec.response_location_name} change."
+    )
+
+
 def _evaluation_questions(spec: CreateWorldTemplateInput) -> list[str]:
     if _is_chinese(spec):
         return [
@@ -598,10 +616,11 @@ def create_bounded_incident_world(
             }
         )
 
+    default_report_scenario = "evidence_delayed"
     simulation_rules = {
         "world_id": world_id,
         "compare_id": f"scenario_{world_id}_template_matrix",
-        "default_report_scenario": "baseline",
+        "default_report_scenario": default_report_scenario,
         "communications_down_field": "communications_down_until",
         "blocked_contacts_field": "blocked_contacts",
         "turn_sequence": [
@@ -895,6 +914,34 @@ def create_bounded_incident_world(
     }
     _write_yaml(world_root / "scenarios" / "baseline.yaml", baseline_scenario)
 
+    injected_scenario_id = f"scenario_{world_id}_{default_report_scenario}"
+    injected_scenario = {
+        "scenario_id": injected_scenario_id,
+        "world_id": world_id,
+        "title": _default_report_scenario_title(spec),
+        "description": _default_report_scenario_description(spec),
+        "seed": 11,
+        "turn_budget": 8,
+        "branch_count": 1,
+        "evaluation_questions": _evaluation_questions(spec),
+        "injections": [
+            {
+                "injection_id": "inj_evidence_delayed",
+                "kind": "delay_document",
+                "target_id": "doc_user_01",
+                "actor_id": _role_persona_id("records_lead"),
+                "params": {
+                    "delay_turns": 2,
+                    "cause": "routing_delay",
+                },
+                "rationale": (
+                    f"{spec.evidence_document_name} is delayed two turns before it reaches the decision loop."
+                ),
+            }
+        ],
+    }
+    _write_yaml(world_root / "scenarios" / f"{default_report_scenario}.yaml", injected_scenario)
+
     product_payload = {
         "world_id": world_id,
         "world_name": spec.world_name,

backend/tests/test_cli.py

@@ -179,6 +179,7 @@ def test_cli_create_world_writes_runtime_world_pack(tmp_path: Path, capsys, monk
     assert (runtime_root / "state" / "worlds" / world_id / "config" / "world_model.yaml").exists()
     assert (runtime_root / "state" / "worlds" / world_id / "config" / "simulation_rules.yaml").exists()
     assert (runtime_root / "state" / "worlds" / world_id / "config" / "decision_schema.yaml").exists()
+    assert (runtime_root / "state" / "worlds" / world_id / "scenarios" / "evidence_delayed.yaml").exists()
     assert (runtime_root / "state" / "artifacts" / "worlds" / world_id / "graph" / "graph.json").exists()
     assert (runtime_root / "state" / "artifacts" / "worlds" / world_id / "personas" / "personas.json").exists()
 
@@ -198,6 +199,37 @@ def test_cli_create_world_writes_runtime_world_pack(tmp_path: Path, capsys, monk
     assert session_payload["world_id"] == world_id
 
 
+def test_cli_runtime_created_world_passes_eval_world(tmp_path: Path, capsys, monkeypatch) -> None:
+    runtime_root = tmp_path / "repo"
+    runtime_root.mkdir(parents=True, exist_ok=True)
+    monkeypatch.setattr(cli_module, "get_settings", lambda: _runtime_settings(runtime_root))
+
+    assert main(["create-world", "--spec", json.dumps(_safe_world_template_spec())]) == 0
+    create_payload = json.loads(capsys.readouterr().out)
+    world_id = create_payload["world_id"]
+
+    assert main(["eval-world", "--world", world_id]) == 0
+    eval_payload = json.loads(capsys.readouterr().out)
+
+    assert eval_payload["world_id"] == world_id
+    assert eval_payload["status"] == "pass"
+    assert eval_payload["metrics"]["tracked_outcome_count"] == 5
+    assert eval_payload["metrics"]["tracked_outcome_fields_covered"] == 5
+    assert eval_payload["metrics"]["compare_outcome_fields_covered"] == 5
+    assert eval_payload["metrics"]["changed_tracked_outcome_count"] >= 1
+    assert eval_payload["metrics"]["default_report_changed_outcome_count"] >= 1
+    assert eval_payload["metrics"]["report_compare_sourced"] is True
+    assert eval_payload["metrics"]["transfer_proof_world_local"] is True
+
+    artifacts_root = runtime_root / "state" / "artifacts" / "worlds" / world_id
+    claims = json.loads((artifacts_root / "report" / "claims.json").read_text(encoding="utf-8"))
+    assert claims
+    assert all(claim.get("label") for claim in claims)
+    assert all(claim.get("evidence_ids") for claim in claims)
+    assert (artifacts_root / "eval" / "summary.json").exists()
+    assert (artifacts_root / "compare" / "scenario_harbor_night_drill_template_matrix" / "compare.json").exists()
+
+
 def test_cli_create_world_uses_locale_for_generated_product_copy(
     tmp_path: Path, capsys, monkeypatch
 ) -> None:

backend/tests/test_post_phase67_runtime_created_world_eval_proof.py

@@ -0,0 +1,109 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+
+PROOF_PATH = Path("docs/plans/post-phase-67-runtime-created-world-eval-proof-2026-06-08.md")
+MINIMUM_LOOP = (
+    "corpus -> chunks -> graph -> personas -> scenarios -> deterministic runs -> "
+    "report/claims -> eval"
+)
+
+
+def _read(path: Path) -> str:
+    assert path.exists(), path
+    return path.read_text(encoding="utf-8")
+
+
+def test_runtime_created_world_eval_proof_exists_with_guardrails() -> None:
+    proof = _read(PROOF_PATH)
+    required_sections = [
+        "# Post-Phase-67 Runtime-Created World Eval Proof",
+        "## Scope",
+        "## Source Evidence",
+        "## Validation Evidence",
+        "## Queue Decision",
+        "## Boundaries",
+        "## Validation Commands",
+    ]
+    for section in required_sections:
+        assert section in proof
+
+    required_phrases = [
+        f"`{MINIMUM_LOOP}`",
+        "runtime-created bounded world can pass `eval-world`",
+        "This closes only the narrow runtime-created template proof question",
+        "The queue remains paused.",
+        "Do not open Phase 68 from this proof.",
+        "does not claim future-world readiness",
+        "Do not present Mirror as a real-world prediction machine.",
+        "Do not build real-person personas or digital doubles.",
+        "Do not build political persuasion, hidden surveillance, law-enforcement scoring, hiring, credit, medical, or judicial decision systems.",
+    ]
+    for phrase in required_phrases:
+        assert phrase in proof, phrase
+
+
+def test_runtime_created_world_eval_proof_is_source_backed() -> None:
+    proof = _read(PROOF_PATH)
+    template_service = _read(Path("backend/app/world_templates/service.py"))
+    eval_service = _read(Path("backend/app/evals/service.py"))
+    cli_tests = _read(Path("backend/tests/test_cli.py"))
+
+    for phrase in [
+        'default_report_scenario = "evidence_delayed"',
+        '"default_report_scenario": default_report_scenario',
+        'world_root / "scenarios" / f"{default_report_scenario}.yaml"',
+        '"kind": "delay_document"',
+    ]:
+        assert phrase in template_service
+        assert phrase in proof
+
+    for phrase in [
+        "def _default_redlines_path(repo_root: Path) -> Path:",
+        "runtime_path = repo_root / \"evals\" / \"assertions\" / \"redlines.yaml\"",
+        "Path(__file__).resolve().parents[3] / \"evals\" / \"assertions\" / \"redlines.yaml\"",
+    ]:
+        assert phrase in eval_service
+
+    for phrase in [
+        "test_cli_runtime_created_world_passes_eval_world",
+        'main(["eval-world", "--world", world_id])',
+        'eval_payload["metrics"]["tracked_outcome_fields_covered"] == 5',
+        'eval_payload["metrics"]["compare_outcome_fields_covered"] == 5',
+        'eval_payload["metrics"]["report_compare_sourced"] is True',
+        'eval_payload["metrics"]["transfer_proof_world_local"] is True',
+        'claim.get("label")',
+        'claim.get("evidence_ids")',
+        '"scenario_harbor_night_drill_template_matrix"',
+    ]:
+        assert phrase in cli_tests
+        assert phrase in proof
+
+
+def test_current_docs_point_to_runtime_created_world_eval_proof_without_phase68() -> None:
+    docs = [
+        Path("README.md"),
+        Path("docs/plans/current-state-baseline.md"),
+        Path("docs/plans/phase-execution-queue.md"),
+        Path("docs/plans/automation-roadmap.md"),
+    ]
+    required_phrases = [
+        "`docs/plans/post-phase-67-runtime-created-world-eval-proof-2026-06-08.md`",
+        "Post-Phase-67 runtime-created world eval proof keeps the queue paused",
+        "runtime-created bounded world can pass `eval-world`",
+        "does not claim future-world readiness",
+        "Do not open Phase 68 from this proof.",
+    ]
+    forbidden_phrases = [
+        "Phase 68 is active",
+        "Phase 68 execution queue is open",
+        "`audit-github-queue` reports `ready` for Phase 68",
+        "milestone `Phase 68",
+    ]
+    for path in docs:
+        text = _read(path)
+        for phrase in required_phrases:
+            assert phrase in text, f"{path} is missing runtime-created eval proof pointer: {phrase}"
+        for phrase in forbidden_phrases:
+            assert phrase not in text, f"{path} opens Phase 68 prematurely: {phrase}"

docs/plans/automation-roadmap.md

@@ -29,6 +29,7 @@ Day 0 bootstrap is complete, Phase 5 closeout is complete, Phase 6 closeout is c
 - Phase 67 closeout shorthand: `#507` `Phase 67 exit gate` closed by PR `#514`; `#508` closed by PR `#510`; `#509` closed by PR `#512`; `#511` closed by PR `#513`.
 - Post-Phase-67 successor intake audit keeps the queue paused; `docs/plans/post-phase-67-successor-intake-audit-2026-06-08.md` records the successor-intake decision. No Phase 68 successor queue is opened by this audit. Do not open Phase 68 as an execution queue until this intake audit identifies a new source-backed minimum-loop gap or protected-core contract blocker. The intake remains tied to `corpus -> chunks -> graph -> personas -> scenarios -> deterministic runs -> report/claims -> eval`.
 - Post-Phase-67 outcome/report/eval generalization audit keeps the queue paused; `docs/plans/post-phase-67-outcome-report-eval-generalization-audit-2026-06-08.md` records that selected-world proof is current for `fog-harbor-east-gate`, `museum-night`, and `library-rain`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this audit.
+- Post-Phase-67 runtime-created world eval proof keeps the queue paused; `docs/plans/post-phase-67-runtime-created-world-eval-proof-2026-06-08.md` records that a runtime-created bounded world can pass `eval-world`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this proof.
 - Phase 1 and Phase 2 gates are closed.
 - Phase 3 is closed locally and in GitHub.
 - Phase 3 exit issue `#4` is closed and milestone `Phase 3 - Eval/UI/Demo` is closed.

docs/plans/current-state-baseline.md

@@ -28,6 +28,8 @@ Post-Phase-67 successor intake audit keeps the queue paused; `docs/plans/post-ph
 
 Post-Phase-67 outcome/report/eval generalization audit keeps the queue paused; `docs/plans/post-phase-67-outcome-report-eval-generalization-audit-2026-06-08.md` records that selected-world proof is current for `fog-harbor-east-gate`, `museum-night`, and `library-rain`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this audit.
 
+Post-Phase-67 runtime-created world eval proof keeps the queue paused; `docs/plans/post-phase-67-runtime-created-world-eval-proof-2026-06-08.md` records that a runtime-created bounded world can pass `eval-world`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this proof.
+
 ## Snapshot
 
 - Local quality baseline:

docs/plans/phase-execution-queue.md

@@ -118,6 +118,7 @@ Phase 67 - Blueprint Calibration and Minimum-Loop Value Gate
   - Do not open another adjacent surface/readiness/fidelity/continuity gate as the primary successor scope without a source-backed tie to scenario/intervention/branch-comparison/eval value.
   - Post-Phase-67 successor intake audit keeps the queue paused; `docs/plans/post-phase-67-successor-intake-audit-2026-06-08.md` records the successor-intake decision. No Phase 68 successor queue is opened by this audit. Do not open Phase 68 as an execution queue until this intake audit identifies a new source-backed minimum-loop gap or protected-core contract blocker. The intake remains tied to `corpus -> chunks -> graph -> personas -> scenarios -> deterministic runs -> report/claims -> eval`.
   - Post-Phase-67 outcome/report/eval generalization audit keeps the queue paused; `docs/plans/post-phase-67-outcome-report-eval-generalization-audit-2026-06-08.md` records that selected-world proof is current for `fog-harbor-east-gate`, `museum-night`, and `library-rain`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this audit.
+  - Post-Phase-67 runtime-created world eval proof keeps the queue paused; `docs/plans/post-phase-67-runtime-created-world-eval-proof-2026-06-08.md` records that a runtime-created bounded world can pass `eval-world`, does not claim future-world readiness, and does not prove a Phase 68 blocker. Do not open Phase 68 from this proof.
   - If a future minimum-loop audit finds a contract gap, split a separate protected-core contract issue before changing schema, scenario DSL, claim labels, run trace shape, or artifact layout.
 
 ## Phase 66 Closed Queue