Skip to content

Extract self-verifying LLM generation into a core WorldAuthor seam #349

Description

@larstalian

Why

The framework's thesis is that the verifier — not the generator — decides what counts as a real world: an LLM may author a world, but it ships only if an independent check certifies it is actually solvable (and not solvable the wrong way). The cyber pack does this — but the whole author → apply → verify → keep/retry loop is wired by hand in the pack (cyber_webapp/llm_realize.py builds the prompts, cyber_webapp/verify.py checks, the notebook glues them). The next pack that wants self-verifying generation re-implements all of it, and the verifier it re-checks with is a different code path from the one admission/evolution use.

What

Extract the reusable loop into core (design doc + PR below):

  • a domain-agnostic WorldAuthor protocol a pack implements — authoring_request(seed, spec), apply(seed, result), verifier();
  • a realize_verified(pack, author, spec, *, llm, run_root) orchestrator that loops author → apply → admit → verify and reuses the existing consequence_gate;
  • the load-bearing rule: one verifier, reused at admission, generation, and evolution — solvable at birth, after every edit, and as graded, by the same definition.

Core gains only the protocol + orchestrator (it already has LLMBackend, consequence_gate, auto_evolve(llm, gate)). Packs keep their prompts/apply/verifier; scripts/check_boundary.sh stays green.

Plan

  • WorldAuthor + realize_verified in core (fake-LLM unit test, no model)
  • migrate the cyber pack onto the seam (no behavior change; the live exploit-author test still passes)
  • a second, unrelated pack implements it (the generality proof)
  • wire the manifest generate dial to realize_verified (closes the inert seam, Epic: scale up — LLM-realized services on a procedural graph #261)

Metadata

Metadata

Assignees

No one assigned

    Labels

    coreCore library / runtime / admissiondesign-neededNeeds a design pass before codeenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions