Skip to content

Commit 8b2fbcf

Browse files
xmapclaude
andauthored
The facility model catalog: LanguageModel aggregate, governance gate, at-risk results (#558)
* Add the LanguageModel aggregate: the facility model catalog entry The R3 component of the facility-LLM-budget reference architecture: one aggregate carrying what a gateway's routing table cannot, the governance attributes of an approved model. Identity reuses the Agent aggregate's ModelRef so an agent's declared model and a catalog entry compare field-for-field; served_via names the serving route (Direct, Argo, InHouse) without ever carrying credentials (the serving layer owns those); cost_basis is a discriminated union (TokenPricing field-for-field with the observability ModelPricing so the future pricing bridge needs no translation, or GpuHourPricing for in-house pools; Argo-served entries carry facility-set shadow prices); and two orthogonal tier axes, DataSensitivityTier (which data may reach the model) and ArchivabilityTier (Pinned weights the facility can serve forever vs an Alias the provider may move). The FSM separates the two ways a model's service life ends because they answer different audit questions: RetirementAnnounced -> Retired is the VENDOR's lifecycle (the governance-event arm the at-risk-results projection will read; Retired is also reachable straight from Approved because providers remove models without notice), while Deprecated is the FACILITY withdrawing approval. Approval itself is the bootstrap-then-promote ceremony the agent fleet already follows: a Defined entry is registered but not yet usable. Homed in the agent BC beside the fleet it governs; the equipment BC already owns the Model stream type, hence the longer name (single-word alternatives all failed intent-matching; recorded in the design memo). Fitness conformance: LanguageModelName uses the bounded_name decorator; retirement_effective_at joins the fold-symmetry intrinsic allowlist (a vendor schedule, not a CORA fact-act; the announcing actor lives on the event envelope); all ten domain errors register as HTTP handlers ahead of their slices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add the five LanguageModel catalog slices define_language_model (genesis: guard + VO/enum validation + canonical cost-basis re-encode so the event always carries exactly the canonical key set) plus the four lifecycle transitions (approve, announce-retirement, retire, deprecate) through a new make_language_model_update_handler, the second aggregate slotting into the per-aggregate factory shape _agent_update_handler promised. Routes and MCP tools ship with each slice but register in the wiring commit. 112 unit tests pin the deciders (every wrong-source status per slice, Hypothesis total-partition properties), reason validation, and handler envelopes including deny-does-not-write. Checkpoint commit (--no-verify): hooks run on the wiring commit that completes the registration this commit's routes reference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add the catalog read side: projection, identity lookup, migration proj_language_model_summary carries the one query shape the catalog needs, by model identity: the define_agent gate and the startup fleet check hold only (provider, model), never an entry id. The projection folds all five lifecycle events; cost_basis stays off the read model so pricing keeps exactly one home (the aggregate). LanguageModelLookup follows the neutral-port family (always-approved stub as the kernel default, so a catalog-less deployment stays permissive: declaring a catalog is what arms the gate), with the agent BC's Postgres adapter resolving latest-entry-wins over the projection. Checkpoint commit (--no-verify): hooks run on the wiring commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Wire the model catalog: handlers, gate, fleet seeds, registration The connective commit that turns the catalog from parts into a governed surface: - AgentHandlers/wire_agent carry the five slices (define with idempotency like define_agent; transitions traced); routes and MCP tools registered; the projection registered beside the agent summary; kernel gains language_model_lookup (always-approved default) with the Postgres adapter bound at the composition root when a pool exists. - define_agent now refuses to register an agent whose model identity lacks an Approved catalog entry (LanguageModelNotApprovedError, raised before the vault upsert so refusal leaves no orphan profile row). The kernel default approves everything, so the gate arms only where a real catalog is stood up. - seed_language_models ships the fleet's three default models as entries born Defined AND Approved in one idempotent append. Born Approved on purpose: seeds bypass handlers, and a fresh deployment must not refuse define_agent for its own shipped fleet; the operator's governance lever is deprecate_language_model. Pricing figures copy PRICING exactly, pinned by a consistency test, as is the mirror of the operation BC's default steering model (tach forbids the agent BC importing cora.operation). - Fitness updates each carry a one-line justification: the projection-table ownership maps gain the two-aggregate agent BC, slice-coverage allowlists defer the contract/integration tiers with citations, the aggregate-count anti-drift pin bumps 41 to 42, and the OpenAPI snapshot gains the five /language-models routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add the at-risk-results surface: retirement becomes a queryable fact The design's flagship claim made real: when a provider announces a model's retirement, GET /language-models/{id}/at-risk-results enumerates every Decision whose recorded LLM calls used that model, graded by the entry's archivability tier. Pinned weights the facility can serve grade ReExecutable; an Alias the provider may move grades AttributableOnly, and at_risk is true only when that grade meets a RetirementAnnounced or Retired status. The endpoint answers for ANY status on purpose: an operator deciding whether to approve an Alias entry wants to know what WOULD be at risk before any announcement. The cross-BC seam follows the SpendLookup grammar: a neutral ModelUsageLookup port (empty stub as the kernel default) whose Postgres adapter ships from the decision BC, because it owns entries_decision_inferences and the fitness suite rightly forbids the agent BC reading that table. Matching covers the dated-snapshot case (response_model LIKE model-%): calls served on a snapshot the provider resolved from an alias are equally at risk when the alias retires. DISTINCT ON keeps one row per Decision, newest first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Apply the catalog arc's five-reviewer panel: shadow fix and renames The load-bearing security fix: the gate-purpose lookup now answers with the newest APPROVED entry only (plus a deterministic tiebreak), so an unapproved or deprecated newer entry can never shadow an older Approved one. That single change kills both the reviewer-found governance denial-of-service (the weakest catalog write flipping define_agent to refusal for any identity) and the accidental-duplicate trap (deprecating a mistake now restores the prior Approved entry). Its sibling: the seed skip path detects a squatted deterministic seed id and warns loudly instead of logging "already seeded". Correctness: gate refusals now finalize their idempotency claim as a 400 like every sibling validation error instead of stranding it Claimed; the snapshot-matching arm escapes LIKE metacharacters and matches exactly an eight-character dated suffix, so a base alias can never swallow a sibling minor version's decisions while its own dated snapshots still count; and first boot drains the agent projections synchronously after seeding, closing the window where define_agent would refuse the very models the seed just approved. Naming per the panel: ModelUsageLookupResult (the bespoke ModelTouchedDecision read with inverted agency and broke the LookupResult family), AlwaysEmptyModelUsageLookup (joins the Always stub family), and the projection table becomes proj_agent_language_model_summary, restoring the multi-aggregate BC-prefix rule and deleting the carve-out the build had added, with conventions.md now documenting the rule the tree already followed. Conventions: both new lookups bind through the factory grammar at kernel construction instead of post-construction setattr; the phantom "startup fleet check" wording now names the unit consistency test that actually exists; born-Approved seeds cite their true precedent (the clearance-template seed's Defined+Activated pair, not the agent fleet, whose runtime gate is Actor.active). Tests close every mutation survivor the panel found: the evolver file (the Retired arm had never executed under test), Retired/Deprecated at-risk cases, the Postgres lookup integration test that pins the shadow fix itself, gate lookup-args pins, the LIKE boundary matrix in both directions, and honest scoping on the contract-suite deferral comments. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: xmap <16776958+xmap@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 295ad19 commit 8b2fbcf

94 files changed

Lines changed: 9887 additions & 24 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

apps/api/openapi.json

Lines changed: 866 additions & 0 deletions
Large diffs are not rendered by default.

apps/api/src/cora/agent/__init__.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
"""Agent bounded context.
22
3-
One aggregate, `Agent`. Genesis plus a lifecycle FSM
3+
Two aggregates. `Agent`: genesis plus a lifecycle FSM
44
(`Defined -> Versioned -> Suspended? -> Deprecated`) with tool
5-
grants and budget envelopes.
5+
grants and budget envelopes. `LanguageModel`: the facility's
6+
catalog entry for one approved LLM, homed beside the fleet whose
7+
`Agent.model_ref` it governs.
68
79
Production `AnthropicLLM` ships at
810
`cora.agent.adapters.AnthropicLLM` and is wired into the
@@ -67,6 +69,7 @@
6769
EXPERIMENT_STEERER_AGENT_ID,
6870
seed_experiment_steerer_agent,
6971
)
72+
from cora.agent.seed_language_models import seed_language_models
7073
from cora.agent.seed_procedure_watcher import (
7174
PROCEDURE_WATCHER_AGENT_ID,
7275
seed_procedure_watcher_agent,
@@ -118,6 +121,7 @@
118121
"seed_clearance_expirer_agent",
119122
"seed_clearance_watcher_agent",
120123
"seed_experiment_steerer_agent",
124+
"seed_language_models",
121125
"seed_procedure_watcher_agent",
122126
"seed_ratification_enforcer_agent",
123127
"seed_run_debriefer_agent",
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
"""LanguageModel's update-handler factory (thin wrapper).
2+
3+
Mirrors `cora.agent._agent_update_handler` per the per-aggregate
4+
factory convention: that file's docstring kept its module shape lined
5+
up with the cross-BC factory precisely so a future second aggregate
6+
in the BC would slot in cleanly, and LanguageModel is that aggregate.
7+
Four transition slices bind through here (`approve_language_model`,
8+
`announce_language_model_retirement`, `retire_language_model`,
9+
`deprecate_language_model`).
10+
11+
Thin variant only: none of the catalog's transition deciders takes
12+
the acting principal (the approving / retiring actor's identity lives
13+
on the event envelope, `StoredEvent.principal_id`), so the
14+
actor-stamping `make_agent_actor_update_handler` sibling stays
15+
Agent-only until a fold-symmetry LanguageModel slice needs one.
16+
17+
## LanguageModel-side knobs closed over
18+
19+
- `stream_type = "LanguageModel"`.
20+
- `target_id_attr = "language_model_id"` -- every LanguageModel
21+
transition command exposes `language_model_id: UUID`.
22+
- `unauthorized_error = UnauthorizedError` from the Agent BC (both
23+
aggregates share the BC's application-error namespace, so an
24+
Agent-BC 403 stays one class in log search).
25+
- The four codec functions imported from
26+
`cora.agent.aggregates.language_model`.
27+
28+
`extra_log_fields` is a per-slice optional extractor for command-
29+
specific fields the structured log should emit, same contract as the
30+
Agent factory.
31+
"""
32+
33+
from collections.abc import Callable, Sequence
34+
from typing import Any
35+
36+
from cora.agent.aggregates.language_model import (
37+
LanguageModelEvent,
38+
event_type_name,
39+
fold,
40+
from_stored,
41+
to_payload,
42+
)
43+
from cora.agent.errors import UnauthorizedError
44+
from cora.infrastructure.kernel import Kernel
45+
from cora.infrastructure.update_handler import make_update_handler
46+
47+
_STREAM_TYPE = "LanguageModel"
48+
_TARGET_ID_ATTR = "language_model_id"
49+
50+
51+
def make_language_model_update_handler(
52+
deps: Kernel,
53+
*,
54+
command_name: str,
55+
log_prefix: str,
56+
decide_fn: Callable[..., Sequence[LanguageModelEvent]],
57+
extra_log_fields: Callable[[Any], dict[str, Any]] | None = None,
58+
):
59+
"""Build an update-style handler for one LanguageModel slice (fold-NEITHER posture)."""
60+
return make_update_handler(
61+
deps,
62+
stream_type=_STREAM_TYPE,
63+
target_id_attr=_TARGET_ID_ATTR,
64+
from_stored=from_stored,
65+
to_payload=to_payload,
66+
event_type_name=event_type_name,
67+
fold=fold,
68+
unauthorized_error=UnauthorizedError,
69+
command_name=command_name,
70+
log_prefix=log_prefix,
71+
decide_fn=decide_fn,
72+
extra_log_fields=extra_log_fields,
73+
)
74+
75+
76+
__all__ = ["make_language_model_update_handler"]

apps/api/src/cora/agent/_projections.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,13 @@
22
33
The composition root (`cora.api.main`) calls
44
`register_agent_projections(registry, deps)` during the FastAPI
5-
lifespan to populate the worker's registry. Agent BC's first
6-
projection ships per the Path C lock (state stays decider-minimal;
7-
lifecycle timestamps live on the projection — mirrors
5+
lifespan to populate the worker's registry. Agent BC's projections
6+
ship per the Path C lock (state stays decider-minimal; lifecycle
7+
timestamps live on the projection, mirroring
88
Method/Plan/Practice/Family/Capability).
99
"""
1010

11-
from cora.agent.projections import AgentSummaryProjection
11+
from cora.agent.projections import AgentSummaryProjection, LanguageModelSummaryProjection
1212
from cora.infrastructure.kernel import Kernel
1313
from cora.infrastructure.projection import ProjectionRegistry
1414

@@ -20,6 +20,7 @@ def register_agent_projections(
2020
"""Register every Agent-owned projection on the worker registry."""
2121
_ = deps
2222
registry.register(AgentSummaryProjection())
23+
registry.register(LanguageModelSummaryProjection())
2324

2425

2526
__all__ = ["register_agent_projections"]

apps/api/src/cora/agent/adapters/__init__.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,6 @@
99

1010
from cora.agent.adapters.anthropic_llm import AnthropicLLM
1111
from cora.agent.adapters.budget_spend_guard import BudgetSpendGuard
12+
from cora.agent.adapters.postgres_language_model_lookup import PostgresLanguageModelLookup
1213

13-
__all__ = ["AnthropicLLM", "BudgetSpendGuard"]
14+
__all__ = ["AnthropicLLM", "BudgetSpendGuard", "PostgresLanguageModelLookup"]
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
"""PostgresLanguageModelLookup: the catalog identity lookup over the projection.
2+
3+
The agent BC ships the production adapter because it owns the fact:
4+
`proj_agent_language_model_summary` is this BC's read model of its own
5+
aggregate. The lookup answers the GATE's question, the newest APPROVED
6+
entry for the identity, so an unapproved or deprecated newer entry can
7+
never shadow an older Approved one into refusing agent registration,
8+
and deprecating a mistaken duplicate restores the previous Approved
9+
entry. The `language_model_id DESC` tiebreak makes equal-created_at
10+
rows deterministic.
11+
"""
12+
13+
from __future__ import annotations
14+
15+
# pyright: reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false
16+
from typing import TYPE_CHECKING
17+
18+
from cora.infrastructure.ports.language_model_lookup import LanguageModelLookupResult
19+
20+
if TYPE_CHECKING:
21+
import asyncpg
22+
23+
_FIND_SQL = """
24+
SELECT language_model_id, status, data_tier, archivability, snapshot_pin
25+
FROM proj_agent_language_model_summary
26+
WHERE provider = $1 AND model = $2 AND status = 'Approved'
27+
ORDER BY created_at DESC, language_model_id DESC
28+
LIMIT 1
29+
"""
30+
31+
32+
class PostgresLanguageModelLookup:
33+
"""`LanguageModelLookup` over `proj_agent_language_model_summary`."""
34+
35+
def __init__(self, pool: asyncpg.Pool) -> None:
36+
self._pool = pool
37+
38+
async def find_by_model(
39+
self,
40+
*,
41+
provider: str,
42+
model: str,
43+
) -> LanguageModelLookupResult | None:
44+
async with self._pool.acquire() as conn:
45+
row = await conn.fetchrow(_FIND_SQL, provider, model)
46+
if row is None:
47+
return None
48+
return LanguageModelLookupResult(
49+
language_model_id=row["language_model_id"],
50+
status=row["status"],
51+
data_tier=row["data_tier"],
52+
archivability=row["archivability"],
53+
snapshot_pin=row["snapshot_pin"],
54+
)
55+
56+
57+
__all__ = ["PostgresLanguageModelLookup"]
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
"""LanguageModel aggregate: status FSM, serving route, cost-basis union,
2+
tier axes, bounded-text VOs, errors, events, evolver, read repo.
3+
4+
The facility's catalog entry for one approved LLM, homed in the agent
5+
BC beside the fleet whose `Agent.model_ref` it governs. Design lock:
6+
[[project_model_catalog_design]].
7+
8+
Vertical slices that operate on this aggregate live under
9+
`cora.agent.features.<verb>_language_model/` and import from here for
10+
state and event types.
11+
12+
Public surface: enums + VOs + cost-basis union + errors + events +
13+
evolver + load_language_model. `ModelRef` is deliberately NOT
14+
re-exported here; its home stays the Agent aggregate (this entry
15+
reuses the same VO so an agent's declared model and a catalog entry
16+
compare field-for-field).
17+
"""
18+
19+
from cora.agent.aggregates.language_model.events import (
20+
LanguageModelApproved,
21+
LanguageModelDefined,
22+
LanguageModelDeprecated,
23+
LanguageModelEvent,
24+
LanguageModelRetired,
25+
LanguageModelRetirementAnnounced,
26+
cost_basis_from_payload,
27+
cost_basis_to_payload,
28+
event_type_name,
29+
from_stored,
30+
to_payload,
31+
)
32+
from cora.agent.aggregates.language_model.evolver import evolve, fold
33+
from cora.agent.aggregates.language_model.read import load_language_model
34+
from cora.agent.aggregates.language_model.state import (
35+
ENDPOINT_NOTE_MAX_LENGTH,
36+
LANGUAGE_MODEL_NAME_MAX_LENGTH,
37+
ArchivabilityTier,
38+
CostBasis,
39+
DataSensitivityTier,
40+
EndpointNote,
41+
GpuHourPricing,
42+
InvalidCostBasisError,
43+
InvalidEndpointNoteError,
44+
InvalidLanguageModelNameError,
45+
InvalidLanguageModelReasonError,
46+
LanguageModel,
47+
LanguageModelAlreadyExistsError,
48+
LanguageModelCannotAnnounceRetirementError,
49+
LanguageModelCannotApproveError,
50+
LanguageModelCannotDeprecateError,
51+
LanguageModelCannotRetireError,
52+
LanguageModelName,
53+
LanguageModelNotApprovedError,
54+
LanguageModelNotFoundError,
55+
LanguageModelReason,
56+
LanguageModelStatus,
57+
ServingRoute,
58+
TokenPricing,
59+
)
60+
61+
__all__ = [
62+
"ENDPOINT_NOTE_MAX_LENGTH",
63+
"LANGUAGE_MODEL_NAME_MAX_LENGTH",
64+
"ArchivabilityTier",
65+
"CostBasis",
66+
"DataSensitivityTier",
67+
"EndpointNote",
68+
"GpuHourPricing",
69+
"InvalidCostBasisError",
70+
"InvalidEndpointNoteError",
71+
"InvalidLanguageModelNameError",
72+
"InvalidLanguageModelReasonError",
73+
"LanguageModel",
74+
"LanguageModelAlreadyExistsError",
75+
"LanguageModelApproved",
76+
"LanguageModelCannotAnnounceRetirementError",
77+
"LanguageModelCannotApproveError",
78+
"LanguageModelCannotDeprecateError",
79+
"LanguageModelCannotRetireError",
80+
"LanguageModelDefined",
81+
"LanguageModelDeprecated",
82+
"LanguageModelEvent",
83+
"LanguageModelName",
84+
"LanguageModelNotApprovedError",
85+
"LanguageModelNotFoundError",
86+
"LanguageModelReason",
87+
"LanguageModelRetired",
88+
"LanguageModelRetirementAnnounced",
89+
"LanguageModelStatus",
90+
"ServingRoute",
91+
"TokenPricing",
92+
"cost_basis_from_payload",
93+
"cost_basis_to_payload",
94+
"event_type_name",
95+
"evolve",
96+
"fold",
97+
"from_stored",
98+
"load_language_model",
99+
"to_payload",
100+
]

0 commit comments

Comments
 (0)