feat(ai): per-action Gemini model routing — Socratic/hint to flash-lite (#868)

[thomgabriel]

Closes #868 (epic #838 item 3b / economics doc P1-1).

The AI Partner and the reflection reply each pinned their own GEMINI_URL to gemini-3.5-flash. This replaces both with one routing module (apps/web/src/lib/ai/models.ts) that owns the action→model table, the price sheet, and the URL builder — so routing and billing can't drift apart.

Priority context: the just-merged assist ladder (#887) runs Socratic turns — the cheapest shape the tutor emits and the highest-volume tier (10→30 turns per lesson, 60 with the self-reset) — on the expensive pinned model. That's what this lands to stop.

Routing table

Caller	Model	Why
hint	gemini-3.5-flash-lite	short, high-volume
ask	gemini-3.6-flash	quality-sensitive reasoning
propose	gemini-3.6-flash	diff contract + comprehension check
review	gemini-3.6-flash	not in the doc's table; same quality shape as ask, so it follows ask, not the cheap tier
Socratic-tier hint/ask	gemini-3.5-flash-lite	overrides the base mapping — one diagnostic question at a hint-sized cap
Socratic-tier propose/review	gemini-3.6-flash	§4.2 keeps propose's full diff contract at every tier; review is post-pass
reflection-reply	gemini-3.5-flash-lite	one unstructured note, hard-capped at 512 output tokens, best-effort, discarded on any failure — nothing needs 3.6-flash's headroom (reasoning is recorded in-file)

Both target models were verified reachable on the prod key by the 2026-07-28 curls (#838, AIE-04), with thinkingBudget: 0 verified honored (AIE-05).

Defaults live in code. Optional per-action env overrides (AI_MODEL_HINT|ASK|PROPOSE|REVIEW|SOCRATIC|REFLECTION) exist as an escape hatch for a price move or a deprecation; a value outside the priced model set is ignored with a warning, because routing to an unpriced model would silently break the ledger's accounting.

Ledger pricing (this was single-model — it is now per-model)

lib/ai/spend-ledger.ts priced every billed call from one global rate pair (AI_SPEND_{INPUT,OUTPUT}_USD_PER_MTOK, defaults $0.30/$2.50). Two problems: (a) those defaults are flash-lite's sheet, so the pinned gemini-3.5-flash traffic was already under-reported ~3× on output; (b) with per-action routing, one rate pair mis-prices every action that isn't the routed default.

recordAiSpend now takes the model it actually called and prices it from MODEL_RATES (economics doc §2.1):

Model	Input $/M	Output $/M
gemini-3.6-flash	1.50	7.50
gemini-3.5-flash-lite	0.30	2.50
gemini-3.5-flash (fallback pin, kept)	1.50	9.00

The two env vars survive as an emergency all-models override, now unset by default (usdNumOpt) so the per-model sheet is what bills.

Red-proof (the pricing change is behaviour, not refactor): the new spend-ledger.test.ts was run against origin/main's ledger in a detached worktree — 8 tests fail, including bills the SAME usage differently per routed model (main books p_micro_usd: 0 where the routed model should book 5250). Green on this branch.

Cost impact

Per the economics doc §2.1/§2.2 shapes (thinking off, verified):

• Socratic turn: $0.004 → **$0.0007** (≈5×)
• hint: $0.0036 → ~$0.0010
• ask: $0.0068 → ~$0.0060
• propose: $0.0113 → ~$0.0098
• reflection reply (512 tok ceiling): ~5× cheaper on output

The doc's headline is a ~32% blended cut; the ladder's Socratic volume makes the realized cut larger. The #591 fail-closed spend ledger remains the actual ceiling — this widens the headroom under it, it doesn't relax it.

Failure semantics

A routed model that 404s fails closed: the existing !response.ok branch refunds the assist tier and 502s. There is deliberately no silent fallback to another model — that would bill a learner's turn against a model the ledger wasn't told about. Covered by a test asserting exactly one upstream attempt.

Stale comments corrected

route.ts:69-70 claimed the 2.5 family 404s "for new keys" and that 2.0-flash is retired. The 2026-07-28 curl record disproves the gating claim (2.5-flash/-lite appear in this key's model list; all three candidates returned HTTP 200) and attributes the original 404s to a first-connection artifact — empty-bodied, gone on retry. The corrected note carries that "re-test twice before believing a 404" rule forward. The same stale claim in degradedMaxTokens's doc comment (flash-lite "gated for new keys") is fixed too — flash-lite is now the default for hint and Socratic.

Not in scope

Flex-tier spike is NOT built. Owner decision D-7 (spike owner + exit criterion, suggested p95 < 4s on hint) is unresolved, and the issue conditions the spike on that criterion being demonstrated and recorded. No Flex code ships here.

Verification

• pnpm typecheck clean
• Full web suite: 240 files / 2202 tests passed
• pnpm lint: no new warnings (109 → 109 pre-existing)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
superteam-academy-web	Ready	Preview	Jul 30, 2026 9:09pm

[claude]

Claude finished @thomgabriel's task in 3m 1s —— View job

---

Code review — #890 (per-action Gemini model routing, closes #868)

Task list

• Read CLAUDE.md / docs/TASK-CODES.md conventions
• Verify diff meets issue Per-action Gemini model routing: ask/propose → 3.6-flash, hint → 3.5-flash-lite (item 3b) #868 "Done when" criteria (via PR body's own spec — gh issue view was blocked in this sandbox, see note below)
• Review apps/web/src/lib/ai/models.ts (new routing module)
• Review apps/web/src/lib/ai/spend-ledger.ts per-model pricing changes
• Review apps/web/src/app/api/ai/partner/route.ts changes
• Review apps/web/src/lib/ai/reflection-reply.ts + env.server.ts
• Check tests for coverage of routing/pricing/failure semantics
• Scan for secrets/trust-boundary/RLS/on-chain concerns (scope check)
• Post final review

Blocking

None found.

Scope check (repo NEVER-ship-broken areas)

Not applicable — this PR touches only apps/web/src/lib/ai/**, apps/web/src/app/api/ai/partner/route.ts, env.server.ts, and apps/web/CLAUDE.md. No supabase/schema.sql/RLS, no onchain-academy/**, no new client-exposed env vars (GEMINI_API_KEY handling is unchanged and still server-only), no client-trusted validation paths.

What checks out

• models.ts is a clean single-source-of-truth: GEMINI_MODELS/MODEL_RATES gate what can be routed to (an unpriced model is unreachable), modelForAction/modelForReflectionReply implement exactly the table in the PR body, and geminiUrl centralizes the URL. DEFAULT_ACTION_MODEL and the Socratic override (hint/ask only, not propose/review) match the description and are covered by models.test.ts.
• spend-ledger.ts's ratesFor(model) correctly prices per-model with AI_SPEND_{INPUT,OUTPUT}_USD_PER_MTOK as an opt-in override (usdNumOpt in env.server.ts:27-30, unset by default) — confirmed no other call site still reads those two vars as non-optional.
• route.ts and reflection-reply.ts both resolve model once and thread it through the fetch URL, the error/empty-output logs, and recordAiSpend — can't drift apart. Fail-closed on a routed-model 404 is real: partner-route.test.ts's new test asserts fetchSpy called exactly once and recordAiSpend never called on the 404 path (route.ts:498-516) — no silent fallback model.
• Socratic override correctly re-derived per turn (route.ts:398-401) from the same socratic flag already used for maxTokensFor, and the test at partner-route.test.ts:1345 (ask → flash-lite, propose → 3.6-flash on the same tier) matches §4.2 as described.
• Traced every recordAiSpend/ratesFor call site — model param threaded everywhere it's called (route.ts, reflection-reply.ts, and both test files), no stale call left on the old 4-arg signature.
• apps/web/CLAUDE.md env-var table updated in the same commit to document the 6 new AI_MODEL_* overrides and the changed meaning of AI_SPEND_{INPUT,OUTPUT}_USD_PER_MTOK — matches the code.
• Stale-comment corrections (route.ts old header, degradedMaxTokens doc comment) read consistent with the AIE-04/AIE-05 record described in the PR body.

Non-blocking

1. Misleading test name — apps/web/src/app/api/ai/__tests__/partner-route.test.ts:1362, "no action still calls the old pinned gemini-3.5-flash URL". The body actually asserts the opposite (.not.toContain(...gemini-3.5-flash:generateContent...)) across hint/ask/propose. Assertion is correct, title just reads backwards — worth a rename for a future reader skimming test output. Fix this →
2. Pre-existing, unrelated to this PR: route.ts:549-552's conservative spend-fallback estimate calls maxTokensFor(action) without { socratic }, unlike the actual generation call at route.ts:485-486. It only fires when usageMetadata is absent, and the effect is a conservative over-estimate on Socratic turns (never under-reports), so it's not a correctness bug — just noting it in case it's worth tightening in a follow-up.
3. AI_MODEL_* overrides are read directly via process.env in models.ts rather than through the serverEnv zod schema (unlike AI_SPEND_*). This matches the existing AI_PARTNER_DEBUG precedent in the same route file, so it's consistent with prior art, not a new pattern — flagging only for anyone expecting all env access centralized in env.server.ts.

Verification note

This sandbox has no pnpm/node_modules installed, so I could not independently run pnpm typecheck/test/lint to confirm the PR body's reported "240 files / 2202 tests passed" and clean typecheck/lint. I traced the diff statically instead (routing table ↔ pricing sheet ↔ call sites ↔ tests) and everything is internally consistent. CI (CI, Lighthouse) was still in progress at review time — that remains the hard gate per repo policy, and this review is advisory only.