e2e-evolve 2026-07-04: drain the two lane-B blockers (extract $/mo ZodError + followup error_max_turns)#43
Merged
Merged
Conversation
…so the Rule-2 reclassify hatch runs (lane-B $/mo ZodError) + bounded extract retry cap Lane B (Claude OAuth) deterministically ZodErrored on any dealer reply carrying a monthly-payment line ($/mo): the mode-contract superRefine (lease requires money-factor/residual, finance requires APR) was embedded in the EMIT schema, but z.toJSONSchema drops refinements — so the model faithfully emits lease + $389/mo and the emit-boundary parse threw Rule-2 before reclassifyRule2Failures could demote it. Lane A had the same defect (the deepseek payment-only flakiness). - Split DealerReplyQuoteRowLooseSchema (structural .strict(), no refinement) from the refined row schema; the emit schema validates with the loose one. The model-facing JSON schema is byte-identical (refinements never reached it); only the post-parse strictness at the emit boundary changes, so the designed demote-to-unspecified hatch now runs and the persist floor still re-validates the full refined DealerQuoteSchema (a Rule-1 cross-mode bleed still fails closed per-message). - Prompt grounding: a monthly payment is never otd_total — it belongs in that mode's monthly-payment field. - Bounded retry cap: messages.quote_extraction_attempts (migration 0009); a failed message is re-queued only while attempts < MAX_EXTRACT_ATTEMPTS (was unbounded — the armed scheduler retried one poisoned message 46x). - Unify the lane-B fail_reason ledger label (ZodError -> zod_validation, lane A's). - harness/seed.test.ts: apply the full migration journal in both fixture DBs via a shared helper (the hand-picked 0000-0002 list lacked the new column). Live-verified on the claude lane: a real $/mo lease reply now extracts (emit parse OK), the lease-gap row demotes to unspecified (otd null, never ranks), the finance OTD row persists, no ZodError. green GREEN (4054 tests); fresh-context APPROVE + SAFE + drizzle PASS.
…tools on the lane-B prose path (error_max_turns fix) On lane B (Claude OAuth, Agent SDK 0.3.195) one per-thread draft call hitting error_max_turns killed the whole follow-up batch (0 sends). Root cause: claudeOAuth set allowedTools:[] — a no-op (the SDK never forwards an empty list; it is only a permission pre-approval list) — so the CLI still advertised its default toolset; on a long/ambiguous thread the model emitted a tool_use, the deny-all canUseTool aborted it, and under maxTurns:1 the abort counted as a turn -> error_max_turns. - claudeOAuth: on the PROSE path only, set tools:[] (advertise zero tools -> the model cannot emit a tool_use -> the single turn suffices). Not set on the json_schema path (keeps the CLI's internal structured-output delivery). maxTurns/allowedTools/ disallowedTools/canUseTool unchanged — a strictly-more-restrictive fourth lock. - negotiation_followup: a per-thread draft failure is CAUGHT (documented fail-closed degradation, inv #4): the thread keeps draft_body null (never fabricated), is recorded to draftFailures, excluded from the gate + the send by the existing !== null filters, and voiced in the summary; it stays a candidate next round. Every draft failing still rethrows (no zero-target gate). The budget/payment belts stay inside the try — a violating draft is discarded, never gated or sent. - CLAUDE.md inv #4: document the per-thread catch as an approved degradation; note the extract re-queue is now bounded by MAX_EXTRACT_ATTEMPTS. Live-verified on the claude lane: the exact long/noisy draft prompt that reproduced error_max_turns now returns prose, no error. green GREEN; fresh-context APPROVE + SAFE.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Drains the two open blockers the 2026-07-04
/e2e-loop --provider clauderun handed off.B1 —
dealer_reply_extractlane-B ZodError on monthly-payment repliesThe mode-contract
superRefinewas enforced at the EMIT boundary, butz.toJSONSchemadrops refinements, so a reallease + $389/moemit deterministically failed the emit parse before the designedreclassifyRule2Failuresdemote hatch could run (lane A had the same defect = the deepseek payment-only flakiness). Fix: validate the emit boundary with a structural (unrefined) schema so the hatch runs; the persist floor still re-validates the full refined schema (Rule-1 bleed still fails closed). Plus a bounded extract retry cap (quote_extraction_attempts, migration 0009 — the armed scheduler had retried one poisoned message 46×) and lane-Bfail_reasonlabel unified tozod_validation.B2 —
negotiation_followuplane-Berror_max_turnswhole-batch deathallowedTools:[]is a no-op in Agent SDK 0.3.195, so the CLI still advertised its default toolset; a long/noisy thread made the model emit a tool_use, the deny-allcanUseToolaborted it, and undermaxTurns:1the abort =error_max_turns→ the throw killed the whole batch. Fix: advertise zero tools on the prose path (tools:[]) so the model can't emit a tool_use; add an extract-style per-thread draft catch (documented inv #4 degradation — the thread keepsdraft_bodynull, is voiced + ledgered, stays queued; a draft is never fabricated).Verification
RUN_UI_FUNCTIONAL=1 green.shGREEN (4054 tests, ui:functional lane included).$/moreply extracts, the lease-gap demotes tounspecified, both rows persist, no ZodError; B2 — the exact long/noisy draft prompt that reproducederror_max_turnsnow returns prose.