Harden response-side spec drift: promote additive warnings to hard-fail

[TexasCoding]

Goal

Promote response-side OpenAPI/AsyncAPI spec drift detection from UserWarning to hard-fail. Closes the regression path that allowed Balance.balance_dollars to ship missing in v2.1.0 despite five rounds of review.

This is the single highest-leverage open item on ROADMAP.md (lines 50–54). v2.1's contract tests cover request body drift (hard-fail) but only warn for response model drift, which is why the missing required field on the Balance response model slipped through.

Current state (as of v2.1.0)

Running uv run pytest tests/ -W error::UserWarning --ignore=tests/integration produces 60 failing assertions:

Bucket	Count	Disposition
Additive drift on REST response models	17 models, 46 fields	42 to backfill, 4 to allowlist
Required-mismatch (spec required, SDK Optional) on REST	18 models, 126 fields	Out of scope — see PR5
WS additive drift	11 payloads, 34 fields	33 to backfill, 1 to allowlist
WS required-mismatch	11 payloads, 78 fields	Out of scope
WS payload without contract map entry	1 (ErrorPayload)	Register in PR5

Plus a related ROADMAP item: WS envelope models (FillMessage, TickerMessage, etc.) and helpers (ErrorMessage, SubscribedMessage, …) still use extra="ignore". #143 made WS payloads extra="allow" but missed envelopes. Closed in PR5.

Plan — 6 PRs

#	Issue	Title	Depends on	Effort
PR0	#158	Wire response-side drift through EXCLUSIONS map (no behavior change)	—	~80 lines test infra
PR1	#159	Backfill v3.18.0 fields on Market / Order / Fill (+23 fields)	#158	~80 model + ~30 tests
PR2	#160	Backfill v3.18.0 fields on events / portfolio / historical / incentive (+9 fields)	#158	~50 model + ~20 tests
PR3	#161	Backfill v3.18.0 fields on communications + order_groups (+8 fields)	#158	~30 model + ~15 tests
PR4	#162	Backfill v0.14+ fields on WS payloads (+33 fields including ms timestamps)	#158	~90 model + ~30 tests
PR5	#163	Promote additive drift to hard-fail; register ErrorPayload; WS envelope extra="allow"	#159–#162	~50 lines

Tracking checklist

• PR0: test infra — wire response-side drift through EXCLUSIONS map #158 — PR0 test infra
• PR1: backfill v3.18.0 fields on Market / Order / Fill #159 — PR1 Market / Order / Fill backfill
• PR2: backfill v3.18.0 fields on events / portfolio / historical / incentive_programs #160 — PR2 events / portfolio / historical / incentive backfill
• PR3: backfill v3.18.0 fields on communications / order_groups #161 — PR3 communications / order_groups backfill
• PR4: backfill v0.14+ fields on WebSocket payloads (ms timestamps, outcome/book_side) #162 — PR4 WS payload backfill
• PR5: promote additive drift to hard-fail; register ErrorPayload; WS envelope extra=allow #163 — PR5 flip warn → fail, ErrorPayload, WS envelope policy

PR0 is the load-bearing prerequisite. PR1–PR4 are independent and parallelizable (different files, different models). PR5 is the final flip and pulls the trigger only once the backfills are in place.

Design decisions captured

1. Reuse EXCLUSIONS for response drift. No parallel RESPONSE_EXCLUSIONS — the existing (fqn, name) → Exclusion(reason, kind) shape already supports it. Existing kind values (spec_deprecated, wire_normalization) match the semantics; no new kinds needed.

2. Required-but-Optional axis stays as warning-only. It's a 204-entry policy question, not a v2.1-style hole. The SDK is intentionally permissive because demo has been known to omit spec-required fields. Separate decision; file a follow-up if/when the team wants to tighten.

3. *_ms timestamp fields are surfaced as int, not datetime. The AsyncAPI spec promotes them as the primary form going forward; latency-sensitive consumers want the integer form directly. Existing ts: datetime fields stay.

4. outcome_side / book_side are typed as Literal. Existing SideLiteral / BookSideLiteral aliases in kalshi/models/orders.py already define the values; reuse them.

5. Allowlist is small and reasoned. 5 REST + 1 WS entries cover spec-deprecated fields and one wire-normalization case (Orderbook.orderbook_fp — SDK reshapes the nested object into yes/no level lists at the resource layer).

Allowlist (lands in PR0)

REST:
  Market.response_price_units                          spec_deprecated
  Order.action                                          spec_deprecated
  CreateRFQRequest.target_cost_centi_cents              spec_deprecated
  CreateOrderRequest.sell_position_floor                spec_deprecated
  Orderbook.orderbook_fp                                wire_normalization
  CreateRFQRequest.contracts_fp                         wire_normalization

WS:
  TickerPayload.time                                    spec_deprecated

Verification at each step

• After PR0: uv run pytest tests/test_contracts.py -W error::UserWarning should show ~53 failures, not 60 (60 baseline minus 7 allowlisted).
• After each backfill PR: a specific subset of test_additive_drift[*] cases pass without warnings.
• After PR5: uv run pytest tests/test_contracts.py -v passes with zero UserWarning on additive checks. Deleting a backfilled field locally causes a real failure.

Related

• ROADMAP.md "Next milestone" — Response-side spec drift detection (lines 50–54)
• ROADMAP.md — WS envelope extra="allow" (closed by PR5)
• Audit response models for consistent extra= policy #114 — uniform extra="allow" policy (this stack extends it to WS envelopes)
• feat: sync spec to v3.18.0 + fix asyncapi drift false alarm #155 — v3.18.0 spec sync that introduced most of the backfill items
• The v2.1.0 Balance.balance_dollars regression that motivated this work

Out of scope

• Required-but-Optional drift policy (separate; ~204 entries)
• Nested $ref resolution in body-schema drift (Resolve nested $ref pointers in request-body schema drift check #53)
• Verify json={} workaround under prod credentials (Verify json={} workaround under prod credentials (subaccounts.create, communications.confirm_quote, order_groups.reset/trigger) #45)
• Any v2.0 polish backlog sub-items from #106