refactor(historical): drop on-miss upstream resolution, serve from table only

[matheus1lva]

Summary

Exact and batch historical routes no longer resolve table misses upstream (the 5s deadline / 10-cap path behind #50 and #52). They answer from token_prices only. Warmup and backfill fill the table.

Changes

• exact: a DB miss returns NOT_FOUND. batch: missing pairs are omitted and the response is marked partial. Both handlers drop the registry param and no longer read env.
• Remove the route-side budget machinery: the 10-resolution cap, the 5s deadline, the round-robin interleave, and resolveMisses.
• Remove persistResolvedPrices, ResolvedPriceRecord, and the observation-window guard from routes/historical/shared.ts.
• Negative cache TTLs: 404 and partial batch/range go from 1h (max-age=3600; 404 also had stale-while-revalidate=14400) to public, s-maxage=300, max-age=300.
• Docs: README, lander, docs/routes.md, and src/sources/README.md now say table-only and name each source's writer. chainlink and defillama-alias have no writer.

Kept

The registry and its sources are untouched: HistoricalSourceRegistry.resolveBatch, the DeFiLlama getBatchHistoricalPrices source method, and the historicalSourceRegistry() factory stay in place. Nothing on the request path calls resolveBatch any more. Removing it is a separate decision.

Risk / observable output

• Exact miss: was 200 with a live price (or 500 INTERNAL_ERROR on DeFiLlama 5xx). Now always 404.
• Batch miss: was filled for up to 10 pairs per request. Now omitted + partial.
• ?source=chainlink / defillama-alias: live resolve is gone; only leftover rows match. A token priceable only through Chainlink has no request-path price.
• Browser 404 and partial responses refresh in 5 min, not 1 h. The Worker Cache API still does not store 404s.
• yearn/kong packages/ingest/prices.ts: a batch miss still falls through to exact, with a comment that exact "also resolves upstream". Exact is table-only now; kong already maps 404 → missing → $0. The extra exact call is a wasted 404. Kong's comment is stale; not changed here.
• yearn/yearn.fi holdings: rangeHistorical was already table-only; batchHistorical sparse lookups lose the 10 live fills. Missing prices already surface as missingPrice.
• Any token the warmup/backfill manifests do not cover stays a 404 until a job writes it.

Test plan

• Exact miss: NOT_FOUND, no fetch. Worker 404 header is the literal public, s-maxage=300, max-age=300.
• Closed-day hit immutable; today hit short-lived.
• Incomplete past batch omits the missing day, uses the same 300s header, and does not INSERT.
• Current-day integration miss is NOT_FOUND with no upstream call.

Not run here: tsc, full suite, live Enso (ENSO_API_KEY), prod-shaped DB / live RPC.

12 files, +110 / -1041.

[matheus1lva]

/review-workflow

[murderteeth]

Summary

Removes request-path upstream resolution from the exact and batch historical routes; both now answer from token_prices only, with a DB miss returning NOT_FOUND (exact) or an omitted pair plus a partial header (batch). Docs for those two routes are updated; the registry and its sources are left in place with no production caller.

Issues

• src/routes/historical/exact.ts:21 - Cacheable false 404 for not-yet-warmed rows (medium) — a miss that only means "the hourly warmup has not run yet" (for example any current-day lookup shortly after UTC midnight) is now returned as a 404 carrying a public one-hour max-age, so clients keep seeing "no price" well after the row lands. Before this PR that request resolved live and returned 200, and the not-found cache policy was chosen when 404 meant upstream had nothing.
  • Done when: a 404 the exact route emits for a key the warmup/backfill jobs will fill does not instruct clients to cache it beyond the warmup cadence, or the docs state that historical 404s are no longer permanent and say how long a consumer should wait before retrying.
  • Provenance: pre-existing (32f5c14)
• docs/routes.md:69 - Docs describe writers that do not exist (medium) — the new sentence says chainlink and defillama-alias rows are produced by the offline jobs, but no script references the Chainlink source at all, and the gap backfill stores alias hits under defillama. This PR deleted the only writer of both source names, so ?source=chainlink and ?source=defillama-alias can only ever match rows stored before this deploy, and a token priceable only via Chainlink (the #42 feature) now has no path to a price on any route or job.
  • Done when: the source section names, for each listed source, the code path that actually writes its rows, or states plainly that chainlink and defillama-alias currently have no writer and that Chainlink-only tokens are not priced.
  • Provenance: d9880d1
• README.md:55 - Stale fallback description (low) — the README still says single-token historical lookups try DefiLlama, Chainlink, derived and DefiLlama alias "when the DB has no record", and contrasts that with DB-only batch/range. An operator reading it expects a 404 to self-heal on the next request; it never will until a backfill runs.
  • Done when: the "Price sources" section describes how historical rows reach the table after this PR and agrees with docs/routes.md on the exact route being table-only.
  • Provenance: pre-existing (6ee8f66)
• src/sources/README.md:95 - Contributor guide points at a dead path (low) — the source-authoring guide says registering a historical source in createHistoricalSources makes the registry serve it, and forbids touching routes. After this PR nothing in production instantiates that registry, so a source added by the book passes its unit tests and changes nothing in production.
  • Done when: a contributor can tell from the guide which production path consumes a registered historical source, or the guide states that historical sources have no request-path consumer and names the offline job that must be extended instead.
  • Provenance: pre-existing (d1736d8)

Verdict

COMMENT

---

How This Was Reviewed

Reviewed with the review-pr-workflow skill —
5 review lenses, each finding independently verified by claude. 0 candidate findings were refuted and dropped.

[matheus1lva]

/review-workflow

[github-actions]

Review started (review-pr-workflow): https://github.com/yearn/yearn-prices/actions/runs/33803895095

[github-actions]

Summary

Historical exact and batch routes now read token_prices only — the request-path source registry, its persistence, resolution budget and deadline are deleted (~1,040 lines) — and the negative cache TTLs drop to 5 minutes to match. The removal is clean and the docs/lander were updated alongside it.

gitconfig-mask: unreadable

Not verified in this run: lint, typecheck and the test suite could not execute (no node_modules, no network); visual verification of the lander change was skipped for the same reason. Findings below come from reading the code.

Issues

• test/prices-historical.test.ts:138 - Vacuous guard on the new TTLs (medium) — the test added to protect this PR's central change passes on the values the change removed, so a future edit could restore the hour-long negative caching and CI would stay green. The header assertion compares against the constant itself, and the numeric one allows 3600 and ignores stale-while-revalidate.

  • Done when: setting either negative-TTL constant to a policy whose s-maxage, max-age or stale-while-revalidate reaches the hourly warmup cadence fails the suite, and the failing assertion reads a header off a response the route produced rather than the constant.
  • Provenance: 065d0f2

• README.md:3 - README contradicts itself (low) — the opening paragraph still tells readers the worker persists historical prices to Postgres, which stopped being true in this PR; fifty lines below, text this PR added says the opposite. The equivalent sentence in the lander was rewritten, so only the README headline is stale.

  • Done when: no sentence in README.md attributes historical-price persistence to the worker, and the opening paragraph agrees with README.md:55-57 on which providers the worker queries at request time.
  • Provenance: pre-existing (a7c1899)

Verdict

COMMENT

---

How This Was Reviewed

Reviewed with the review-pr-workflow skill —
5 review lenses, each finding independently verified by claude. 5 candidate findings were refuted and dropped.

[matheus1lva]

/review-workflow

[github-actions]

Review started (review-pr-workflow): https://github.com/yearn/yearn-prices/actions/runs/33872495788

[github-actions]

Summary

Historical routes now serve from token_prices only: request-path upstream resolution and persistence are removed from the exact and batch routes, both negative cache TTLs drop to 300s, and the docs, lander, and source-authoring guide are updated to say which offline job writes each source. The dead-code retention of the historical registry is documented rather than hidden, which is the right call.

Lint, typecheck, and tests were not runnable here — no network and no node_modules (biome: not found, npx vitest blocked by a registry 403). Treat CI status as unverified; all findings below come from reading code.

Issues

• docs/routes.md:146 - Docs promise misses self-heal, but most never will (high) — the new docs tell callers a historical 404 or omitted batch pair is temporary and clears within the hourly warmup. The only scheduled writer covers Kong origin=yearn vault tokens and their underlyings over a rolling 7-day window; any other token, or any older day, stays missing until an operator runs a manual backfill. Because the shortened 300s negative TTL is not edge-absorbed on the 404 path, those clients now re-hit the worker and DB every 5 minutes instead of hourly, forever. Pre-PR the same request filled itself.
  • Done when: for a token outside the Kong yearn-origin vault list, or a UTC day older than the warmup window, documented behavior matches actual behavior — either a scheduled job covers what a client can legitimately request, or the route docs state plainly that such a 404 is not transient and should not be polled.
  • Provenance: d9880d1

Verdict

COMMENT

gitconfig-mask: sentinel

---

How This Was Reviewed

Reviewed with the review-pr-workflow skill —
5 review lenses, each finding independently verified by claude. 6 candidate findings were refuted and dropped.

[murderteeth]

Summary

Exact and batch historical routes now answer from token_prices only, the request-path registry call, persistence and budget machinery are gone, and both negative TTLs drop to 300s. Lint, typecheck and the unit suite (453 tests) pass on the PR head. The prior review's four findings (cacheable false 404, phantom chainlink/alias writers, stale README fallback text, contributor guide pointing at a dead path) are all addressed at HEAD.

Issues

• docs/routes.md:146 - Docs promise misses will self-heal, but most never will (medium) — the exact-route, batch-route and caching sections say a 404 or omitted pair "is not permanent" and gaps "are filled by the offline warmup and backfill jobs". The only scheduled writer covers Kong origin=yearn vault and underlying tokens over the trailing 7 days; the gap backfill is manual and manifest-scoped. Any other token or older day stays a 404 forever, and a consumer following the docs polls a miss that never fills (404s are not edge-stored, so each poll reaches the worker and Neon). The README and the PR body already say this; the route reference contradicts them.

  • Done when: the exact-route, batch-route and caching paragraphs of docs/routes.md name what the scheduled warmup actually covers and state that a token or day outside that set only gets a row through a manually run backfill, so a reader cannot infer an arbitrary 404 will resolve on its own.
  • Provenance: d9880d1 (lines 146, 288), 842d950 (line 395)

• docs/routes.md:395 - 300s negative TTL overstated for today-touching batch and range (low) — the new caching paragraph and the matching comment in src/cache/headers.ts say an omitted pair carries a 5-minute negative TTL. Whenever a batch or range includes today's UTC day, the today policy wins before the partial check runs, so a missing closed past day in that response keeps the 1h browser max-age plus 4h stale-while-revalidate. The bullet list at line 391 scopes the 300s header to past days correctly; the prose and the comment do not. The header precedence is pre-existing; the removal of the live fill makes the omission the normal outcome.

  • Done when: docs/routes.md line 395 and the src/cache/headers.ts comment state that the 300s negative TTL applies only to batches and ranges made entirely of closed days, or today-touching partial responses carry a browser max-age at or below the warmup cadence.
  • Provenance: 842d950

• test/prices-batch-range.test.ts:69 - Batch route's table-only contract has no regression guard (low) — the exact route's tests assert no fetch occurs on a miss; the batch tests never stub or observe fetch. The old batch tests guarded this through an injected registry mock, and this PR removed that injection point with the assertion. A reintroduced request-path resolve in the batch handler that fails in a networkless sandbox would still omit the pair, set the partial header and skip the INSERT, so this test would keep passing.

  • Done when: a unit test for handleBatchHistorical fails when the handler performs any network call on a table miss, without depending on network availability.
  • Provenance: d9880d1

Verdict

COMMENT

---

How This Was Reviewed

Reviewed with the review-pr-workflow skill —
5 review lenses, each finding independently verified by claude. 0 candidate findings were refuted and dropped.

[murderteeth]

Summary

Exact and batch historical routes now answer from token_prices only, the request-path registry call, persistence and budget machinery are gone, and both negative TTLs are 300s. All three findings from the previous round are addressed at HEAD: docs/routes.md and the src/cache/headers.ts comment now scope warmup coverage to Kong origin=yearn vaults and underlyings over the trailing 7 days and scope the 300s negative TTL to closed-day-only batches and ranges, and the batch route now has a stubbed-fetch regression guard alongside its no-INSERT check. Lint, typecheck and the unit suite (52 files, 453 tests) pass at HEAD; the lander copy renders as intended.

Issues

None.

Verdict

APPROVE

---

How This Was Reviewed

Reviewed with the review-pr-workflow skill —
5 review lenses, each finding independently verified by claude. 1 candidate finding was refuted and dropped.