Skip to content

feat(router): reject previous_response_id on a router collection - #3497

Open
Bekhouche wants to merge 1 commit into
lemonade-sdk:mainfrom
Bekhouche:feat/router-response-chain-guard
Open

feat(router): reject previous_response_id on a router collection#3497
Bekhouche wants to merge 1 commit into
lemonade-sdk:mainfrom
Bekhouche:feat/router-response-chain-guard

Conversation

@Bekhouche

Copy link
Copy Markdown
Collaborator

Summary

Reject previous_response_id on a router collection. /v1/responses now
returns 400 with router_response_chain_unsupported when the addressed model
is a collection.router, before dispatch and before any model load. A response
chain is state held inside the backend that created it, so it cannot survive a
turn routing to a different candidate than served the last one; previously the
field was ignored everywhere in src/ and the selected candidate silently
answered without the chain's context.

Part of #2957 (Guard 2 of 3), not Fixes. Guard 1 needs a route_policy
schema key, which would conflict on schema-lock.json with #3169 and #3339
while both are open; Guard 3 belongs on top of the candidate-filter framework
#3339 introduces. Please leave #2957 open on merge.

Scope

src/cpp/server/server.cpp — the guard at the top of handle_responses, a
response shaper mirroring set_router_residency_conflict_response, and
router_collection_info() extracted from apply_router_collection_dispatch so
the guard and dispatch cannot disagree about what counts as a router
collection. error_types.h / server.h — the error code and the helper's
declaration. Plus docs/api/openai.md and test/server_router.py.

Testing

C++ build + full cpp-ci CTest suite (69/69). Three new cases in
server_router.py covering the 400's shape, a concrete-model control, and an
empty-string control.

Also checked end-to-end against a live lemond, using a collection whose
components point at nonexistent checkpoints so the loader always fails: only
the collection with a non-empty previous_response_id short-circuits with the
400, while an empty string, an absent field, a concrete model, and
/chat/completions on the same collection all reach the loader instead.

Documentation

docs/api/openai.md — Limitations note on POST /v1/responses.

Breaking Changes

Narrow: that one request shape previously returned 200 and now returns 400. The
old success answered without the chain's context, and the error names both
alternatives — resend the full input, or address a concrete model.

AI-assisted contribution

Yes.

A response chain is state held inside the backend that created it, so
`previous_response_id` cannot survive a turn routing to a different candidate
than served the last one. `/v1/responses` now returns 400 with
`router_response_chain_unsupported` when the addressed model is a
collection.router, before dispatch and before any model load, rather than
letting a new candidate answer without the chain's context.

The check reuses a single `router_collection_info()` helper that dispatch also
calls, so the guard and dispatch cannot disagree about what counts as a router
collection. Scoped narrowly: an empty or non-string `previous_response_id`, a
concrete model, and every other endpoint are all unaffected.

Guard 2 of lemonade-sdk#2957. Guards 1 and 3 are deliberately left out — the tool-loop
condition needs a schema key, and responses-capability filtering belongs on
top of the candidate-filter framework lemonade-sdk#3339 is introducing.
@Bekhouche Bekhouche self-assigned this Sep 3, 2026
@github-actions github-actions Bot added area::api HTTP REST API surface and route handlers enhancement New feature or request labels Sep 3, 2026
@Bekhouche
Bekhouche marked this pull request as ready for review September 3, 2026 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area::api HTTP REST API surface and route handlers enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant