fix(endpoints): update-endpoint wipes GPU SKU exclusions on v2 (#63) - #70
Draft
justinwlin wants to merge 8 commits into
Draft
fix(endpoints): update-endpoint wipes GPU SKU exclusions on v2 (#63)#70justinwlin wants to merge 8 commits into
justinwlin wants to merge 8 commits into
Conversation
…#63) Split from #67 (GPU half). v2 REST cannot represent a gpuIds exclusion, so every v2 update-endpoint reads gpuIds via GraphQL and re-asserts exclusions the PATCH dropped; get-endpoint gains includeGpuIds; set-endpoint-gpus and update-endpoint refuse selections the server would silently discard; create/update-pod reject a lone volumeInGb/volumeMountPath.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
justinwlin
marked this pull request as ready for review
July 30, 2026 20:37
The gate fired on backend.version alone, so a CPU serverless endpoint — which has no gpuIds and cannot suffer the exclusion loss the gate exists to prevent — was refused with a demand for GPU pools it cannot hold. When a v2 update names no GPU replacement, read the endpoint on the same v2 REST host first: an unambiguous CPU endpoint (cpu set, gpu absent) proceeds; a GPU config, an unrecognizable reply, or a failed read keeps the fail-closed refusal, which now also reports the current pools/count. Compute type is fixed at creation, so unlike an exclusion pre-read this read cannot go stale between the read and the PATCH. Verified live against the dev facade: the v2 PATCH pipeline itself still rejects CPU-endpoint updates (400 "gpuId(s) is required for a gpu endpoint"), so the tool wraps that 400 with the CPU context and the RUNPOD_REST_VERSION=v1 escape (verified working) instead of parroting an error about a GPU endpoint the caller does not have. Client-side the exemption stands, so no MCP change is needed when the facade gains CPU update support.
justinwlin
marked this pull request as draft
August 3, 2026 17:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split from #67 — the GPU-exclusion half. The Windows install half (#56) is in #69. Tracked in CON-870.
Closes #63
TL;DR
Two server-side bugs in the v2 REST API make
update-endpointdangerous or broken, and this PR makes the MCP safe against both today. The real fix is server-side in rphttp2#175 (the v2 REST facade, API team's repo); once that deploys, most of this PR's guard can be relaxed.The two bugs (both confirmed live on dev)
AMPERE_16,-NVIDIA RTX A4500means "any AMPERE_16 card except the A4500". The v2 REST shape only hasgpu.pools, with nowhere to keep the-entries, and the facade rebuilds the whole selection from that lossy shape on every PATCH. Repro: patched only{"image": ...}on an endpoint with two exclusions → both gone, response showed nothing.instanceIds, the backend derives compute type from them, so every CPU PATCH fails with400 "gpuId(s) is required for a gpu endpoint"— misidentifying the endpoint. (Found while live-testing this PR's CPU handling.)Root cause for both is one flaw in rphttp2's PATCH merge — it merges from the REST view instead of the raw GraphQL fields. Full write-up in rphttp2#175.
What this PR does about it (client-side)
GPU endpoints — fail closed instead of losing data silently. A v2 update must state the complete replacement selection: non-empty
gpuPoolIds, positivegpuCount, andreplaceGpuSelection: true. The refusal reports the endpoint's current pools/count so the caller can comply in one step. Deliberately not a pre-read-and-repair: no GraphQL read can make the later REST PATCH atomic, and a compensating write has its own damage modes (details in the changeset).CPU endpoints — exempt, verified by a read. They have no GPU selection to lose, so the gate would be pure obstruction. A v2 update naming no GPU fields first GETs the endpoint on the same host: unambiguous CPU (
cpuset,gpuabsent) proceeds; anything else keeps the refusal. This read can't go stale — compute type is fixed at creation, unlike exclusions. Since the deployed facade still 400s the CPU PATCH itself (bug 2), that error is rewrapped with the real cause and the workingRUNPOD_REST_VERSION=v1fallback.Plus:
get-endpoint includeGpuIds— first way to read exclusions over v2 (read-only GraphQL enrichment, fails closed across unverified REST/GraphQL host pairs)set-endpoint-gpusvalidates exclusions against the catalog (a typo'd exclusion is accepted by the API and excludes nothing), refuses inexpressible combos, requires acknowledgement for theworkersStandbyresetcreate-endpointrejects v1-only fields instead of silently dropping them (computeType:'CPU'was silently creating a GPU endpoint billed at GPU rates)How it was tested
api.runpod.dev/v2), real endpoints created and deleted:currentGpuSelection, endpoint confirmed unwrittenAfter rphttp2#175 deploys
Follow-up: relax the GPU gate to a plain passthrough (unrelated PATCHes stop needing the replacement selection) and drop the CPU 400 rewrap. The CPU read-exemption and the read-only/validation tooling stay useful regardless.