fix: capture-poisoned engine wedge + planner-driven KV pool sizing

[kekzl]

Fixes #874.

Problem 1 — capture failure permanently wedged the server (#874)

Loading a GGUF Q*_K MoE model (Ornith-1.0-35B Q4_K_M, qwen3_5_moe) with default settings: the first prefill ≥ ~1700 tokens hit the 2048-chunk prefill graph capture, cuBLAS failed under capture (status 14, VRAM nearly exhausted), the MoE host-args guard threw — and the exception unwound past the open capture. cudaStreamEndCapture was never called, so the prefill stream stayed in capture state forever: every subsequent request failed with internal_error ("operation failed due to a previous error during capture") while /health kept reporting ok.

Fixes (defense in depth):

• CudaGraphRunner::execute: try/catch around the captured forward — abort the capture (CudaGraphCapture::abort_capture()), mark capture_failed_, re-run eagerly. The request succeeds; capture is permanently disabled for that runner.
• Engine::invalidate_graphs(): also invalidates the prefill runner and closes any stray open capture on the prefill stream (abort_stream_capture) — covers future mid-capture throw paths.
• Scheduler gate: moe_prefill_uncapturable() — GGUF Q*_K MoE models (no CUTLASS NVFP4 grouped workspace) always take the legacy host-args MoE prefill, which can never be captured; don't attempt it (saves a wasted forward per chunk).
• /health now returns 503 unhealthy once the worker has declared the CUDA context poisoned and stopped (previously hard-coded ok).

Problem 2 — KV pool ate all post-weight VRAM, weight caches built 0 tensors

compute_vram_budget reserved only the nvfp4 heuristic (~459 MiB) before letting the KV backstop fill all remaining VRAM. Q4_K sources are not nvfp4_beneficial, so the heuristic misses them entirely — but the StoragePlanner (source-aware) routes them to the FP16 cache (4088 MiB projected, logged as "diagnostic only (5.1.5)"). Result on Ornith-35B Q4_K_M: KV pool 9970 blocks / 6.2 GiB (~159k tokens nobody asked for), FP16/NVFP4 caches 0 tensors, decode 11 tok/s via on-the-fly dequant.

Fix: mirror the existing native-NVFP4 reserve pattern for GGUF sources — reserve the planner projection (plus phase reserve) before the KV backstop, gated on a 2× planner/heuristic divergence so tuned Q6_K/Q8 configs keep their pools unchanged, and capped so one full max_seq_len sequence always fits (concurrency stays bounded by scheduler admission).

Verification (local GPU, RTX 5090)

• New tests: CaptureAbort.ThrowDuringCaptureLeavesStreamUsable (test-kv), VramBudgetReserve.* (test-e2e) — green; full make test-unit + make test-gpu green.
• E2E Ornith-35B Q4_K_M, default settings (graphs ON): previously deterministic wedge on the long-context probe — now degen_suite 33 checks / 0 FAIL including full long-context, server stays healthy afterwards. Startup now shows planner-driven weight-cache reserve 5410.8 MiB, KV 30720→2048 blocks, caches build (fp16=132, nvfp4=27, nvfp4_moe=15 tensors). Decode 11 → 22 tok/s on the same probe.
• Perf gate: decode −4.67% on Qwen3-8B-Q8 flagged as depressed-host by the gate itself (mem 10401 MHz / 342 W during bench, Q8_0 GGUF perf drift since 2026-05-29 baseline: tg128 259->238 (-8%), pp512 8260->7150 (-13%) #526 signature) and degraded to WARN; prefill +6.7%/+74% PASS. The graphs ON/OFF gate read 1.10× on this branch — cross-checked against the v0.16.2 release binary on the same host: 0.75×, i.e. host-state noise, not this diff. No intentional perf change to baseline configs (reservation is gated off for heuristic-covered sources — pinned by VramBudgetReserve.BeneficialSourcesKeepFullKvPool).

🤖 Generated with Claude Code