[Bug]: GPT OSS Router GEMM Causing NaNs #38754
Description
Your current environment
Details
reproduced on various commits since b1169d7, on 1xB200. Multiple fresh installs, both cu129 and cu130
🐛 Describe the bug
The custom router GEMM kernel is causing infrequent NaNs in an internal test that poisons the correctness of the EAGLE draft model. It seems to only occur in very specific scenarios involving cuda graphs, prefix caching, and chunked prefills.
Identified root-cause by manual bisection: #37205, originally manifested as acceptance rates of EAGLE3 going to zero. No special flags needed to reproduce, just running gpt-oss-120b on 1xB200 with EAGLE3 enabled.
Unfortunately I cannot share the script to reproduce the failure, it is an internal script for multi-turn chat replay.
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.