When restore fails due to insufficient GPU memory (CUDA_ERROR_OUT_OF_MEMORY, code 2), all subsequent restore attempts fail with CUDA_ERROR_OPERATING_SYSTEM (304) — even after freeing GPU memory. The process is permanently stuck: restore returns 304, unlock returns CUDA_ERROR_ILLEGAL_STATE (401). The only option is to kill the process.
The failed restore appears to partially modify internal driver state that isn't rolled back on failure. We reproduced this by calling cuCheckpointProcessRestore directly via the driver API (not through the cuda-checkpoint binary), so this is a limitation in the underlying driver, not the CLI tool.
Environment
- Driver 580.95.05, CUDA 13.0, Tesla T4 (15GB)
Reproduction
- Start a CUDA process that allocates ~4GB via
cudaMalloc
cuda-checkpoint --action lock --pid $PID → successcuda-checkpoint --action checkpoint --pid $PID → success- Fill GPU memory with a second process (leave ~256MB free)
cuda-checkpoint --action restore --pid $PID → out of memory (state stays CHECKPOINTED)- Kill the memory-filling process (~8.4GB now free)
cuda-checkpoint --action restore --pid $PID → OS call failed or operation not supported on this OS (304, permanently)
Without step 4-5, restore works fine every time.
Expected behavior
A failed restore due to OOM should leave the process in a clean CHECKPOINTED state so restore can be retried once memory is available.
When
restorefails due to insufficient GPU memory (CUDA_ERROR_OUT_OF_MEMORY, code 2), all subsequent restore attempts fail withCUDA_ERROR_OPERATING_SYSTEM(304) — even after freeing GPU memory. The process is permanently stuck: restore returns 304, unlock returnsCUDA_ERROR_ILLEGAL_STATE(401). The only option is to kill the process.The failed restore appears to partially modify internal driver state that isn't rolled back on failure. We reproduced this by calling
cuCheckpointProcessRestoredirectly via the driver API (not through thecuda-checkpointbinary), so this is a limitation in the underlying driver, not the CLI tool.Environment
Reproduction
cudaMalloccuda-checkpoint --action lock --pid $PID→ successcuda-checkpoint --action checkpoint --pid $PID→ successcuda-checkpoint --action restore --pid $PID→out of memory(state stays CHECKPOINTED)cuda-checkpoint --action restore --pid $PID→OS call failed or operation not supported on this OS(304, permanently)Without step 4-5, restore works fine every time.
Expected behavior
A failed restore due to OOM should leave the process in a clean CHECKPOINTED state so restore can be retried once memory is available.