fix: CUDA forward compat for GPU arch checks (Ada sm_89 falsely rejected) - #23
Open
utsmok wants to merge 1 commit into
Open
fix: CUDA forward compat for GPU arch checks (Ada sm_89 falsely rejected)#23utsmok wants to merge 1 commit into
utsmok wants to merge 1 commit into
Conversation
The GPU compatibility checks in both setup.py and launcher.py used
exact-match against torch.cuda.get_arch_list(), which falsely rejects
Ada GPUs (sm_89) because PyTorch only ships sm_86 and sm_90 kernels,
skipping sm_89.
Ada (sm_89) is fully backward-compatible with Ampere (sm_86) kernels.
CUDA guarantees that a GPU with compute capability X.Y can execute
code compiled for any architecture <= X.Y.
The same issue affects any architecture that PyTorch skips in its
build targets (e.g., sm_87 Orin) but that can run a lower compiled
arch natively.
Changes:
- setup.py: _unsupported_torch_devices() now compares parsed
(major, minor) tuples instead of exact string match
- launcher.py: _check_gpu_deps() same fix
- tests: updated for new semantics, added sm_89 forward-compat test
Fixes: RTX 4060 Ti, RTX 4070, RTX 4080, RTX 4090, RTX 4090 D,
and all other Ada (sm_89) GPUs falsely rejected by setup.
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Updates GPU architecture compatibility checks to account for forward compatibility when determining whether a CUDA-enabled PyTorch build supports the visible GPUs.
Changes:
- Update
_unsupported_torch_devices()to treat a GPU as supported if PyTorch includes any compiled arch <= the device’s compute capability. - Mirror similar forward-compat parsing/logic in launcher dependency checks.
- Refresh and expand tests to cover “genuinely missing SM” and a forward-compat scenario; adjust CUDA wheel selection expectations.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| tests/test_setup.py | Updates/expands tests for unsupported device detection and expected CUDA wheel tag/url. |
| python/krasis/setup.py | Implements forward-compat-aware GPU support detection in _unsupported_torch_devices(). |
| python/krasis/launcher.py | Updates runtime GPU dependency checks to use forward-compat-aware arch parsing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+380
to
+390
| # Parse arch tokens like "sm_86" into (major, minor) tuples. | ||
| compiled_caps: List[Tuple[int, int]] = [] | ||
| for tok in arch_list_raw: | ||
| t = str(tok) | ||
| if t.startswith("sm_") or t.startswith("compute_"): | ||
| v = t.split("_", 1)[1] | ||
| try: | ||
| major, minor = int(v[0]), int(v[1]) if len(v) > 1 else 0 | ||
| compiled_caps.append((major, minor)) | ||
| except (ValueError, IndexError): | ||
| pass |
Comment on lines
+2800
to
+2816
| # Parse compiled arch tokens (e.g. "sm_86") into (major, minor). | ||
| compiled = [] | ||
| for tok in arch_list: | ||
| t = str(tok) | ||
| if t.startswith("sm_") or t.startswith("compute_"): | ||
| v = t.split("_", 1)[1] | ||
| try: | ||
| compiled.append((int(v[0]), int(v[1]) if len(v) > 1 else 0)) | ||
| except (ValueError, IndexError): | ||
| pass | ||
| for i in range(torch.cuda.device_count()): | ||
| major, minor = torch.cuda.get_device_capability(i) | ||
| token = f"sm_{major}{minor}" | ||
| if token not in arch_list: | ||
| dev_cap = (major, minor) | ||
| # GPU can run any kernel compiled for arch <= its capability. | ||
| if compiled and not any(ca <= dev_cap for ca in compiled): | ||
| name = torch.cuda.get_device_properties(i).name | ||
| unsupported.append(f"GPU {i} {name} ({token})") | ||
| unsupported.append(f"GPU {i} {name} (sm_{major}{minor})") |
Comment on lines
+32
to
+43
| def test_unsupported_torch_devices_reports_genuinely_missing_sm(self): | ||
| """A pre-Ampere GPU (sm_75) is unsupported when torch only has sm_80+.""" | ||
| probe = { | ||
| "installed": True, | ||
| "cuda_available": True, | ||
| "arch_list": ["sm_80", "sm_86", "sm_90"], | ||
| "devices": [ | ||
| {"index": 0, "name": "NVIDIA GeForce RTX 5090", "capability": "12.0"}, | ||
| {"index": 1, "name": "NVIDIA RTX A4500", "capability": "8.6"}, | ||
| {"index": 0, "name": "NVIDIA GeForce GTX 1650", "capability": "7.5"}, | ||
| ], | ||
| } | ||
| unsupported = setup._unsupported_torch_devices(probe) | ||
| self.assertEqual(unsupported, ["GPU 0 NVIDIA GeForce RTX 5090 (sm_120)"]) | ||
| self.assertEqual(unsupported, ["GPU 0 NVIDIA GeForce GTX 1650 (sm_75)"]) |
Comment on lines
+380
to
+390
| # Parse arch tokens like "sm_86" into (major, minor) tuples. | ||
| compiled_caps: List[Tuple[int, int]] = [] | ||
| for tok in arch_list_raw: | ||
| t = str(tok) | ||
| if t.startswith("sm_") or t.startswith("compute_"): | ||
| v = t.split("_", 1)[1] | ||
| try: | ||
| major, minor = int(v[0]), int(v[1]) if len(v) > 1 else 0 | ||
| compiled_caps.append((major, minor)) | ||
| except (ValueError, IndexError): | ||
| pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
RTX 40 Ada GPUs (sm_89) — including RTX 4060 Ti, 4070, 4080, 4090 — are falsely rejected by both
krasis-setupandkrasisat startup:Even after
krasis-setupcompletes,krasisrefuses to start with the same error.Root Cause
Two locations perform an exact string match of the GPU's SM architecture token against
torch.cuda.get_arch_list():python/krasis/setup.py:366—_unsupported_torch_devices()python/krasis/launcher.py:2793—_check_gpu_deps()PyTorch 2.12.0+cu126 returns
['sm_50', 'sm_60', 'sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90']— it skipssm_89because Ada GPUs runsm_86(Ampere) kernels natively via CUDA forward compatibility. The exact-match check (sm_89 in arch_list) fails, triggering a--force-reinstallloop in setup that can never succeed.Fix
Replace exact-match with CUDA's actual forward-compatibility rule: a GPU with compute capability X.Y can execute code compiled for any architecture ≤ X.Y. The device is only flagged as unsupported when no compiled arch in the torch binary is ≤ the device's capability.
This correctly handles:
sm_86 ≤ sm_89✓sm_80 ≤ sm_87✓sm_80+-only torch → genuinely unsupported ✓Files Changed
python/krasis/setup.py_unsupported_torch_devices()— parse arch tokens into(major, minor)tuples, compare numericallypython/krasis/launcher.py_check_gpu_deps()— same fixtests/test_setup.pytest_unsupported_torch_devices_accepts_forward_compatfor the sm_89 case