-
Notifications
You must be signed in to change notification settings - Fork 55
feat: Add documentation for debugging Numba CUDA programs with CUDA GDB and VSCode #665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: Add documentation for debugging Numba CUDA programs with CUDA GDB and VSCode #665
Conversation
…DB and VSCode The CUDA Toolkit 13.1 release provides beta support for debugging Numba CUDA programs using CUDA GDB and VSCode. This PR includes the documentation for setup and debugging, and an example VSCode workspace that can be used as both a tutorial and as a starting point for other development projects.
Greptile SummaryThis PR adds comprehensive debugging documentation for Numba CUDA programs using CUDA GDB and Visual Studio Code, supporting the beta debugging features released in CUDA Toolkit 13.1. Key additions:
Minor issues found:
The documentation is well-structured, thorough, and provides clear instructions for both GUI and command-line debugging workflows. The example workspace serves as an excellent starting point for users. Confidence Score: 4/5
Important Files Changed
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (2)
-
docs/source/user/debugging.rst, line 135 (link)style: The PYTHONPATH in this documentation snippet (
${workspaceFolder}/../../misc) differs from the actuallaunch.jsonfile which uses${workspaceFolder}. Consider aligning these for consistency. -
docs/source/user/debugging.rst, line 127 (link)logic: The referenced file
numba-cuda/misc/gdb_print_extension.pydoes not exist in this repository. Themisc/directory at the repo root is missing. Is this file intended to be added in a separate PR, or should the documentation be updated to reflect its actual location?
4 files reviewed, 2 comments
| ] | ||
| } | ||
|
|
||
| Debugging Host Python Code |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we call out that we can debug via the CLI, a howto that walks the user through how to debug hello.py via cuda-gdb could be a potentially good addition.
rparolin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, the documentation looks good. I made a minor comment about adding a walk through for cli debugging via cuda-gdb.
docs/source/user/debugging.rst
Outdated
|
|
||
| CUDA GDB supports extensions to the debugger written in Python. The Numba CUDA cuda-gdb pretty printer is such an extension. This is used to provide a more readable representation of Numba CUDA arrays in the debugger. | ||
|
|
||
| The Numba CUDA pretty printer extension is located in the ``numba-cuda/misc/gdb_print_extension.py`` file. This extension is loaded automatically by the ``launch.json`` file at the beginning of the debugging session. The `launch.json` example below assumes that you've opened the `debugging.code-workspace` workspace file in the `numba-cuda/examples/debugging` directory (due to the use of `..`). If the path to the ``gdb_print_extension.py`` file is not correct, simply edit the path to the ``misc/`` directory below. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gdb_print_extension.py appears to be in the numba repo -- not in numba-cuda.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
❯ find . -type f | rg gdb_print
./.pixi/envs/cu-13-0-py313/lib/python3.13/site-packages/numba/misc/gdb_print_extension.py
./.pixi/envs/cu-13-0-py313/lib/python3.13/site-packages/numba/misc/__pycache__/gdb_print_extension.cpython-313.pyc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gdb_print_extension.pyappears to be in thenumbarepo -- not innumba-cuda.
Yes, but to make it work properly with multiple address spaces it needed to be enhanced to use some CUDA specific APIs which are not available in GDB itself. I would suggest including this CUDA aware version of the gdb_print_extension.py file here instead of relying on the Numba repository. PR and merge request for that coming soon (if you concur).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that gdb_print_extension.py needs adding to Numba-CUDA. If it comes in a separate PR, then this PR needs to be merged only after the separate PR, otherwise we'll be publishing instructions that won't work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Opened #692 to add an unmodified copy of the GDB pretty printer from the Numba repository. CUDA specific enhancements will be handled with follow-on PRs / merge requests.
rparolin
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing filepaths in the documentation should be corrected.
docs/source/user/debugging.rst
Outdated
| .. image:: ../_static/starting-debugging.png | ||
| :alt: Starting debugging. | ||
|
|
||
| Because ``breakOnLaunch`` is set to ``true`` in the ``launch.json`` configuration file, the program will automatically stop on the first source line of any launched kernel, which can be useful for getting started debugging. If this is not desired, change ``breakOnLaunch`` to be ``false``. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe something like:
Setting breakOnLaunch to true in the launch.json configuration file causes the program to pause automatically on the very first source line of any kernel that is launched, which is a helpful starting point for the debugging process.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, incorporated in the next revision.
docs/source/user/debugging.rst
Outdated
|
|
||
| The following is a description of the functionality of each of the run control buttons and their corresponding CUDA GDB CLI commands. | ||
|
|
||
| * ``Continue`` will continue the program running until the next breakpoint or the program terminates. Equivalent to the ``continue`` command in CUDA GDB. When pressed, this icon will change to a pause icon. Pressing the pause icon will pause the program and return control to the user for further debugging. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"change to the pause icon"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Incorporated.
docs/source/user/debugging.rst
Outdated
|
|
||
| CUDA GDB supports extensions to the debugger written in Python. The Numba CUDA cuda-gdb pretty printer is such an extension. This is used to provide a more readable representation of Numba CUDA arrays in the debugger. | ||
|
|
||
| The Numba CUDA pretty printer extension is located in the ``numba-cuda/misc/gdb_print_extension.py`` file. This extension is loaded automatically by the ``launch.json`` file at the beginning of the debugging session. The `launch.json` example below assumes that you've opened the `debugging.code-workspace` workspace file in the `numba-cuda/examples/debugging` directory (due to the use of `..`). If the path to the ``gdb_print_extension.py`` file is not correct, simply edit the path to the ``misc/`` directory below. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Found it. There were single backquotes instead of doubles. Addressed in the next version.
|
|
||
| .. code-block:: bash | ||
|
|
||
| cd numba-cuda/examples/debugging |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mmason-nvidia Why didn't I find the examples directory?
test@cudatools-daniel:/daniel$ git clone https://github.com/NVIDIA/numba-cuda.git/daniel$ cd numba-cuda/
Cloning into 'numba-cuda'...
remote: Enumerating objects: 7877, done.
remote: Counting objects: 100% (470/470), done.
remote: Compressing objects: 100% (248/248), done.
remote: Total 7877 (delta 371), reused 226 (delta 218), pack-reused 7407 (from 3)
Receiving objects: 100% (7877/7877), 4.89 MiB | 10.07 MiB/s, done.
Resolving deltas: 100% (5308/5308), done.
test@cudatools-daniel:
test@cudatools-daniel:~/daniel/numba-cuda$ ls
ci conda CONTRIBUTING.md greptile.json LICENSE.numba pixi.lock pyproject.toml scripts site-packages toolshed
CLA.md configs docs LICENSE numba_cuda pixi.toml README.md setup.py testing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This merge request introduces the new examples/ directory - it won't show up in your clone until after it's been merged.
Addressed in the next revision. |
docs/source/user/debugging.rst
Outdated
| To begin you’ll need an installation of CUDA Toolkit 13.1 and corresponding CUDA Driver for Linux. The installer for Linux can be downloaded from https://developer.nvidia.com/cuda-downloads. Follow the instructions provided on the download page to install it in ``/usr/local/cuda-13.1`` and create a symlink to it called ``/usr/local/cuda`` (which is handled by the install script). | ||
|
|
||
| You will also need a working Numba CUDA development environment. The documentation for Numba CUDA including the installation instructions can be found at: https://nvidia.github.io/numba-cuda. It is highly recommended that you use Anaconda, venv, or another Python virtual environment. The examples in this documentation are based on an Anaconda environment. In particular, make sure the required packages are installed in the virtual environment you wish to debug in. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are multiple ways a CUDA toolkit and driver could have been installed, so it seems better to me to just leave it at "you’ll need an installation of CUDA Toolkit 13.1 and corresponding CUDA Driver for Linux."
Is there any reason that a user specifically needs to use the CUDA installer from the download page and have the symlink to the 13.1 toolkit? Or would another valid method of installation work (e.g. driver installed from the driver downloads, CUDA toolkit from conda-forge)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's not really a user requirement to use that specific installation method, it was more a matter of offering a pointer to install directions. I've revised it to still include the download link, but now also mention that other CTK installation methods can be used instead.
docs/source/user/debugging.rst
Outdated
| The following commands can be used to create a Numba CUDA development environment using Anaconda. This example creates a virtual environment called ``numba-cuda-debug`` using Python 3.12. | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| conda create --name numba-cuda-debug python=3.12 | ||
| conda activate numba-cuda-debug | ||
| pip install cuda-python numba numba-cuda |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We already have installation instructions in the documentation: https://nvidia.github.io/numba-cuda/user/installation.html
This seems to be partially duplicating them, so I think it's probably be more concise and straightforward to omit this too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the "pip install" part, but left the Anaconda environment creation. Please let me know if you'd prefer that to be removed as well.
docs/source/user/debugging.rst
Outdated
| ] | ||
| } | ||
|
|
||
| The CUDA GDB pretty printer uses the Python ``numpy`` package to inspect and print numpy arrays. The ``launch.json`` file sets up the environment necessary for debugging. However, the pretty printer runs inside of cuda-gdb and not as part of the Python program being debugged. This means that the ``numpy`` package must be installed in both: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A blank link is needed for the unordered list to render correctly:
| The CUDA GDB pretty printer uses the Python ``numpy`` package to inspect and print numpy arrays. The ``launch.json`` file sets up the environment necessary for debugging. However, the pretty printer runs inside of cuda-gdb and not as part of the Python program being debugged. This means that the ``numpy`` package must be installed in both: | |
| The CUDA GDB pretty printer uses the Python ``numpy`` package to inspect and print numpy arrays. The ``launch.json`` file sets up the environment necessary for debugging. However, the pretty printer runs inside of cuda-gdb and not as part of the Python program being debugged. This means that the ``numpy`` package must be installed in both: | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. Fixed.
gmarkall
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Many thanks for the documentation!
I think as a tutorial of new functionality it's a good addition. In time I think we will want to evolve the documentation so that it presents it as less of a new feature / release note / tutorial all rolled into one, but keeping it in the form it is in now and getting it merged as soon as we can resolve the issues is good.
I left a couple of extra notes on the diff - mainly I think we should avoid telling people how to install things here, and a small issue with the rendering.
|
/ok to test 50bf976 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (2)
-
docs/source/user/debugging.rst, line 127-138 (link)logic: PYTHONPATH in documentation shows
../../miscbut actuallaunch.jsonuses${workspaceFolder}(line 50). These need to match. -
docs/source/user/debugging.rst, line 127 (link)logic: The file
numba-cuda/misc/gdb_print_extension.pydoesn't exist in the repository. Verify this file was included or update documentation to reference the correct path. Should gdb_print_extension.py be added in this PR, or does it already exist in a different location?
4 files reviewed, 2 comments
Includes a new section on CUDA GDB command-line debugging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (4)
-
docs/source/user/debugging.rst, line 329 (link)syntax: Missing space between "optionally" and "uses"
-
docs/source/user/debugging.rst, line 58-59 (link)logic: The referenced
gdb_print_extension.pyfile does not exist in the repository. The documentation mentions it should be innumba-cuda/misc/gdb_print_extension.py, but there is nomisc/directory at the repository root. The onlymisc/directory is atnumba_cuda/numba/cuda/misc/and doesn't containgdb_print_extension.py. This file needs to be added or the documentation andlaunch.jsonconfiguration need to be updated to either remove references to it or point to the correct location. -
examples/debugging/.vscode/launch.json, line 48 (link)logic: The path
${workspaceFolder}/../../miscdoes not exist in the repository. Thegdb_print_extension.pyfile referenced in line 35 cannot be found at this location. Either themisc/gdb_print_extension.pyfile needs to be created at the repository root, or this path needs to be corrected to point to an existing location. -
docs/source/_static/shared-array.png, line 1 (link)style: This image file is included in the PR but is not referenced anywhere in the documentation. Consider removing it if unused, or add a reference to it in
debugging.rstif it's meant to illustrate a debugging concept.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
4 files reviewed, 4 comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (4)
-
docs/source/user/debugging.rst, line 281 (link)logic: The
gdb_print_extension.pyfile referenced here doesn't exist in the repository. The documentation mentions it should be innumba-cuda/misc/gdb_print_extension.py, but themisc/directory doesn't exist. This file needs to be added to the repository for the pretty printer functionality to work. -
docs/source/user/debugging.rst, line 316 (link)logic: References
misc/gdb_print_extension.pyfile which doesn't exist in the repository. The launch.json configuration will fail to load the pretty printer extension. -
examples/debugging/.vscode/launch.json, line 48 (link)logic: PYTHONPATH references
${workspaceFolder}/../../miscbut themisc/directory doesn't exist in the repository. The pretty printer extension file needs to be added at this location. -
docs/source/user/debugging.rst, line 329 (link)syntax: typo:
optionallyusesshould be two words
4 files reviewed, 4 comments
|
Regarding comments 1, 2, and 3, the gdb_print_extension.py file is part of a separate merge request, which will be merged before this one.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Comments (3)
-
docs/source/user/debugging.rst, line 58-59 (link)logic: The
misc/gdb_print_extension.pyfile referenced here doesn't exist in the repository. Verify this file exists or update the documentation. -
docs/source/user/debugging.rst, line 330-336 (link)logic: This section references
numba-cuda/misc/gdb_print_extension.pybut this file doesn't exist in the repository. Themisc/directory itself is also missing. Is the gdb_print_extension.py file supposed to be added in this PR, or does it exist elsewhere? -
examples/debugging/.vscode/launch.json, line 47-48 (link)logic: The path
${workspaceFolder}/../../miscpoints to a non-existent directory. Themisc/gdb_print_extension.pyfile doesn't exist in the repository. Should this file be included in the PR or is it provided separately by CUDA Toolkit?
4 files reviewed, 3 comments
|
The initial version of the gdb_print_extension.py file has been submitted, which should resolve most if not all the remaining issues. /ok to test 2833d91 |
|
/ok to test e2b0f91 |
…bugging-documentation
- Exclude debugging example from the debug_prints check - Add SPDX header to debugging example. I think the env line isn't needed so I just replaced it. - Fix ruff-identified issues - Exclude the committed VSCode workspace file from the SPDX check.
|
/ok to test e36bd4b |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
5 files reviewed, 2 comments
| CUDA thread hit application kernel entry function breakpoint, hello<<<(1,1,1),(8,1,1)>>> (input=..., output=...) at hello.py:18 | ||
| 18 gid = cuda.grid(1) | ||
| (cuda-gdb) list | ||
| 13 # Editor window, and are shown as red dots next to the line numbers. The can also be set |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Grammar error: "The can also be set" should be "They can also be set"
| 13 # Editor window, and are shown as red dots next to the line numbers. The can also be set | |
| 13 # Editor window, and are shown as red dots next to the line numbers. They can also be set |
| # breakpoints. | ||
| # | ||
| # Breakpoints can also be set by clicking to the left of the source line numbers in the | ||
| # Editor window, and are shown as red dots next to the line numbers. The can also be set |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Same grammar error as in the documentation: "The can also be set" should be "They can also be set"
| # Editor window, and are shown as red dots next to the line numbers. The can also be set | |
| # Breakpoints can also be set by clicking to the left of the source line numbers in the |
The CUDA Toolkit 13.1 release provides beta support for debugging Numba CUDA programs using CUDA GDB and VSCode.
This PR includes the documentation for setup and debugging, and an example VSCode workspace that can be used as both a tutorial and as a starting point for other development projects.
Closes #664