[FR]: Attaching debuggers to py_binary is difficult

[zfy0701]

What happened?

when use default debugger for py_binary

the command generated is like:

/usr/local/bin/python3.12 -X pycache_prefix=/.../JetBrains/PyCharm2024.3/cpython-cache /Users/fuyaoz/Applications/PyCharm Professional Edition.app/Contents/plugins/python-ce/helpers/pydev/pydevd.py --multiprocess --qt-support=auto --client 127.0.0.1 --port 55690 --file /.../server/server 

notice the last parameter /../server/server is a shell instead of python file, the debugger will fail here:

switch back to official rules_python works

Version

Development (host) and target OS/architectures:

Output of bazel --version: 7.4.1

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file: MODULE.bazel

Language(s) and/or frameworks involved: pyth

python plugin 1.0.0~ 1.3.2 behaviors similar

How to reproduce

Any other information?

No response

[arrdem]

It's difficult to simultaneously support debuggers which want to supervise or control debuggee startup with the needs of configuring a hermetic Python interpreter.

I suspect this will soon be broken in rules_python upstream too if you turn on the new /bin/sh based launcher script and only happens to work using the old non-hermetic python3 based launcher shim. This will continue to not work post- #551 which again uses a shell-based launcher.

Probably the only way to harmonize the two would be to teach IDEA how to request Bazel build and run a configuration of a Python program which embeds the IDEA pydev helper and treats that helper as its entrypoint.

[jab]

@zfy0701, in case you can take a similar approach with PyCharm, in https://youtu.be/GTsy7IfmywQ?si=2eVPMRPLPySsjoQA&t=286 @alexeagle shows how to get VSCode's Python debugger to work with a rules_py project essentially by bypassing Bazel and just using the rules_py-generated virtualenv directly, which is not difficult.

You also may be interested in liking/subscribing to https://youtrack.jetbrains.com/issue/PY-63403/Support-debugpy which tracks adding support for debugpy to PyCharm. debugpy allows IDEs to set and hit breakpoints even for Python programs started via Bazel. I've tried it with VSCode and it works like a (wait for it) charm.

[joel-aspect]

@jab do you need this at work?

We wish we could use a network-based debugger like in Java. We can just document how to do that.

[jab]

We wish we could use a network-based debugger like in Java.

I'm not too experienced with developing Java with Bazel, but doesn't debugpy fit the bill of a network-based debugger for Python?

@jab do you need this at work?

I think all rules_py users (for work and otherwise) would benefit from a more out-of-the-box debugging experience that doesn't require bypassing Bazel (myself included). Until that's possible, I wonder if rules_py could do more to help users avoid the bad developer experience where the developer adds a breakpoint() somewhere in their code, naively bazel runs their py_binary expecting it to work, but instead it just hangs. Maybe just documentation, if not something fancier that uses e.g. ruff's flake8-debugger rules.

[arrdem]

By design rules_py doesn't provide Python dependencies, which makes supporting "features" like a debugger or coverage or what have you conceptually difficult.

We'd need a way for a user to bring their own debugger wrapper which can provide dependencies which we aren't hardcoding. That's probably a toolchain.

Probably what does belong in rules_py is some machinery as part of py_binary which slots the debugger in, either as activated by default code via a _99_debug.pth file, or as part of the generated launcher script.

[arrdem]

Ignoring #581, having a way to 1) inject debugpy as a dep and 2) shove -m debugpy --listen ${DEBUGPY_HOST:-0.0.0.0}":"${DEBUGPY_PORT:-5678}" "$@" into the entrypoint script would probably do the trick?

[arrdem]

This will get easier with #650, which makes it somewhat easier to pull in dependencies like pytest and debugpy in a consistent way.