Failed to find python3 binary

[ThomasMader]

I cloned the modular repository at 9855690a6a40f87f7c2948a59d417dee21d966f1 and tried to build the mojo standard library with

sh bazelw build //mojo/stdlib/...

from within the repository but that fails with

Error in fail: Failed to find python3 binary
ERROR: @llvm-project//lldb:enable_libedit :: Error loading option @llvm-project//lldb:enable_libedit: Failed to find python3 binary

I realized that the failing part is coming from toolchains_llvm so I thought it's best to create an issue here.
This is the URL used to fetch the repo: https://github.com/bazel-contrib/toolchains_llvm/archive/6eef27401b4dff9de4ca6f66aea7592a90305789.tar.gz

I am on NixOS and make use of nix-ld and envfs to make running bazel/bazelisk work.
See also https://discourse.nixos.org/t/bazel-on-nix-use-bazelisk/64575/17.

It doesn't matter if python3 is installed system wide or provided in the $PATH via pixi shell.

[keith]

I think this is actually failing in llvm, which does this https://github.com/llvm/llvm-project/blob/1a65e63c596d9459f49e4495e92cdecac2795f71/utils/bazel/configure.bzl#L39, so it requires you have a host python3 in your PATH

[keith]

also sidenote you should probably avoid sh bazelw since this will exec it with sh instead of bash as that script expects, which may or may not work. If you want to submit a PR to that script to change it to #!/usr/bin/env bash to support nixos I can land that for you

[ThomasMader]

I think this is actually failing in llvm, which does this https://github.com/llvm/llvm-project/blob/1a65e63c596d9459f49e4495e92cdecac2795f71/utils/bazel/configure.bzl#L39, so it requires you have a host python3 in your PATH

This is why I don't understand it because before I execute the build I enter a pixi shell and that brings python3 into PATH.

$ python --version
Python 3.13.7

$ echo $PATH
/home/thomad/dev/modular/.pixi/envs/default/bin:/run/wrappers/bin:/home/thomad/.nix-profile/bin:/nix/profile/bin:/home/thomad/.local/state/nix/profile/bin:/etc/profiles/per-user/thomad/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin

$ which python3
/home/thomad/dev/modular/.pixi/envs/default/bin/python3

also sidenote you should probably avoid sh bazelw since this will exec it with sh instead of bash as that script expects, which may or may not work. If you want to submit a PR to that script to change it to #!/usr/bin/env bash to support nixos I can land that for you

I was playing around with that too and in the end I always executed the downloaded bazelisk binary in the build directory directly. But bazelw only downloads the bazelisk binary and starts it so I think it is not related to which shell is used later after the binary takes over execution or am I wrong?

There are already PRs for changing that:

#548
#543
#497
#459

I also tried to take the patch of the first PR in the list above and apply it with the other patch in the modular repository but that didn't help either.

[keith]

Today the PATH in the modular repo is limited to /usr/bin:/bin which likely doesn't have a python3 on your system. And for the /bin/bash thing I meant modular's bazelw not the files in this repo. I submitted a PR to the modular repo that will be out in the next nightly to relax those 2 constraints which should fix this. If not please file an issue there!

[ThomasMader]

Thanks @keith for your help.

Today the PATH in the modular repo is limited to /usr/bin:/bin which likely doesn't have a python3 on your system.

You are right, normally on NixOS there is no /usr/bin:/bin but I enabled envfs which provides those paths if something tries to execute those binaries.

[nix-shell:~/dev/modular]$ /bin/python3 --version
Python 3.12.11

What does not work though is to use ls for example.

[nix-shell:~/dev/modular]$ ls -l /bin/python3
ls: cannot access '/bin/python3': No such file or directory

That is what I missed but now I understand the problem.

And for the /bin/bash thing I meant modular's bazelw not the files in this repo.

I worked around the bazelw issues I had by using envfs but it's cool if that can be solved as well clearly.
I tested with '#!/usr/bin/env bash' and it works but I also had to set it for tools/bazel script in the modular repo.