From a303e8339574e85d942e585661fdfec78f2f8489 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Thu, 31 Jul 2025 21:13:56 -0700 Subject: [PATCH 1/2] installation_and_usage --- gazelle/README.md | 139 ---------------------- gazelle/docs/index.md | 1 + gazelle/docs/installation_and_usage.md | 156 +++++++++++++++++++++++++ 3 files changed, 157 insertions(+), 139 deletions(-) create mode 100644 gazelle/docs/installation_and_usage.md diff --git a/gazelle/README.md b/gazelle/README.md index df3085bb37..11a9e5b2ba 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -6,145 +6,6 @@ ReadTheDocs. Please see https://rules-python.readthedocs.io/gazelle/docs/index.h ::: -## Example - -We have an example of using Gazelle with Python located [here](https://github.com/bazel-contrib/rules_python/tree/main/examples/bzlmod). -A fully-working example without using bzlmod is in [`examples/build_file_generation`](../examples/build_file_generation). - -The following documentation covers using bzlmod. - -## Adding Gazelle to your project - -First, you'll need to add Gazelle to your `MODULE.bazel` file. -Get the current version of Gazelle from there releases here: https://github.com/bazelbuild/bazel-gazelle/releases/. - - -See the installation `MODULE.bazel` snippet on the Releases page: -https://github.com/bazel-contrib/rules_python/releases in order to configure rules_python. - -You will also need to add the `bazel_dep` for configuration for `rules_python_gazelle_plugin`. - -Here is a snippet of a `MODULE.bazel` file. - -```starlark -# The following stanza defines the dependency rules_python. -bazel_dep(name = "rules_python", version = "0.22.0") - -# The following stanza defines the dependency rules_python_gazelle_plugin. -# For typical setups you set the version. -bazel_dep(name = "rules_python_gazelle_plugin", version = "0.22.0") - -# The following stanza defines the dependency gazelle. -bazel_dep(name = "gazelle", version = "0.31.0", repo_name = "bazel_gazelle") - -# Import the python repositories generated by the given module extension into the scope of the current module. -use_repo(python, "python3_9") -use_repo(python, "python3_9_toolchains") - -# Register an already-defined toolchain so that Bazel can use it during toolchain resolution. -register_toolchains( - "@python3_9_toolchains//:all", -) - -# Use the pip extension -pip = use_extension("@rules_python//python:extensions.bzl", "pip") - -# Use the extension to call the `pip_repository` rule that invokes `pip`, with `incremental` set. -# Accepts a locked/compiled requirements file and installs the dependencies listed within. -# Those dependencies become available in a generated `requirements.bzl` file. -# You can instead check this `requirements.bzl` file into your repo. -# Because this project has different requirements for windows vs other -# operating systems, we have requirements for each. -pip.parse( - name = "pip", - requirements_lock = "//:requirements_lock.txt", - requirements_windows = "//:requirements_windows.txt", -) - -# Imports the pip toolchain generated by the given module extension into the scope of the current module. -use_repo(pip, "pip") -``` -Next, we'll fetch metadata about your Python dependencies, so that gazelle can -determine which package a given import statement comes from. This is provided -by the `modules_mapping` rule. We'll make a target for consuming this -`modules_mapping`, and writing it as a manifest file for Gazelle to read. -This is checked into the repo for speed, as it takes some time to calculate -in a large monorepo. - -Gazelle will walk up the filesystem from a Python file to find this metadata, -looking for a file called `gazelle_python.yaml` in an ancestor folder of the Python code. -Create an empty file with this name. It might be next to your `requirements.txt` file. -(You can just use `touch` at this point, it just needs to exist.) - -To keep the metadata updated, put this in your `BUILD.bazel` file next to `gazelle_python.yaml`: - -```starlark -load("@pip//:requirements.bzl", "all_whl_requirements") -load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") -load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") - -# This rule fetches the metadata for python packages we depend on. That data is -# required for the gazelle_python_manifest rule to update our manifest file. -modules_mapping( - name = "modules_map", - wheels = all_whl_requirements, -) - -# Gazelle python extension needs a manifest file mapping from -# an import to the installed package that provides it. -# This macro produces two targets: -# - //:gazelle_python_manifest.update can be used with `bazel run` -# to recalculate the manifest -# - //:gazelle_python_manifest.test is a test target ensuring that -# the manifest doesn't need to be updated -gazelle_python_manifest( - name = "gazelle_python_manifest", - modules_mapping = ":modules_map", - # This is what we called our `pip_parse` rule, where third-party - # python libraries are loaded in BUILD files. - pip_repository_name = "pip", - # This should point to wherever we declare our python dependencies - # (the same as what we passed to the modules_mapping rule in WORKSPACE) - # This argument is optional. If provided, the `.test` target is very - # fast because it just has to check an integrity field. If not provided, - # the integrity field is not added to the manifest which can help avoid - # merge conflicts in large repos. - requirements = "//:requirements_lock.txt", - # include_stub_packages: bool (default: False) - # If set to True, this flag automatically includes any corresponding type stub packages - # for the third-party libraries that are present and used. For example, if you have - # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` - # package will be automatically included in the BUILD file. - # - # Enabling this feature helps ensure that type hints and stubs are readily available - # for tools like type checkers and IDEs, improving the development experience and - # reducing manual overhead in managing separate stub packages. - include_stub_packages = True -) -``` - -Finally, you create a target that you'll invoke to run the Gazelle tool -with the rules_python extension included. This typically goes in your root -`/BUILD.bazel` file: - -```starlark -load("@bazel_gazelle//:def.bzl", "gazelle") - -# Our gazelle target points to the python gazelle binary. -# This is the simple case where we only need one language supported. -# If you also had proto, go, or other gazelle-supported languages, -# you would also need a gazelle_binary rule. -# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example -gazelle( - name = "gazelle", - gazelle = "@rules_python_gazelle_plugin//python:gazelle_binary", -) -``` - -That's it, now you can finally run `bazel run //:gazelle` anytime -you edit Python code, and it should update your `BUILD` files correctly. - - ### Directives You can configure the extension using directives, just like for other diff --git a/gazelle/docs/index.md b/gazelle/docs/index.md index ea20e9c3e0..c04efd6a41 100644 --- a/gazelle/docs/index.md +++ b/gazelle/docs/index.md @@ -42,4 +42,5 @@ the `update` command (the default) does anything for Python code. ```{toctree} :maxdepth: 1 +installation_and_usage ``` diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md new file mode 100644 index 0000000000..0863e95750 --- /dev/null +++ b/gazelle/docs/installation_and_usage.md @@ -0,0 +1,156 @@ +# Installation and Usage + +## Example + +Examples of using Gazelle with Python can be found in the `rules_python` +repo: + +* bzlmod: {gh-path}`examples/bzlmod_build_file_generation` +* WORKSPACE: {gh-path}`examples/build_file_generation` + +:::{note} +The following documentation covers using bzlmod. +::: + + +## Adding Gazelle to your project + +First, you'll need to add Gazelle to your `MODULE.bazel` file. Get the current +version of Gazelle from their [releases page][gazelle-releases]. + +[gazelle-releases]: https://github.com/bazel-contrib/bazel-gazelle/releases/ + +See the installation `MODULE.bazel` snippet on the `rules_python` +[releases page][rules-python-releases] in order to configure `rules_python`. + +[rules-python-releases]: https://github.com/bazel-contrib/rules_python/releases + +You will also need to add the {bzl:obj}`bazel_dep` for configuration for +`rules_python_gazelle_plugin`. + +Here is a snippet of a `MODULE.bazel` file. + +```starlark +# The following stanza defines the dependency rules_python. +bazel_dep(name = "rules_python", version = "0.22.0") + +# The following stanza defines the dependency rules_python_gazelle_plugin. +# For typical setups you set the version. +bazel_dep(name = "rules_python_gazelle_plugin", version = "0.22.0") + +# The following stanza defines the dependency gazelle. +bazel_dep(name = "gazelle", version = "0.31.0", repo_name = "bazel_gazelle") + +# Import the python repositories generated by the given module extension into +# the scope of the current module. +use_repo(python, "python3_9") +use_repo(python, "python3_9_toolchains") + +# Register an already-defined toolchain so that Bazel can use it during +# toolchain resolution. +register_toolchains( + "@python3_9_toolchains//:all", +) + +# Use the pip extension +pip = use_extension("@rules_python//python:extensions.bzl", "pip") + +# Use the extension to call the `pip_repository` rule that invokes `pip`, with +# `incremental` set. +# Accepts a locked/compiled requirements file and installs the dependencies listed within. +# Those dependencies become available in a generated `requirements.bzl` file. +# You can instead check this `requirements.bzl` file into your repo. +# Because this project has different requirements for windows vs other +# operating systems, we have requirements for each. +pip.parse( + name = "pip", + requirements_lock = "//:requirements_lock.txt", + requirements_windows = "//:requirements_windows.txt", +) + +# Imports the pip toolchain generated by the given module extension into the +# scope of the current module. +use_repo(pip, "pip") +``` + +Next, we'll fetch metadata about your Python dependencies, so that gazelle can +determine which package a given import statement comes from. This is provided +by the {bzl:obj}`modules_mapping` rule. We'll make a target for consuming this +{bzl:obj}`modules_mapping`, and writing it as a manifest file for Gazelle to read. +This is checked into the repo for speed, as it takes some time to calculate +in a large monorepo. + +Gazelle will walk up the filesystem from a Python file to find this metadata, +looking for a file called `gazelle_python.yaml` in an ancestor folder +of the Python code. Create an empty file with this name. It might be next +to your `requirements.txt` file. (You can just use {command}`touch` at +this point, it just needs to exist.) + +To keep the metadata updated, put this in your `BUILD.bazel` file next +to `gazelle_python.yaml`: + +```starlark +load("@pip//:requirements.bzl", "all_whl_requirements") +load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") +load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") + +# This rule fetches the metadata for python packages we depend on. That data is +# required for the gazelle_python_manifest rule to update our manifest file. +modules_mapping( + name = "modules_map", + wheels = all_whl_requirements, +) + +# Gazelle python extension needs a manifest file mapping from +# an import to the installed package that provides it. +# This macro produces two targets: +# - //:gazelle_python_manifest.update can be used with `bazel run` +# to recalculate the manifest +# - //:gazelle_python_manifest.test is a test target ensuring that +# the manifest doesn't need to be updated +gazelle_python_manifest( + name = "gazelle_python_manifest", + modules_mapping = ":modules_map", + # This is what we called our `pip_parse` rule, where third-party + # python libraries are loaded in BUILD files. + pip_repository_name = "pip", + # This should point to wherever we declare our python dependencies + # (the same as what we passed to the modules_mapping rule in WORKSPACE) + # This argument is optional. If provided, the `.test` target is very + # fast because it just has to check an integrity field. If not provided, + # the integrity field is not added to the manifest which can help avoid + # merge conflicts in large repos. + requirements = "//:requirements_lock.txt", + # include_stub_packages: bool (default: False) + # If set to True, this flag automatically includes any corresponding type stub packages + # for the third-party libraries that are present and used. For example, if you have + # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` + # package will be automatically included in the BUILD file. + # + # Enabling this feature helps ensure that type hints and stubs are readily available + # for tools like type checkers and IDEs, improving the development experience and + # reducing manual overhead in managing separate stub packages. + include_stub_packages = True +) +``` + +Finally, you create a target that you'll invoke to run the Gazelle tool +with the `rules_python` extension included. This typically goes in your root +`/BUILD.bazel` file: + +```starlark +load("@bazel_gazelle//:def.bzl", "gazelle") + +# Our gazelle target points to the python gazelle binary. +# This is the simple case where we only need one language supported. +# If you also had proto, go, or other gazelle-supported languages, +# you would also need a gazelle_binary rule. +# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example +gazelle( + name = "gazelle", + gazelle = "@rules_python_gazelle_plugin//python:gazelle_binary", +) +``` + +That's it, now you can finally run `bazel run //:gazelle` anytime +you edit Python code, and it should update your `BUILD` files correctly. From feaec567966ce6f21b30bf653ff46220abb03fa3 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Sun, 3 Aug 2025 21:02:06 -0700 Subject: [PATCH 2/2] updates for review --- gazelle/docs/installation_and_usage.md | 103 ++++++++++++------------- 1 file changed, 49 insertions(+), 54 deletions(-) diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md index 0863e95750..e764957581 100644 --- a/gazelle/docs/installation_and_usage.md +++ b/gazelle/docs/installation_and_usage.md @@ -16,67 +16,53 @@ The following documentation covers using bzlmod. ## Adding Gazelle to your project First, you'll need to add Gazelle to your `MODULE.bazel` file. Get the current -version of Gazelle from their [releases page][gazelle-releases]. +version of [Gazelle][bcr-gazelle] from the [Bazel Central Registry][bcr]. Then +do the same for [`rules_python`][bcr-rules-python] and +[`rules_python_gazelle_plugin`][bcr-rules-python-gazelle-plugin]. -[gazelle-releases]: https://github.com/bazel-contrib/bazel-gazelle/releases/ +[bcr-gazelle]: https://registry.bazel.build/modules/gazelle +[bcr]: https://registry.bazel.build/ +[bcr-rules-python]: https://registry.bazel.build/modules/rules_python +[bcr-rules-python-gazelle-plugin]: https://registry.bazel.build/modules/rules_python_gazelle_plugin -See the installation `MODULE.bazel` snippet on the `rules_python` -[releases page][rules-python-releases] in order to configure `rules_python`. - -[rules-python-releases]: https://github.com/bazel-contrib/rules_python/releases - -You will also need to add the {bzl:obj}`bazel_dep` for configuration for -`rules_python_gazelle_plugin`. - -Here is a snippet of a `MODULE.bazel` file. +Here is a snippet of a `MODULE.bazel` file. Note that most of it is just +general config for `rules_python` itself - the Gazelle plugin is only two lines +at the end. ```starlark -# The following stanza defines the dependency rules_python. -bazel_dep(name = "rules_python", version = "0.22.0") - -# The following stanza defines the dependency rules_python_gazelle_plugin. -# For typical setups you set the version. -bazel_dep(name = "rules_python_gazelle_plugin", version = "0.22.0") +################################################ +## START rules_python CONFIG ## +## See the main rules_python docs for details ## +################################################ +bazel_dep(name = "rules_python", version = "1.5.1") -# The following stanza defines the dependency gazelle. -bazel_dep(name = "gazelle", version = "0.31.0", repo_name = "bazel_gazelle") +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.12.2") +use_repo(python, "python_3_12_2") -# Import the python repositories generated by the given module extension into -# the scope of the current module. -use_repo(python, "python3_9") -use_repo(python, "python3_9_toolchains") - -# Register an already-defined toolchain so that Bazel can use it during -# toolchain resolution. -register_toolchains( - "@python3_9_toolchains//:all", -) - -# Use the pip extension pip = use_extension("@rules_python//python:extensions.bzl", "pip") - -# Use the extension to call the `pip_repository` rule that invokes `pip`, with -# `incremental` set. -# Accepts a locked/compiled requirements file and installs the dependencies listed within. -# Those dependencies become available in a generated `requirements.bzl` file. -# You can instead check this `requirements.bzl` file into your repo. -# Because this project has different requirements for windows vs other -# operating systems, we have requirements for each. pip.parse( - name = "pip", + hub_name = "pip", requirements_lock = "//:requirements_lock.txt", requirements_windows = "//:requirements_windows.txt", ) - -# Imports the pip toolchain generated by the given module extension into the -# scope of the current module. use_repo(pip, "pip") + +############################################## +## START rules_python_gazelle_plugin CONFIG ## +############################################## + +# The Gazelle plugin depends on Gazelle. +bazel_dep(name = "gazelle", version = "0.33.0", repo_name = "bazel_gazelle") + +# Typically rules_python_gazelle_plugin is version matched to rules_python. +bazel_dep(name = "rules_python_gazelle_plugin", version = "1.5.1") ``` Next, we'll fetch metadata about your Python dependencies, so that gazelle can determine which package a given import statement comes from. This is provided -by the {bzl:obj}`modules_mapping` rule. We'll make a target for consuming this -{bzl:obj}`modules_mapping`, and writing it as a manifest file for Gazelle to read. +by the `modules_mapping` rule. We'll make a target for consuming this +`modules_mapping`, and writing it as a manifest file for Gazelle to read. This is checked into the repo for speed, as it takes some time to calculate in a large monorepo. @@ -90,6 +76,7 @@ To keep the metadata updated, put this in your `BUILD.bazel` file next to `gazelle_python.yaml`: ```starlark +# `@pip` is the hub_name from pip.parse in MODULE.bazel. load("@pip//:requirements.bzl", "all_whl_requirements") load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") @@ -111,9 +98,11 @@ modules_mapping( gazelle_python_manifest( name = "gazelle_python_manifest", modules_mapping = ":modules_map", - # This is what we called our `pip_parse` rule, where third-party + + # This is what we called our `pip.parse` rule in MODULE.bazel, where third-party # python libraries are loaded in BUILD files. pip_repository_name = "pip", + # This should point to wherever we declare our python dependencies # (the same as what we passed to the modules_mapping rule in WORKSPACE) # This argument is optional. If provided, the `.test` target is very @@ -121,12 +110,12 @@ gazelle_python_manifest( # the integrity field is not added to the manifest which can help avoid # merge conflicts in large repos. requirements = "//:requirements_lock.txt", + # include_stub_packages: bool (default: False) # If set to True, this flag automatically includes any corresponding type stub packages # for the third-party libraries that are present and used. For example, if you have # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` # package will be automatically included in the BUILD file. - # # Enabling this feature helps ensure that type hints and stubs are readily available # for tools like type checkers and IDEs, improving the development experience and # reducing manual overhead in managing separate stub packages. @@ -139,16 +128,22 @@ with the `rules_python` extension included. This typically goes in your root `/BUILD.bazel` file: ```starlark -load("@bazel_gazelle//:def.bzl", "gazelle") +load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") + +gazelle_binary( + name = "gazelle_multilang", + languages = [ + # List of language plugins. + # If you want to generate py_proto_library targets PR #3057), then + # the proto language plugin _must_ come before the rules_python plugin. + #"@bazel_gazelle//lanugage/proto", + "@rules_python_gazelle_plugin//python", + ], +) -# Our gazelle target points to the python gazelle binary. -# This is the simple case where we only need one language supported. -# If you also had proto, go, or other gazelle-supported languages, -# you would also need a gazelle_binary rule. -# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example gazelle( name = "gazelle", - gazelle = "@rules_python_gazelle_plugin//python:gazelle_binary", + gazelle = ":gazelle_multilang", ) ```