chore: Down with WORKSPACE (#597)

This is a massive combined patch aimed at starting to address a whole
pile of WORKSPACE related techdebt and migration pain points.

Major points:
- Delete the `WORKSPACE` file
- Delete `WORKSPACE` supporting private files
- Retain `WORKSPACE` supporting `py/repositories.bzl` for now; will drop
in future 2.0 release
- Synchronize how we're configuring Rust and the crossbuild toolchain(s)
with the Aspect CLI, other projects

As future items we can go through and consider bumping dependencies

### Changes are visible to end-users: no

### Test plan

- This is the test plan

### Remaining items
- [x] Eliminate dev proto dependencies
- [x] Drop source build Gazelle for prebuilt  
- [x] Get local tests to green
- [x] Inline out `include()` statements which block subrepo/e2e tests
(cf. bazelbuild/bazel#27137)
- [x] Update the release generation flow to discard all the dev
dependencies and retain only the desired prod MODULE.bazel content
- [x] Test the release artifact generation flow; ensure release artifact
naming expectations are preserved

### Future work

Need to come back to the e2e tests which were removed. Under
bzlmod the e2e tests are quite expensive due to sysroot and LLVM
setup, to the point of being prohibitive (~10min of setup for 0.5s of test).

However we did give up some coverage there.

---------

Co-authored-by: aspect-marvin[bot] <[email protected]>

Author: arrdem

.aspect/workflows/config.yaml

@@ -2,16 +2,9 @@
 
 # This list is maintained by tools/e2e/sync.py
 workspaces:
-  # SYNC: e2e
-  # {{{
   - .
-  - e2e/cross-repo-610
-  - e2e/interpreter-version-541
-  - e2e/repository-rule-deps
-  - e2e/smoke
-  - e2e/system-interpreter
+  - e2e
   - examples/uv_pip_compile
-# }}}
 
 tasks:
   # TODO(arrdem): Rosetta doesn't like HEAD version strings seemingly
@@ -24,37 +17,23 @@ tasks:
   #           rm .bazelversion
   #           echo "USE_BAZEL_VERSION=last_green" > .bazeliskrc
 
-  # TODO(arrdem): Bazel 8+ dropped the original PyInfo etc. which the ancient
-  # rules_python we have on the WORKSPACE side isn't ready for yet. Once we kill
-  # off WORKSPACE we can flip these back on but practically this is blocked on
-  # getting that change in.
-  #
-  # - test:
-  #     name: test-os:linux-bzl:latest
-  #     queue: aspect-large
-  #     hooks:
-  #       - type: before_task
-  #         command: |
-  #           rm .bazelversion
-  #           echo "USE_BAZEL_VERSION=latest" > .bazeliskrc
-
-  # - test:
-  #     name: test-os:linux-bzl:8
-  #     queue: aspect-large
-  #     hooks:
-  #       - type: before_task
-  #         command: |
-  #           rm .bazelversion
-  #           echo "USE_BAZEL_VERSION=8.2.1" > .bazeliskrc
+  - test:
+      name: test-os:linux-bzl:latest
+      queue: aspect-large
+      hooks:
+        - type: before_task
+          command: |
+            rm .bazelversion
+            echo "USE_BAZEL_VERSION=latest" > .bazeliskrc
 
   - test:
-      name: test-os:linux-bzl:7
+      name: test-os:linux-bzl:8
       queue: aspect-large
       hooks:
         - type: before_task
           command: |
             rm .bazelversion
-            echo "USE_BAZEL_VERSION=7.6.1" > .bazeliskrc
+            echo "USE_BAZEL_VERSION=8.4.0" > .bazeliskrc
 
   - finalization:
       queue: aspect-small

.bazelrc

@@ -1,32 +1,41 @@
-# Bazel settings that apply to this repository.
-# Take care to document any settings that you expect users to apply.
-# Settings that apply only to CI are in .github/workflows/ci.bazelrc
+import %workspace%/bazel/defaults.bazelrc
 
-test --test_output=errors
+common --enable_bzlmod
+
+# Ignore slow manual and release targets
+# Prevents materializing crossbuild
+build --build_tag_filters=-manual,-release
+test --test_tag_filters=-manual,-release
 
-# TODO(alex): enable
-common --noenable_bzlmod
-# For very recent bazel we have to set this explicitly
-common --enable_workspace
+test --test_output=errors
 
 # Define value used by tests
 build --define=SOME_VAR=SOME_VALUE
 
-common --features=-libtool
+#common --features=-libtool
 common --incompatible_enable_cc_toolchain_resolution
+
+# TODO(bzlmod): Don't break proto
 common --per_file_copt=external/.*protobuf.*@--PROTOBUF_WAS_NOT_SUPPOSED_TO_BE_BUILT
 common --host_per_file_copt=external/.*protobuf.*@--PROTOBUF_WAS_NOT_SUPPOSED_TO_BE_BUILT
 
-# Use local rules_python
-# Enable with --config=dev
-common:dev --override_repository=rules_python=~/workspace/rules_python
-# No external runfiles
-common --nolegacy_external_runfiles
+# Don't try and auto detect the cc toolchain, as we use our own gcc toolchains.
+common --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
+common --incompatible_enable_cc_toolchain_resolution
+
+# Don't link against libunwind on macos as it causes linking failures (https://github.com/bazel-contrib/toolchains_llvm/pull/346)
+common:macos --@toolchains_llvm//toolchain/config:libunwind=False
+
+common:ci --toolchain_resolution_debug='@@bazel_tools//tools/cpp:toolchain_type'
+
+common:release --build_tag_filters=release
+common:release --stamp
+common:release --compilation_mode=opt
+common:release --@rules_rust//rust/settings:lto=fat
 
-# Don’t want to push a rules author to update their deps if not needed.
-# https://bazel.build/reference/command-line-reference#flag--check_direct_dependencies
-# https://bazelbuild.slack.com/archives/C014RARENH0/p1691158021917459?thread_ts=1691156601.420349&cid=C014RARENH0
-common --check_direct_dependencies=off
+# Speed up local development by using the non-hermetic CPP toolchain.
+common:nollvm --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=0
+common:nollvm --noincompatible_enable_cc_toolchain_resolution
 
 # Load any settings specific to the current user.
 # .bazelrc.user should appear in .gitignore so that settings are not shared with team members

.bazelversion

@@ -1,4 +1,4 @@
-7.6.1
+8.4.0
 # The first line of this file is used by Bazelisk and Bazel to be sure
 # the right version of Bazel is used to build and test this repo.
 # This also defines which version is used on CI.