fix rustc_private analysis on read-only / symlinked toolchains - #23251
Open
onlycs wants to merge 4 commits into
Open
fix rustc_private analysis on read-only / symlinked toolchains#23251onlycs wants to merge 4 commits into
onlycs wants to merge 4 commits into
Conversation
… processes NamedTempFile::new opened the file with only `create_new(true)`, which the standard library rejects (creating a file requires write or append access), so every use of NamedTempFile - notably the Cargo.lock copies handed to `cargo metadata` and proc-macro dylib copies on Windows - silently failed. Additionally, on Linux the file was immediately unlinked and re-addressed as /proc/self/fd/N. That path is only meaningful inside the rust-analyzer process; a spawned cargo cannot read it (the fd is CLOEXEC, and /proc/self refers to the child's own fd table). Fix both in new_from_existing by copying into a fresh, uniquely named temporary directory while keeping the original file name. Keeping the name matters because cargo refuses lockfile paths not literally named "Cargo.lock". The copy is also made writable, since the source may live in a read-only toolchain installation (e.g. the nix store). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…data The rustc-dev dist component ships the compiler sources with `rustc-src/rust/compiler/rustc/Cargo.toml` as the entry manifest, but without a workspace root manifest next to the lockfile, which lives at `rustc-src/rust/Cargo.lock`. FetchMetadata only looked for a lockfile right next to the manifest, so for `rust-analyzer.rustc.source = "discover"` setups the metadata fetch ran with `--locked` and no usable lockfile. On read-only toolchain installations (e.g. rustup toolchains on nix) cargo then fails to create one, and rust-analyzer silently degrades to `--no-deps` metadata, dropping rustc_middle and friends from the crate graph entirely - rustc_private projects lose all type information for rustc crates. Walk up from the manifest to find the lockfile and reuse the existing lockfile-copy mechanism, which keeps cargo from touching the original. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scan of the target libdir for rustc_macros & co. used DirEntry::file_type(), which does not follow symlinks. Toolchains assembled out of symlinks (e.g. by nix / oxalica's rust-overlay) link every dylib into the sysroot, so all proc-macro dylibs were skipped. As a result `rustc_queries!` never expanded for rustc_private projects and the macro-generated TyCtxt query getters (`tcx.mir_keys(())` etc.) did not resolve at all. Use fs::metadata, which traverses symlinks, instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With a nix toolchain, every macro-generated
TyCtxtquery getter (tcx.mir_keys(()), e.g.) resolves to{unknown}. Three independent bugs cause this.1. the
rustc-srclockfile is never foundThe
rustc-devcomponent ships compiler sources as[rustc-src]/rust/compiler/rustc/Cargo.toml, with no workspace root manifest. The lockfile lives two levels up at[rustc-src]/rust/Cargo.lock.FetchMetadata::newonly looks for a lockfile directly next to the manifest, so the rustc metadata fetch runs--lockedwith no lockfile available. In a read-only store, cargo cannot create one and errors:FetchMetadata::execthen silently falls back to the--no-depspre-fetch, whose metadata contains only therustc-mainstub package.rustc_middle,rustc_hir, etc. never enter the crate graph.Instead, we walk up the manifest's ancestor directories to find the lockfile, then reuse the existing lockfile-copy mechanism.
2. the lockfile-copy mechanism never worked
Even with the lockfile found,
make_lockfile_copyalways returnedNone:NamedTempFile::newopens with onlycreate_new(true). This is incorrect. Creating or truncating a file requires write or append access, so creation failed unconditionally, on every platform. This also affects the Windows proc-macro dylib copies.On Linux the temp file is unlinked and re-addressed as
/proc/self/fd/N. That path only is only valid to rust-analyzer: in the spawned cargo,/proc/selfrefers to cargo's fd table, and the descriptor isn't inherited across exec (O_CLOEXEC). So the lockfile-copy path handed tocargo metadatawas a dangling name from cargo's pov.cargo additionally rejects lockfile paths not literally named
Cargo.lock, which the{prefix}{pid}-{counter}naming scheme violates.Now,
new_from_existingcopies into a temporary directory, preserves the original file name, and makes the copy writable.3: symlinked proc-macro dylibs are skipped
WorkspaceBuildScripts::rustc_cratesscans the target libdir for the prebuilt proc-macro dylibs (librustc_macros.so& co.) usingDirEntry::file_type(), which does not follow symlinks. Nix sysroots symlink every file into place, so all dylibs were filtered out. Withoutrustc_macros,rustc_queries!never expands, the generatedrustc_with_all_queriesmacro doesn't exist, and none ofdefine_callbacks!'s output (those macro-generated query getters) is ever created.The fix here was to just use
fs::metadatawhich traverses symlinks.AI disclosure
These changes were authored with AI assistance (Claude Code); the commits carry
Co-Authored-Bytrailers. I have reviewed the changes, use them in a current project using a patched build, and can answer questions about them myself.