Skip to content

fix rustc_private analysis on read-only / symlinked toolchains - #23251

Open
onlycs wants to merge 4 commits into
rust-lang:masterfrom
onlycs:fix-rustc-private-readonly-sysroots
Open

fix rustc_private analysis on read-only / symlinked toolchains#23251
onlycs wants to merge 4 commits into
rust-lang:masterfrom
onlycs:fix-rustc-private-readonly-sysroots

Conversation

@onlycs

@onlycs onlycs commented Aug 29, 2026

Copy link
Copy Markdown

Problem

With a nix toolchain, every macro-generated TyCtxt query getter (tcx.mir_keys(()), e.g.) resolves to {unknown}. Three independent bugs cause this.

1. the rustc-src lockfile is never found

The rustc-dev component ships compiler sources as [rustc-src]/rust/compiler/rustc/Cargo.toml, with no workspace root manifest. The lockfile lives two levels up at [rustc-src]/rust/Cargo.lock. FetchMetadata::new only looks for a lockfile directly next to the manifest, so the rustc metadata fetch runs --locked with no lockfile available. In a read-only store, cargo cannot create one and errors:

error: cannot create the lock file /nix/store/…/rustc-src/rust/compiler/rustc/Cargo.lock
because --locked was passed to prevent this

FetchMetadata::exec then silently falls back to the --no-deps pre-fetch, whose metadata contains only the rustc-main stub package. rustc_middle, rustc_hir, etc. never enter the crate graph.

Instead, we walk up the manifest's ancestor directories to find the lockfile, then reuse the existing lockfile-copy mechanism.

2. the lockfile-copy mechanism never worked

Even with the lockfile found, make_lockfile_copy always returned None:

  • NamedTempFile::new opens with only create_new(true). This is incorrect. Creating or truncating a file requires write or append access, so creation failed unconditionally, on every platform. This also affects the Windows proc-macro dylib copies.

  • On Linux the temp file is unlinked and re-addressed as /proc/self/fd/N. That path only is only valid to rust-analyzer: in the spawned cargo, /proc/self refers to cargo's fd table, and the descriptor isn't inherited across exec (O_CLOEXEC). So the lockfile-copy path handed to cargo metadata was a dangling name from cargo's pov.

  • cargo additionally rejects lockfile paths not literally named Cargo.lock, which the {prefix}{pid}-{counter} naming scheme violates.

Now, new_from_existing copies into a temporary directory, preserves the original file name, and makes the copy writable.

3: symlinked proc-macro dylibs are skipped

WorkspaceBuildScripts::rustc_crates scans the target libdir for the prebuilt proc-macro dylibs (librustc_macros.so & co.) using DirEntry::file_type(), which does not follow symlinks. Nix sysroots symlink every file into place, so all dylibs were filtered out. Without rustc_macros, rustc_queries! never expands, the generated rustc_with_all_queries macro doesn't exist, and none of define_callbacks!'s output (those macro-generated query getters) is ever created.

The fix here was to just use fs::metadata which traverses symlinks.

AI disclosure

These changes were authored with AI assistance (Claude Code); the commits carry Co-Authored-By trailers. I have reviewed the changes, use them in a current project using a patched build, and can answer questions about them myself.

onlycs and others added 3 commits August 28, 2026 20:50
… processes

NamedTempFile::new opened the file with only `create_new(true)`, which the
standard library rejects (creating a file requires write or append access),
so every use of NamedTempFile - notably the Cargo.lock copies handed to
`cargo metadata` and proc-macro dylib copies on Windows - silently failed.

Additionally, on Linux the file was immediately unlinked and re-addressed as
/proc/self/fd/N. That path is only meaningful inside the rust-analyzer
process; a spawned cargo cannot read it (the fd is CLOEXEC, and /proc/self
refers to the child's own fd table).

Fix both in new_from_existing by copying into a fresh, uniquely named
temporary directory while keeping the original file name. Keeping the name
matters because cargo refuses lockfile paths not literally named
"Cargo.lock". The copy is also made writable, since the source may live in
a read-only toolchain installation (e.g. the nix store).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…data

The rustc-dev dist component ships the compiler sources with
`rustc-src/rust/compiler/rustc/Cargo.toml` as the entry manifest, but
without a workspace root manifest next to the lockfile, which lives at
`rustc-src/rust/Cargo.lock`. FetchMetadata only looked for a lockfile
right next to the manifest, so for `rust-analyzer.rustc.source =
"discover"` setups the metadata fetch ran with `--locked` and no usable
lockfile. On read-only toolchain installations (e.g. rustup toolchains on
nix) cargo then fails to create one, and rust-analyzer silently degrades
to `--no-deps` metadata, dropping rustc_middle and friends from the crate
graph entirely - rustc_private projects lose all type information for
rustc crates.

Walk up from the manifest to find the lockfile and reuse the existing
lockfile-copy mechanism, which keeps cargo from touching the original.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scan of the target libdir for rustc_macros & co. used
DirEntry::file_type(), which does not follow symlinks. Toolchains
assembled out of symlinks (e.g. by nix / oxalica's rust-overlay) link
every dylib into the sysroot, so all proc-macro dylibs were skipped. As a
result `rustc_queries!` never expanded for rustc_private projects and the
macro-generated TyCtxt query getters (`tcx.mir_keys(())` etc.) did not
resolve at all.

Use fs::metadata, which traverses symlinks, instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants