diff --git a/text/0000-build-std.md b/text/0000-build-std.md new file mode 100644 index 00000000000..fae2ee25096 --- /dev/null +++ b/text/0000-build-std.md @@ -0,0 +1,329 @@ +- Feature Name: `build-std` +- Start Date: 2025-06-05 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + + + +# Summary +[summary]: #summary + +While Rust's pre-built standard library has proven itself sufficient for the +majority of use cases, there are a handful of use cases that are not well +supported: + +1. Rebuilding the standard library to match the user's profile +2. Rebuilding the standard library with ABI-modifying flags +3. Building the standard library for tier three targets + +This RFC proposes a handful of changes to Cargo, the compiler and standard +library with the goal of defining a minimal build-std that has the potential of +being stabilised: + +- Explicitly declaring support for the standard library in target specs +- Explicit and implicit dependencies on the standard library in `Cargo.toml` +- Re-building the standard library when the profile or target modifiers change + +This RFC is co-authored by [David Wood][davidtwco] and +[Adam Gemmell][adamgemmell]. To improve the readability of this RFC, it does not +follow the standard RFC template, while still aiming to capture all of the +salient details that the template encourages. Due to the length of this RFC, it +is split over multiple files to avoid rendering issues and slow loading on some +platforms. + +### Scope +[scope]: #scope + +build-std, as proposed by this RFC, has many restrictions and limitations that +mean it will not support most use cases that those waiting for build-std hope +that it will. This is an explicit and deliberate choice. + +This RFC will focus on resolving the key questions that will enable a MVP of +build-std to be accepted and stabilised. This will lay the foundation for future +proposals to lift restrictions and enable build-std to support more use cases, +without those proposals having to survey the ten+ years of issues, pull requests +and discussion that this RFC has. + +As a general rule, this RFC tries to answer the question "what crates of the +standard library get built and when do they get built" and considers anything +else as likely out-of-scope. + +### Acknowledgements +[acknowledgements]: #acknowledgements + +This RFC would not have been possible without the advice, feedback and support +of [Josh Triplett][joshtriplett], [Eric Huss][ehuss], +[Wesley Wiser][wesleywiser] and [Tomas Sedovic][tomassedovic] throughout this +entire effort. + +Thanks to [mati865] for advising on some of the specifics related to special +object files, [petrochenkov] for his expertise on rustc's dependency loading and +name resolution; [fee1-dead] for their early and thorough reviews; +[Ed Page][epage] for writing about opaque dependencies and his invaluable Cargo +expertise; [Jacob Bramley][jacobbramley] for his feedback on early drafts; as +well as [Amanieu D'Antras][amanieu], [Tobias Bieniek][turbo87], +[Adam Harvey][lawngnome], [James Munns][jamesmunns], +[Jonathan Pallant][thejpster], [Jieyou Xu][jieyouxu], [Jakub Beránek][kobzol], +[Weihang Lo][weihanglo], and [Mark Rousskov][simulacrum] for providing feedback +from their areas of expertise on later drafts. + +### Terminology +[terminology]: #terminology + +The following terminology is used throughout the RFC: + +- "the standard library" is used to refer to multiple of the crates that + constitute the standard library such as `core`, `alloc`, `std`, `test`, + `proc_macro` or their dependencies. +- "std" is used to refer only to the `std` crate, not the entirety of the + standard library + +Throughout the RFC's "Proposal" sections, parentheses with "?" links will be +present that which link the relevant section in the appropriate "Rationale and +alternatives" section to justify a decision or provide alternatives to it. + +Additionally, "note alerts" will be used in the *Proposal* sections to separate +implementation considerations from the core proposal. Implementation detail +should be considered non-normative. These details could change during +implementation and are present solely to demonstrate that the implementation +feasibility has been considered and to provide an example of how implementation +could proceed. + +> [!NOTE] +> +> This is an example of a "note alert" that will be used to separate +> implementation detail from the proposal proper. + +# Contents +[contents]: #contents + +This RFC has two self-contained parts and has been written alongside follow-up +RFCs that extend the infrastructure proposed here. Both parts of this RFC have +value independent of each other while having synergies. + +As build-std is a complex feature with many interdependent design decisions, it +is challenging to draft a proposal that is small enough to have an achievable +scope in the short-to-medium term while making a convincing argument that it is +forward-compatible with any desired future extensions. This RFC contains the +initial infrastructure for build-std and the most basic version of the feature. +Later RFCs will extend this infrastructure to improve usability and utility. + +1. [Summary][summary] (you are here) + + - Introduction to the proposal, its scope, terminology/conventions used and + the structure of the RFC + + - [Proposal-wide rationale and alternatives][rationale-and-alternatives] + +2. [Background](./0000-build-std/1-background.md) + + - Detailed explanations of how relevant and impacted parts of the Rust + toolchain currently work + +3. [History](./0000-build-std/2-history.md) + + - Chronological summary of the various proposals and discussions that have + taken place relating to the ability to rebuild the standard library, and + of the current experimental implementation in Cargo + +4. [Motivation](./0000-build-std/3-motivation.md) + + - Descriptions of the varied problems that build-std has been proposed as a + solution to + +5. [build-std=always](./0000-build-std/4-build-std-always.md) + + - Proposes adding a `build-std = "always|never"` option to the Cargo + configuration which will unconditionally re-build the standard library + crates listed in a new `build-std-crates` option + + - [Proposal](./0000-build-std/4-build-std-always.md#proposal) + + - [Rationale and alternatives](./0000-build-std/4-build-std-always.md#rationale-and-alternatives) + + - [Unresolved questions](./0000-build-std/4-build-std-always.md#unresolved-questions) + + - [Future possibilities](./0000-build-std/4-build-std-always.md#future-possibilities) + +6. [Explicit dependencies](./0000-build-std/5-standard-library-dependencies.md) + + - Proposes supporting explicit dependencies on the standard library crates in + `Cargo.toml` + + - Enables Cargo to determine which standard library crates are required by + the crate graph without `build-std-crates` being set + + - Necessary for future extensions which support public/private standard + library dependencies or enabling features of the standard library + + - [Proposal](./0000-build-std/5-standard-library-dependencies.md#proposal) + + - [Rationale and alternatives](./0000-build-std/5-standard-library-dependencies.md#rationale-and-alternatives) + + - [Unresolved questions](./0000-build-std/5-standard-library-dependencies.md#unresolved-questions) + + - [Future possibilities](./0000-build-std/5-standard-library-dependencies.md#future-possibilities) + +7. [Appendix I: Summary of changes](./0000-build-std/6-appendix-summary-of-changes.md) + + - Summary of each of the changes which would need implemented in the Rust + toolchain, grouped by the project team whose purview the change would fall + under + +8. [Appendix II: Exhaustive literature review](./0000-build-std/7-appendix-literature-review.md) + + - More detailed summaries of the relevant issues, discussions, pull requests + and proposals that comprise the history of the build-std feature since + 2015 + + - [*History*](./0000-build-std/2-history.md) aims to summarise this content further and + cover everything that should be necessary to understand the proposal + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +These rationales and alternatives apply to the proposal as-a-whole: + +## Why not do nothing? +[rationale-why-not-do-nothing]: #why-not-do-nothing + +Support for rebuilding the standard library is a long-standing feature request +from subsets of the Rust community and blocks the work of some project teams +(e.g. sanitisers and branch protection in the compiler team, amongst others). +Inaction forces these users to remain on nightly and depend on the unstable +`-Zbuild-std` flag indefinitely. RFCs and discussion dating back to the first +stable release of the language demonstrate the longevity of build-std as a +need. + +## Shouldn't build-std be part of rustup? +[rationale-in-rustup]: #shouldnt-build-std-be-part-of-rustup + +build-std is effectively creating a new sysroot with a customised standard +library. rustup as Rust's toolchain manager has existing machinery to create and +maintain sysroots, and if it could invoke Cargo to build the standard library +then it could create a new toolchain from a build from a `rust-src` component. +rustup would be invoking tools from the next layer of abstraction (Cargo) in the +same way that Cargo invokes tools from the layer of abstraction after it +(rustc). + +A brief prototype of this idea was created and a +[short design document was drafted][why-not-rustup] before concluding that it +would not be possible. With Cargo's artifact dependencies it may be desirable +to build with a different standard library and if rustup was creating different +toolchains per-customised standard library then Cargo would need to have +knowledge of these to switch between them, which isn't possible (and something +of a layering violation). It is also unclear how Cargo would find and use the +uncustomized host sysroot for build scripts and procedural macros. In addition +rustup's knowledge of sysroots and toolchains is limited to the archives it +unpacks - it becoming a part of the build system is not trivial, especially +considering it uses a different versioning system to Cargo, Rust and the +standard library. + +# Drawbacks +[drawbacks]: #drawbacks + +There are some drawbacks to build-std: + +- build-std overlaps with the initial designs and ideas for opaque dependencies + in Cargo, thereby introducing a risk of constraining or conflicting with the + eventual complete design for opaque dependencies + +[davidtwco]: https://github.com/davidtwco +[adamgemmell]: https://github.com/adamgemmell +[amanieu]: https://github.com/Amanieu +[ehuss]: https://github.com/ehuss +[epage]: https://github.com/epage +[fee1-dead]: https://github.com/fee1-dead +[jacobbramley]: https://github.com/jacobbramley +[jamesmunns]: https://github.com/JamesMunns +[jieyouxu]: https://github.com/jieyouxu +[joshtriplett]: https://github.com/joshtriplett +[kobzol]: https://github.com/kobzol +[lawngnome]: https://github.com/LawnGnome +[mati865]: https://github.com/mati865 +[petrochenkov]: https://github.com/petrochenkov +[simulacrum]: https://github.com/simulacrum +[thejpster]: https://github.com/thejpster +[tomassedovic]: https://github.com/tomassedovic +[turbo87]: https://github.com/Turbo87 +[weihanglo]: https://github.com/weihanglo +[wesleywiser]: https://github.com/wesleywiser + +[why-not-rustup]: https://hackmd.io/@davidtwco/rkYRlKv_1x diff --git a/text/0000-build-std/1-background.md b/text/0000-build-std/1-background.md new file mode 100644 index 00000000000..3b3f60056de --- /dev/null +++ b/text/0000-build-std/1-background.md @@ -0,0 +1,343 @@ +# Background +[background]: #background + +See [*Implementation summary*][implementation-summary] for a summary of the +current unstable build-std feature in Cargo. This section aims to introduce any +relevant details about the standard library and compiler that are assumed +knowledge by referenced sources and later sections. + +## Standard library +[background-standard-library]: #standard-library + +Since the first stable release of Rust, the standard library has been distributed +as a pre-built artifact via rustup, which has a variety of advantages and/or +rationale: + +- It saves Rust users from having to rebuild the standard library whenever they + start a project or do a clean build +- The standard library has and has had dependencies which require a more + complicated build environment than typical Rust projects + - e.g. requiring a working C toolchain to build `compiler_builtins`' `c` + feature +- To varying degrees at different times in its development, the standard + library's implementation has been tied to the compiler implementation and has had + to change in lockstep + +Not all targets have a pre-built standard library distributed via rustup, though +it is a minimum requirement for certain platform support tiers. According to +rustc's [platform support docs][platform-support], for tier three targets: + +> Tier 3 targets are those which the Rust codebase has support for, but which +> the Rust project does not build or test automatically, so they may or may not +> work. Official builds are not available. + +..and tier two targets: + +> The Rust project builds official binary releases of the standard library (or, +> in some cases, only the core library) for each tier 2 target, and automated +> builds ensure that each tier 2 target can be used as build target after each +> change. + +..and finally, tier one targets: + +> The Rust project builds official binary releases for each tier 1 target, and +> automated testing ensures that each tier 1 target builds and passes tests +> after each change. + +As an innate property of the target, not all targets can support the `std` crate +This is independent of its tier, where as stated in the +[Target Tier Policy][target-tier-policy] lower-tier targets may not have a +complete implementation for all APIs in the crates they can support. + +All of the standard library crates leverage permanently unstable features +provided by the compiler that will never be stabilised and therefore require +nightly to build. + +The configuration for the pre-built standard library build is spread across +bootstrap, the standard library workspace, individual standard library crate +manifests and the target specification. The pre-built standard library is +installed into the sysroot. + +At the beginning of compilation, unless the crate has the `#![no_std]` +attribute, the compiler will load the `libstd.rlib` file from the sysroot as a +dependency of the current crate and add an implicit `extern crate std` for it. +This is the mechanism by which every crate has an implicit dependency on the +standard library. + +The standard library sources are distributed in the `rust-src` component by +rustup and placed in the sysroot under `lib/rustlib/src/`. The sources consist +of the `library/` workspace plus `src/llvm-project/libunwind`, which was +required in the past to build the `unwind` crate on some targets. + +Cargo supports explicitly declaring a dependency on crates with the same names +as standard library crates with a `path` source +(e.g. `core = { path = "../my_core" }`), which rustc will load instead of crates +in the sysroot. Crates with these dependencies are not accepted by crates.io, +but there are crates on GitHub that use this pattern, such as +[embed-rs/stm32f7-discovery][embed-rs-cargo-toml], which are used as `git` +dependencies of other crates on GitHub. + +### Dependencies +[background-dependencies]: #dependencies + +Behind the facade, the standard library is split into multiple crates, some of +which are in different repositories and included as submodules or using [JOSH]. + +As well as local crates, the standard library depends on crates from crates.io. +It needs to be able to point these crates' dependencies on the standard library +at the sources of `core`, `alloc` and `std` in the current [rust-lang/rust] +checkout. + +This is achieved through use of the `rustc-dep-of-std` feature. Crates used in +the dependency graph of `std` declare a `rustc-dep-of-std` feature and when +enabled, add new dependencies on `rustc-std-workspace-{core,alloc,std}`. +`rustc-std-workspace-{core,alloc,std}` are empty crates published on crates.io. +As part of the workspace for the standard library, +`rustc-std-workspace-{core,alloc,std}` are patched with a `path` source to the +directory for the corresponding crate. + +Historically, there have necessarily been C dependencies of the standard library, +increasing the complexity of the build environment required. While these have +largely been removed over time - for example, `libbacktrace` previously depended +on `backtrace-sys` but now uses `gimli` ([rust#46439]), a pure-rust +implementation. There are still some C dependencies: + +- `libunwind` will either link to the LLVM `libunwind` or the system's + `libunwind`/`libgcc_s`. LLVM's `libunwind` is shipped as part of the + rustup component for the standard library and will be linked against + when `-Clink-self-contained` is used + - This only applies to Linux and Fuchsia targets +- `compiler_builtins` has an optional `c` feature that will use optimised + routines from `compiler-rt` when enabled. It is enabled for the pre-built + standard library +- `compiler_builtins` has an optional `mem` feature that provides symbols + for common memory routines (e.g. `memcpy`) + - It is enabled automatically on some `no_std` platforms as when `std` is + built `libc` provides these routines. + - Users can rely on weak linkage to override these symbols, but in scenarios + where weak linkage is not supported or where the symbols are to be + overridden from a shared library, then users must directly turn the feature + off. +- To use sanitizers, the sanitizer runtimes from LLVM's compiler-rt need to + be linked against. Building of these is enabled in `bootstrap.toml` + ([`build.sanitizers`][bootstrap-sanitizers]) and they are + included in the rustup components shipped by the project. + +Dependencies of the standard library may use unstable or internal compiler and +language features only when they are a dependency of the standard library. + +### Features +[background-features]: #features + +There are a handful of features defined in the standard library crates' +`Cargo.toml`s. These features are not strictly additive (`llvm-libunwind` and +`system-llvm-libunwind` are mutually exclusive). There is currently no stable +existing mechanism for users to enable or disable these features. The default +set of features is determined by [logic in bootstrap][bootstrap-features-logic] +and [the `rust.std-features` key in `bootstrap.toml`][bootstrap-features-toml]. +The enabled features are often different depending on the target. + +It is also common for user crates to depend on the standard library (via +`#![no_std]`) conditional on Cargo features being enabled or disabled (e.g. a +`std` feature or if `--test` is used). + +### Target support +[background-target-support]: #target-support + +The `std` crate's [`build.rs`][std-build.rs] checks for supported values of the +`CARGO_CFG_TARGET_*` environment variables. These variables are akin to the +conditional compilation [configuration options][conditional-compilation-config-options], +and often correspond to parts of the target triple (for example, +`CARGO_CFG_TARGET_OS` corresponds to the "os" part of a target triple - "linux" +in "aarch64-unknown-linux-gnu"). This filtering is strict enough to distinguish +between built-in targets but loose enough to match similar custom targets. There +is no equivalent mechanism on the `alloc` or `core` crates. + +When encountering an unknown or unsupported operating system then the +`restricted_std` cfg is set. `restricted_std` marks the entire standard library +as unstable, requiring `feature(restricted_std)` to be enabled on any crate that +depends on it. The only way for users to enable the `restricted_std` feature on +behalf of dependencies is the uncommon `-Zcrate-attr=features(restricted_std)` +rustc flag and users commonly report that they are not aware how to do this. + +Cargo and rustc support custom targets, defined in JSON files according to an +unstable schema defined in the compiler. On nightly, users can dump the +target-spec-json for an existing target using `--print target-spec-json`. This +JSON can be saved in a file, tweaked and used as the argument to `--target`. It +is unintentional but custom target specifications can be used with `--target` +even on stable toolchains ([rust#71009] proposes destabilising this behaviour). +However, as custom targets do not have a pre-built standard library and so must +use `-Zbuild-std`, their use is relegated to nightly toolchains in practice. +Custom targets may have `restricted_std` set depending on their `cfg` +configuration options. + +## Prelude +[background-prelude]: #prelude + +rustc has the concept of the "extern prelude" which is the set of crates that +can be referred to without an explicit `extern crate` statement. Originally this +was populated by users writing `extern crate $crate` in their code for each +direct dependency. Since the 2018 edition, crates passed via `--extern` are +added to the extern prelude. `core` is always added to the extern prelude. For +crates without the `#![no_std]` attribute, `std` is added to the extern prelude. + +`core` or `std`'s prelude module (depending on the presence of `#![no_std]`) is +imported by rustc injecting a `use $crate::prelude::rust_20XX::*` statement. + +`extern crate` can still be used and will search for the dependency in locations +where direct dependencies can be found, such as `-L crate=` paths or in the +sysroot. `-L dependency=` paths will not be searched, as these directories only +contain indirect dependencies (i.e. dependencies of direct dependencies). + +Although only `std` or `core` are added to the extern prelude automatically, +users can still write `extern crate alloc` or `extern crate test` to load them +from the sysroot. + +`--extern` has a `noprelude` modifier which will allow the user to use +`--extern` to specify the location at which a crate can be found without adding +it to the extern prelude. This could allow a path for crates like `alloc` or +`test` to be provided without effecting the observable behaviour of the +language. + +## Panic strategies +[background-panic-strategies]: #panic-strategies + +Rust has the concept of a *panic handler*, which is a crate that is responsible +for performing a panic. There are various panic handler crates on crates.io, +such as [panic-abort] (which different from the `panic_abort` panic runtime!), +[panic-halt], [panic-itm], and [panic-semihosting]. Panic handler crates define +a function annotated with `#[panic_handler]`. There can only be one +`#[panic_handler]` in the crate graph. + +`core` uses the panic handler to implement panics inserted by code generation +(e.g. arithmetic overflow or out-of-bounds access) and the `core::panic!` macro +immediately delegates to the panic handler crate. + +`std` defines a panic handler. `std`'s panic handler function and its +`std::panic!` macro print panic information to stderr and delegate to a +*panic runtime* to decide what to do next, determined by the *panic strategy*. + +There are two panic runtime crates in the standard library - `panic_unwind` +(which gracefully unwinds the stack using `libunwind` and performs cleanup) and +`panic_abort` (which terminates the program shortly after being called). Each +target supported by rustc specifies a default panic strategy - either "unwind" +or "abort" - though these are only relevant if `std`'s panic handler is used +(i.e. the target isn't a `no_std` target or being used with a `no_std` crate). + +Rust's `-Cpanic` flag allows the user to choose the panic strategy, with the +target's default as a fallback. If `-Cpanic=unwind` is provided then this +doesn't guarantee that the unwind strategy is used, as the target may not +support it. + +Both crates are compiled and shipped with the pre-built standard library for +targets which support `std`. Some targets have a pre-built standard library with +only the `core` and `alloc` crates, such as the `x86_64-unknown-none` target. +While `x86_64-unknown-none` defaults to the `abort` panic strategy, as this +target does not support the standard library, this default isn't actually +relevant. + +The `std` crate has a `panic_unwind` feature that enables an optional dependency +on the `panic_unwind` crate. + +`core` also has a `panic_immediate_abort` feature which modifies the +`core::panic!` macro to immediately call the abort intrinsic without calling the +panic handler, which can dramatically reduce code size. `std` and `alloc` have +the same feature which enable the feature in `core`. `std`'s feature also adds +an immediate abort to its `panic!` macro. + +## Cargo +[background-cargo]: #cargo + +Cargo's building of the dependency graph is largely driven by the registry +index, except for crates from `git` or `path` sources. + +[Cargo registries][cargo-docs-registry], like crates.io, are centralised sources +for crates. A registry's index is the interface between Cargo and the registry +that Cargo queries to know which versions are available for any given crate, +what its dependencies are, etc. + +Cargo can query registries using a Git protocol which caches the registry on +disk, or using a sparse protocol which exposes the index over HTTP and allows +Cargo to avoid Cargo having a local copy of the whole index, which has become +quite large for crates.io. + +crates.io's registry index is exposed as both a HTTP API and a Git repository - +[rust-lang/crates.io-index] - both are updated automatically by crates.io when +crates are published, yanked, etc. The HTTP API is mostly used. + +Each crate in the registry index has a JSON file, following +[a defined schema][cargo-json-schema] which is jointly maintained by the Cargo +and crates.io teams. Crates may refer to those in other registries, but all +non-`path`/`git` crates in the dependency graph must exist in a registry. As the +registry index drives the building of Cargo's dependency graph, all +non-`path`/`git` crates that end up in the dependency graph must be present a +registry. + +When a package is published, Cargo posts a JSON blob to the registry which is +not a index entry but has sufficient information to generate one. crates.io does +not use Cargo's JSON blob, instead re-generating it from the `Cargo.toml` (this +avoids the index and `Cargo.toml` from going out-of-sync due to bugs or +malicious publishes). As a consequence, changes to the index format must be +duplicated in Cargo and crates.io. Behind the scenes, data from the `Cargo.toml` +extracted by crates.io is written to a database, which is where the index entry +and frontend are generated from. + +Dependency information of crates in the registry are rendered in the crates.io +frontend. + +Registries can have different policies for what crates are accepted. For +example, crates.io does not permit publishing packages named `std` or `core` but +other registries might. + +### Public/private dependencies +[background-pubpriv-dependencies]: #publicprivate-dependencies + +[Public and private dependencies][rust#44663] are an unstable feature which +enables declaring which dependencies form part of a library's public interface, +so as to make it easier to avoid breaking semver compatibility. + +With the `public-dependency` feature enabled, dependencies are marked as +"private" by default which can be overridden with a `public = true` declaration. + +Private dependencies are passed to rustc with an `priv` modifier to the +`--extern` flag. Dependencies without this modifier are treated as public by +rustc for backwards compatibility reasons. rust emits the +`exported-private-dependencies` lint if an item from a private dependency is +re-exported. + +## Target modifiers +[background-target-modifiers]: #target-modifiers + +[rfcs#3716] introduced the concept of *target modifiers* to rustc. Flags marked +as target modifiers must match across the entire crate graph or the compilation +will fail. + +For example, flags are made target modifiers when they change the ABI of +generated code and could result in unsound ABI mismatches if two crates are +linked together with different values of the flag set. + +[implementation-summary]: ./2-history.md#implementation-summary + +[JOSH]: https://josh-project.github.io/josh/intro.html +[panic-abort]: https://crates.io/crates/panic-abort +[panic-halt]: https://crates.io/crates/panic-halt +[panic-itm]: https://crates.io/crates/panic-itm +[panic-semihosting]: https://crates.io/crates/panic-semihosting +[rust-lang/crates.io-index]: https://github.com/rust-lang/crates.io-index +[rust-lang/rust]: https://github.com/rust-lang/rust + +[rfcs#3716]: https://rust-lang.github.io/rfcs/3716-target-modifiers.html +[rust#46439]: https://github.com/rust-lang/rust/pull/46439 +[rust#44663]: https://github.com/rust-lang/rust/issues/44663 +[rust#71009]: https://github.com/rust-lang/rust/issues/71009 + +[bootstrap-features-logic]: https://github.com/rust-lang/rust/blob/00b526212bbdd68872d6f964fcc9a14a66c36fd8/src/bootstrap/src/lib.rs#L732 +[bootstrap-features-toml]: https://github.com/rust-lang/rust/blob/00b526212bbdd68872d6f964fcc9a14a66c36fd8/bootstrap.example.toml#L816 +[bootstrap-sanitizers]: https://github.com/rust-lang/rust/blob/d13a431a6cc69cd65efe7c3eb7808251d6fd7a46/bootstrap.example.toml#L388 +[cargo-docs-registry]: https://doc.rust-lang.org/nightly/nightly-rustc/cargo/sources/registry/index.html +[cargo-json-schema]: https://doc.rust-lang.org/cargo/reference/registry-index.html#json-schema +[conditional-compilation-config-options]: https://doc.rust-lang.org/reference/conditional-compilation.html#set-configuration-options +[embed-rs-cargo-toml]: https://github.com/embed-rs/stm32f7-discovery/blob/e2bf713263791c028c2a897f2eb1830d7f09eceb/Cargo.toml#L21 +[target-tier-policy]: https://doc.rust-lang.org/nightly/rustc/target-tier-policy.html +[std-build.rs]: https://github.com/rust-lang/rust/blob/f315e6145802e091ff9fceab6db627a4b4ec2b86/library/std/build.rs#L17 +[platform-support]: https://doc.rust-lang.org/nightly/rustc/platform-support.html diff --git a/text/0000-build-std/2-history.md b/text/0000-build-std/2-history.md new file mode 100644 index 00000000000..d3224b56275 --- /dev/null +++ b/text/0000-build-std/2-history.md @@ -0,0 +1,565 @@ +# History +[history]: #history + +*The following summary of the prior art is necessarily less detailed than the +source material, which is exhaustively surveyed in +[Appendix II: Exhaustive literature review][appendix-ii].* + +## [rfcs#1133] (2015) +[rfcs-1133-2015]: #rfcs1133-2015 + +build-std was first proposed in a [2015 RFC (rfcs#1133)][rfcs#1133] by +[Ericson2314], aiming to improve support for targets that do not have a +pre-built standard library; to enable building the standard library with +different profiles; and to simplify `rustbuild` (now `bootstrap`). It also was +written with the goal of supporting the user in providing a custom +implementation of the standard library and supporting different implementations +of the language that provide their own standard libraries. + +This RFC proposed that the standard library be made an explicit dependency in +`Cargo.toml` and be rebuilt automatically when required. An implicit dependency +on the standard library would be added automatically unless an explicit +dependency is written. This RFC was written prior to a stable `#![no_std]` +attribute and so does not address the circumstance where a implicit dependency +would make a no-std crate fail to compile on a target that does not support +the standard library. + +There were objectives of and possibilities enabled by the RFC that were not +shared with the project teams at the time, such as the standard library being +a regular crate on crates.io and the concept of the sysroot being retired. +Despite this, the RFC appeared to be close to acceptance before being blocked +by Cargo having a mechanism to have unstable features and then closed in favour +of [cargo#4959]. + +## [xargo] and [cargo#4959] (2016) +[xargo-and-cargo-4959-2016]: #xargo-and-cargo4959-2016 + +While the discussions around [rfcs#1133] where ongoing, [xargo] was released in +2016. Xargo is a Cargo wrapper that builds a sysroot with a customised standard +library and then uses that with regular Cargo operations (i.e. `xargo build` +performs the same operation as `cargo build` but with a customised standard +library). Configuration for the customised standard library was configured in +the `Xargo.toml`, supporting configuring codegen flags, profile settings, Cargo +features and multi-stage builds. It required nightly to build the standard +library as it did not use `RUSTC_BOOTSTRAP`. Xargo had inherent limitations due +to being a Cargo wrapper, leading to suggestions that its functionality be +integrated into Cargo. + +[cargo#4959] is a proposal inspired by [xargo], suggesting that a `[sysroot]` +section be added to `.cargo/config` which would enable similar configuration to +that of `Xargo.toml`. If this configuration is set, Cargo would build and use a +sysroot with a customised standard library according to the configuration +specified and the release profile. This sysroot would be rebuilt whenever +relevant configuration changes (e.g. profiles). [cargo#4959] received varied +feedback: the proposed syntax was not sufficiently user-friendly; it did not +enable the user to customise the standard library implementation; and that +exposing bootstrap stages was brittle and user-unfriendly. [cargo#4959] wasn't +updated after submission so ultimately stalled and remains open. + +[rfcs#1133] and [cargo#4959] took very different approaches to build-std, with +[cargo#4959] proposing a simpler approach that exposed the necessary low-level +machinery to users and [rfcs#1133] attempting to take a more first-class and +user-friendly approach that has many tricky design implications. + +## [rfcs#2663] (2019) +[rfcs-2663-2019]: #rfcs2663-2019 + +In 2019, [*rfcs#2663: `std` Aware Cargo*][rfcs#2663] was opened as the most +recent RFC attempting to advance build-std. [rfcs#2663] shared many of the +motivations of [rfcs#1133]: building the standard library for tier three and +custom targets; customising the standard library with different Cargo features; +and applying different codegen flags to the standard library. It did not concern +itself with build-std's potential use in `rustbuild` or with abolishing the +sysroot. + +[rfcs#2663] was primarily concerned what functionality should be available to +the user and what the user experience ought to be. It proposed that `core`, +`alloc` and `std` be automatically built when the target did not have a pre-built +standard library available through rustup. It would be automatically rebuilt on +any target when the profile configuration was modified such that it no longer +matched the pre-built standard library. If using nightly, the user could enable +Cargo features and modify the source of the standard library. Standard library +dependencies were implicit by default, as today, but would be written explicitly +when enabling Cargo features. It also aimed to stabilise the target-spec-json +format and allow "stable" Cargo features to be enabled on stable toolchains, and +as such proposed the concept of stable and unstable Cargo features be +introduced. + +There was a lot of feedback on [rfcs#2663] which largely stemmed from it being +very high-level, containing many large unresolved questions and details left for +the implementers to work out. For example, it proposed that there be a concept +of stable and unstable Cargo features but did not elaborate any further, leaving +that as an implementation detail. Nevertheless, the proposal was valuable in +more clearly elucidating a potential user experience that build-std could aim +for, and the feedback provided was incorporated into the [wg-cargo-std-aware] +effort, described below. + +## [wg-cargo-std-aware] (2019-) +[wg-cargo-std-aware-2019-]: #wg-cargo-std-aware-2019- + +[rfcs#2663] demonstrated that there was demand for a mechanism for being able to +(re-)build the standard library, and the feedback showed that this was a thorny +problem with lots of complexity, so in 2019, the [wg-cargo-std-aware] repository +was created to organise related work and explore the issues involved in +build-std. + +[wg-cargo-std-aware] led to the current unstable implementation of `-Zbuild-std` +in Cargo, which is described in detail in the [*Implementation summary* +section][implementation-summary] below. + +Issues in the wg-cargo-std-aware repository can be roughly partitioned into seven +categories: + +1. **Exploring the motivations and use cases for the standard library** + + There are a handful of motivations catalogued in the [wg-cargo-std-aware] + repository, corresponding to those raised in the earlier RFCs and proposals: + + - Building with custom profile settings ([wg-cargo-std-aware#2]) + - Building for unsupported targets ([wg-cargo-std-aware#3]) + - Building with different Cargo features ([wg-cargo-std-aware#4]) + - Replacing the source of the standard library ([wg-cargo-std-aware#7]) + - Using build-std in bootstrap/rustbuild ([wg-cargo-std-aware#19]) + - Improving the user experience for `no_std` binary projects + ([wg-cargo-std-aware#36]) + + These are all either fairly self-explanatory, described in the summary of the + previous RFCs/proposals above, or in the [*Motivation*][motivation] section + of this RFC. + +2. **Support for build-std in Cargo's subcommands** + + Cargo has various subcommands where the desired behaviour when used with + build-std needs some thought and consideration. A handful of issues were + created to track this, most receiving little to no discussion: + [`cargo metadata`][wg-cargo-std-aware#20], [`cargo clean`][wg-cargo-std-aware#21], + [`cargo pkgid`][wg-cargo-std-aware#24], and [the `-p` flag][wg-cargo-std-aware#26]. + + [`cargo fetch`][wg-cargo-std-aware#22] had fairly intuitive interactions with + build-std - that `cargo fetch` should also fetch any dependencies of the + standard library - which was implemented in [cargo#10129]. + + The [`--build-plan` flag][wg-cargo-std-aware#45] does not support build-std and its + issue did not receive much discussion, but the future of this flag in its + entirety seems to be uncertain. + + [`cargo vendor`][wg-cargo-std-aware#23] did receive lots of discussion. + Vendoring the standard library is desirable (for the same reasons as any + vendoring), but would lock the user to a specific version of the toolchain + when using a vendored standard library. However, if the `rust-src` component + contained already-vendored dependencies, then `cargo vendor` would not need + to support build-std and users would see the same advantages. + + Vendored standard library dependencies were implemented using a hacky + approach (necessarily, prior to the standard library having its own + workspace), but this was later reverted due to bugs. No attempt has been made + to reimplement vendoring since the standard library has had its own + workspace. + +3. **Dependencies of the standard library** + + There are a handful of dependencies of the standard library that may pose + challenges for build-std by dint of needing a working C toolchain or + special-casing. + + [`libbacktrace`][wg-cargo-std-aware#16] previously required a C compiler to + build `backtrace-sys`, but now uses `gimli` internally. + + [`compiler_builtins`][wg-cargo-std-aware#15] has a `c` feature that uses C + versions of some intrinsics that are more optimised. This is used by the + pre-built standard library, and if not used by build-std, could be a point of + divergence. `compiler-builtins/c` can have a significant impact on code + quality and build size. It also has a `mem` feature which provides symbols + (`memcpy`, etc) for platforms without `std` that don't have these same + symbols provided by `libc`. `compiler_builtins` is also built with a large + number of compilation units to force each function into a different unit, + avoiding unintentionally bringing in a symbol that conflicts with one in the + system's `libgcc`. + + ['unwind'][wg-cargo-std-aware#29] links to the system's version of libunwind. + Enabling the `llvm-libunwind` feature, `-Clink-self-contained` or + `-Ctarget-feature=+crt-static` will statically link to the pre-built + `libunwind` distributed in the standard library component for the target, if + present. + + [Sanitizers][wg-cargo-std-aware#17], when enabled, require a sanitizer + runtime to be present. These are currently built by bootstrap and part of + LLVM. + +4. **Design considerations** + + There are many design considerations discussed in the [wg-cargo-std-aware] + repository: + + [wg-cargo-std-aware#5] explored how/if dependencies on the standard library + should be declared. The issue claims that users should have to opt-in to + build-std, support alternative standard library implementations, and that + Cargo needs to be able to pass `--extern` to rustc for all dependencies. + + It is an open question how to handle multiple dependencies each declaring a + dependency on the standard library. A preference towards unifying standard + library dependencies was expressed (these would have no concept of a version, + so just union all features). + + There was no consensus on how to find a balance between explicitly depending + on the standard library versus implicitly, or on whether the pre-built-ness + of a dependency should be surfaced to the user. + + [wg-cargo-std-aware#6] argues that target-spec-json would be de-facto stable + if it can be used by build-std on stable. While `--target=custom.json` can be + used on stable today, it effectively requires build-std and so a nightly + toolchain. As build-std enables custom targets to be used on stable, this + would effectively be a greater commitment to the current stability of custom + targets than currently exists and would warrant an explicit decision. + + [wg-cargo-std-aware#8] highlighted that a more-portable standard library + would be beneficial for build-std (i.e. a `std` that could build on any + target), but that making the standard library more portable isn't necessarily + in-scope for build-std. + + [wg-cargo-std-aware#11] investigated how build-std could get the standard + library sources. rustup can download `rust-src`, but there was a preference + expressed that rustup not be required. Cargo could have reasonable default + probing locations that could be used by distros and would include where + rustup puts `rust-src`. + + [wg-cargo-std-aware#12] concluded that the `Cargo.lock` of the standard + library would need to be respected so that the project can guarantee that the + standard library works with the project's current testing. + + [wg-cargo-std-aware#13] explored how to determine the default set of cfg + values for the standard library. This is currently computed by bootstrap. + This could be duplicated in Cargo in the short-term, made visible to + build-std through some configuration, or require the user to explicitly + declare them. + + [wg-cargo-std-aware#14] looks into additional rustc flags and environment + variables passed by bootstrap to the compiler. A comparison of the + compilation flags from bootstrap and build-std was + [posted in a comment][wg-cargo-std-aware#14-review]. No solutions were + suggested, other than that it may need a similar mechanism as + [wg-cargo-std-aware#13]. + + [wg-cargo-std-aware#29] tries to determine how to support different panic + strategies. Should Cargo use the profile to decide what to use? How does it + know which panic strategy crate to use? It is argued that Cargo ought to work + transparently - if the user sets the panic strategy differently then a + rebuild is triggered. + + [wg-cargo-std-aware#30] identifies that some targets have special handling in + bootstrap which will need to be duplicated in build-std. Targets could be + allowlisted or denylisted to avoid having to address this initially. + + [wg-cargo-std-aware#38] argues that a forced lock of the standard library + is desirable, to which there was no disagreement. This was more relevant + when build-std did not use the on-disk `Cargo.lock`. + + [wg-cargo-std-aware#39] explores the interaction between build-std and + public/private dependencies ([rfcs#3516]). Should the standard library always + be public? There were no solutions presented, only that if defined in + `Cargo.toml`, the standard library will likely inherit the default from that. + + [wg-cargo-std-aware#43] investigates the options for the UX of build-std. + `-Zbuild-std` flag is not a good experience as it needs added to every + invocation and has few extension points. Using build-std should be a unstable + feature at first. It was argued that build-std should be transparent and + happen automatically when Cargo determines it is necessary. There are + concerns that this could trigger too often and that it should only happen + automatically for ABI-modifying flags. + + [wg-cargo-std-aware#46] observes that some targets link against special + object flags (e.g. `crt1.o` on musl) and that build-std will need to handle + these without hardcoding target-specific logic. There were no conclusions, + but `-Clink-self-contained` might be able to help. + + [wg-cargo-std-aware#47] discusses how to handle targets that typically ship + with a different linker (e.g. `rust-lld` or `gcc`). `rust-lld` is now shipped + by default reducing the potential impact of this, though it is discovered via + the sysroot, and so will need to be found via another mechanism if disabled. + + [wg-cargo-std-aware#50] argues that the impact on build probes ought to be + considered and was later closed as t-cargo do not want to support build + probes. + + [wg-cargo-std-aware#51] plans for removal of `rustc-dep-of-std`, identifying + that if explicit dependencies on the standard library are adopted, that the + need for this feature could be made redundant. + + [wg-cargo-std-aware#68] notices that `profiler_builtins` needs to be compiled + after `core` (i.e. `core` can't be compiled with profiling). The error + message has been improved for this but there was otherwise no commentary. + This has changed since the issue was filed, as `profiler_builtins` is now a + `#![no_core]` crate. + + [wg-cargo-std-aware#85] considers that there has to be a deliberate testing + strategy in place between the [rust-lang/rust] and [rust-lang/cargo] + repositories to ensure there is no breakage. `rust-toolstate` could be used + but is not very good. Alternatively, Cargo could become a [JOSH] subtree of + [rust-lang/rust]. + + [wg-cargo-std-aware#86] proposes that the initial set of targets supported by + build-std be limited at first to further reduce scope and limit exposure to + the trickier issues. + + [wg-cargo-std-aware#88] reports that `cargo doc -Zbuild-std` doesn't generate + links to the standard library. Cargo doesn't think the standard library comes + from crates.io, and bootstrap isn't involved to pass + `-Zcrate-attr="doc(html_root_url=..)"` like in the pre-built standard + library. + + [wg-cargo-std-aware#90] asks how `restricted_std` should apply to custom + targets. `restricted_std` is triggered based on the `target_os` value, which + means it will apply for some custom targets but not others. build-std needs + to determine what guarantees are desirable/expected. Current implementation + wants slightly-modified-from-default target specs to be accepted and + completely new target specs to hit `restricted_std`. + + [wg-cargo-std-aware#92] suggests that some targets could be made "unstable" + and as such only support build-std on nightly. This forces users of those + targets to use nightly where they will receive more frequent fixes for their + target. It would also permit more experimentation with build-std while + enabling stabilisation for mainstream targets. + +5. **Implementation considerations** + These won't be discussed in this summary, see [the implementation summary][implementation-summary] + or [the relevant section of the literature review for more detail][appendix-ii-impl] + +6. **Bugs in the compiler or standard library** + These aren't especially relevant to this summary, see [the relevant section + of the literature review for more detail][appendix-ii-bugs] + +7. **Cargo feature requests narrowly applied to build-std** + These aren't especially relevant to this summary, see [the relevant section + of the literature review for more detail][appendix-ii-cargo-feats] + +Since around 2020, activity in the [wg-cargo-std-aware] repository largely +trailed off and there have not been any significant developments related to +build-std since. + +### Implementation summary +[implementation-summary]: #implementation-summary + +*An exhaustive review of implementation-related issues, pull requests and +discussions can be found in +[the relevant section of the literature review][appendix-ii-impl].* + +There has been an unstable and experimental implementation of build-std in Cargo +since August 2019 ([wg-cargo-std-aware#10]/[cargo#7216]). + +[cargo#7216] added the [`-Zbuild-std`][build-std] flag to Cargo. `-Zbuild-std` +re-builds the standard library crates which rustc then uses instead of the +pre-built standard library from the sysroot. + +Originally, `-Zbuild-std` always build `std` by default. Since the addition of +the `std` field to target metadata in [rust#122305], Cargo only builds `std` by +default if `metadata.std` is true. + +`test` is also built if `std` is being built and tests are being run with the +default harness. + +Optionally, users can provide the list of crates to be built, though this was +intended as an escape hatch to work around bugs - the arguments to the flag are +unstable since the names of crates comprising the standard library are not +stable. + +Cargo has a hardcoded list of what dependencies need to be added for a given +user-requested crate (i.e. `std` implies building `core`, `alloc`, +`compiler_builtins`, etc.). It is common for users to manually specify the +`panic_abort` crate. + +Originally, `-Zbuild-std` required that `--target` be provided +([wg-cargo-std-aware#25]) to force Cargo to use different sysroots for the host +and target , but this restriction was later resolved ([cargo#14317]). + +A second flag, [`-Zbuild-std-features`][build-std-features], was added in +[cargo#8490] and allows overriding the default Cargo features of the standard +library. Like the arguments to `-Zbuild-std`, this values accepted by this flag +are inherently unstable as the library team has not committed to any of the +standard library's Cargo features being stable. Features are enabled on the +`sysroot` crate and propagate down through the crate graph of the standard +library (e.g. `compiler-builtins-mem` is a feature in `sysroot`, `std`, +`alloc`, and `core` until `compiler_builtins`). + +build-std gets the source of the standard library from the `rust-src` rustup +component. This does not happen automatically and the user must ensure the +component has been downloaded themselves. Only the standard library crates from +the [rust-lang/rust] repository are included in the `rust-src` dependency (i.e. +none of the crates.io dependencies). + +When `-Zbuild-std` has been passed, Cargo creates a second workspace for the +standard library based on the `Cargo.{toml,lock}` from the `rust-src` component. +Originally this was an in-memory workspace, prior to the standard library having +a separate workspace from the compiler which could be used independently +([rust#128534]/[cargo#14358]). This workspace is then resolved separately and +the resolve is combined with the user's resolve to produce a dependency graph of +things to build with the user's crates depending on the standard library's +crates. Some additional work is done to deduplicate crates across the graph and +then this crate graph is used to drive work (usually rustc invocations) as +usual. This approach allows for build-time parallelism and sharing of crates +between the two separate resolves but does involve `build-std`-specific logic in +and around unit generation and is very unlike the rest of Cargo +([wg-cargo-std-aware#64]). + +Resolving the standard library separately from the user's crate helps guarantee +that the exact dependency versions of the pre-built standard library are used, +which is a key constraint ([wg-cargo-std-aware#12]). Locking the standard +library could also help ([wg-cargo-std-aware#38]). A consequence of this is that +each of the Cargo subcommands (e.g. `cargo metadata`) need to have special +support for build-std implemented, but this might be desirable. + +The standard library crates are considered non-local packages and so are not +compiled with incremental compilation or dep-info fingerprint tracking and any +warnings will be silenced. + +build-std provides newly-built standard library dependencies to rustc using +`--extern noprelude:$crate`. `noprelude` was added in [rust#67074] to support +build-std and ensure that loading from the sysroot and using `--extern` were +equivalent ([wg-cargo-std-aware#40]). Prior to the addition of `noprelude`, +build-std briefly created new sysroots and used those instead of `--extern` +([cargo#7421]). rustc can still try to load a crate from the sysroot if the user +uses it which is currently a common source of confusing "duplicate lang item" +errors (as the user ends up with build-std `core` and sysroot `core` +conflicting). + +Host dependencies like build scripts and `proc_macro` crates use the +existing pre-built standard library from the sysroot, so Cargo does not +pass `--extern` to those. + +Modifications to the standard library are not supported. While build-std +has no mechanism to detect or prevent modifications to the `rust-src` content, +rebuilds aren't triggered automatically on modifications. The user cannot +override dependencies in the standard library workspace with `[patch]` sections +of their `Cargo.toml`. + +To simplify build-std in Cargo, build-std wants to be able to always build +`std`, which is accomplished through use of the +[`unsupported` module in `std`'s platform abstraction layer][std-unsupported], +and `restricted_std`. `std` checks for unsupported targets in its +[`build.rs`][std-build.rs] and applies the `restricted_std` cfg which marks the +standard library as unstable for unsupported targets. + +Users can enable the `restricted_std` feature in their crates. This mechanism +has been noted as confusing ([wg-cargo-std-aware#87]) and has the issue that the +user cannot opt into the feature on behalf of dependencies +([wg-cargo-std-aware#69]). + +The initial implementation does not include support for build-std in many of +Cargo's subcommands including `metadata`, `clean`, `vendor`, `pkgid` and the +`-p` options for various commands. Support for `cargo fetch` was implemented in +[cargo#10129]. + +## `no_std` Usability +[no_std-usability]: #no_std-usability + +There are also issues related to the usability of `no_std` crates: + +- Discoverability of `no_std` crates is difficult with a mix of categories + (`no-std`) and keywords (`nostd`/`no_std`) that are not used consistently by + `no_std` crates ([crates.io#7306]). + +- `no_std` crates can accidentally and easily depend on crates that use `std` + which can result in build failures in some targets ([cargo#8798]). + +## Related work +[related-work]: #related-work + +There are a variety of ongoing efforts, ideas, RFCs or draft notes describing +features that are related or would be beneficial for build-std: + +- **[Opaque dependencies]**, [epage], May 2025 + - Introduces the concept of an opaque dependency that has its own + `Cargo.lock`, `RUSTFLAGS` and `profile` + - Opaque dependencies could enable a variety of build-time performance + improvements: + - Caching - differences in dependency versions can cause unique instances of + every dependent crate + - Pre-built binaries - can leverage a pre-built artifact for a given opaque + dependency + - e.g. the standard library's distributed `rlib`s + - MIR-only/cross-crate lazy compilation - Small dependencies could be built + lazily and larger dependencies built once + - Optimising dependencies - dependencies could always be optimised when they + are unlikely to be needed during debugging + +[motivation]: ./3-motivation.md +[appendix-ii]: ./9-appendix-literature-review.md +[appendix-ii-impl]: ./9-appendix-literature-review.md#implementation +[appendix-ii-bugs]: ./9-appendix-literature-review.md#bugs-in-the-compiler-or-standard-library +[appendix-ii-cargo-feats]: ./9-appendix-literature-review.md#cargo-feature-requests-narrowly-applied-to-build-std + +[JOSH]: https://josh-project.github.io/josh/intro.html +[rust-lang/cargo]: https://github.com/rust-lang/cargo +[rust-lang/rust]: https://github.com/rust-lang/rust +[wg-cargo-std-aware]: https://github.com/rust-lang/wg-cargo-std-aware +[xargo]: https://github.com/japaric/xargo + +[Opaque dependencies]: https://hackmd.io/@epage/ByGfPtRell + +[cargo#10129]: https://github.com/rust-lang/cargo/pull/10129 +[cargo#14317]: https://github.com/rust-lang/cargo/pull/14317 +[cargo#14358]: https://github.com/rust-lang/cargo/pull/14358 +[cargo#4959]: https://github.com/rust-lang/cargo/issues/4959 +[cargo#7216]: https://github.com/rust-lang/cargo/pull/7216 +[cargo#7421]: https://github.com/rust-lang/cargo/pull/7421 +[cargo#8490]: https://github.com/rust-lang/cargo/pull/8490 +[cargo#8798]: https://github.com/rust-lang/cargo/issues/8798 +[crates.io#7306]: https://github.com/rust-lang/crates.io/pull/7306 +[rfcs#1133]: https://github.com/rust-lang/rfcs/pull/1133 +[rfcs#2663]: https://github.com/rust-lang/rfcs/pull/2663 +[rfcs#3516]: https://rust-lang.github.io/rfcs/3516-public-private-dependencies.html +[rust#122305]: https://github.com/rust-lang/rust/pull/128534 +[rust#128534]: https://github.com/rust-lang/rust/pull/128534 +[rust#67074]: https://github.com/rust-lang/rust/issues/67074 +[wg-cargo-std-aware#10]: https://github.com/rust-lang/wg-cargo-std-aware/issues/10 +[wg-cargo-std-aware#11]: https://github.com/rust-lang/wg-cargo-std-aware/issues/11 +[wg-cargo-std-aware#12]: https://github.com/rust-lang/wg-cargo-std-aware/issues/12 +[wg-cargo-std-aware#13]: https://github.com/rust-lang/wg-cargo-std-aware/issues/13 +[wg-cargo-std-aware#14-review]: https://github.com/rust-lang/wg-cargo-std-aware/issues/14#issuecomment-2315878717 +[wg-cargo-std-aware#14]: https://github.com/rust-lang/wg-cargo-std-aware/issues/14 +[wg-cargo-std-aware#15]: https://github.com/rust-lang/wg-cargo-std-aware/issues/15 +[wg-cargo-std-aware#16]: https://github.com/rust-lang/wg-cargo-std-aware/issues/16 +[wg-cargo-std-aware#17]: https://github.com/rust-lang/wg-cargo-std-aware/issues/17 +[wg-cargo-std-aware#19]: https://github.com/rust-lang/wg-cargo-std-aware/issues/19 +[wg-cargo-std-aware#20]: https://github.com/rust-lang/wg-cargo-std-aware/issues/20 +[wg-cargo-std-aware#21]: https://github.com/rust-lang/wg-cargo-std-aware/issues/21 +[wg-cargo-std-aware#22]: https://github.com/rust-lang/wg-cargo-std-aware/issues/22 +[wg-cargo-std-aware#23]: https://github.com/rust-lang/wg-cargo-std-aware/issues/23 +[wg-cargo-std-aware#24]: https://github.com/rust-lang/wg-cargo-std-aware/issues/24 +[wg-cargo-std-aware#25]: https://github.com/rust-lang/wg-cargo-std-aware/issues/25 +[wg-cargo-std-aware#26]: https://github.com/rust-lang/wg-cargo-std-aware/issues/26 +[wg-cargo-std-aware#29]: https://github.com/rust-lang/wg-cargo-std-aware/issues/29 +[wg-cargo-std-aware#2]: https://github.com/rust-lang/wg-cargo-std-aware/issues/2 +[wg-cargo-std-aware#30]: https://github.com/rust-lang/wg-cargo-std-aware/issues/30 +[wg-cargo-std-aware#36]: https://github.com/rust-lang/wg-cargo-std-aware/issues/36 +[wg-cargo-std-aware#38]: https://github.com/rust-lang/wg-cargo-std-aware/issues/38 +[wg-cargo-std-aware#39]: https://github.com/rust-lang/wg-cargo-std-aware/issues/39 +[wg-cargo-std-aware#3]: https://github.com/rust-lang/wg-cargo-std-aware/issues/3 +[wg-cargo-std-aware#40]: https://github.com/rust-lang/wg-cargo-std-aware/issues/40 +[wg-cargo-std-aware#43]: https://github.com/rust-lang/wg-cargo-std-aware/issues/43 +[wg-cargo-std-aware#45]: https://github.com/rust-lang/wg-cargo-std-aware/issues/45 +[wg-cargo-std-aware#46]: https://github.com/rust-lang/wg-cargo-std-aware/issues/46 +[wg-cargo-std-aware#47]: https://github.com/rust-lang/wg-cargo-std-aware/issues/47 +[wg-cargo-std-aware#4]: https://github.com/rust-lang/wg-cargo-std-aware/issues/4 +[wg-cargo-std-aware#50]: https://github.com/rust-lang/wg-cargo-std-aware/issues/50 +[wg-cargo-std-aware#51]: https://github.com/rust-lang/wg-cargo-std-aware/issues/51 +[wg-cargo-std-aware#5]: https://github.com/rust-lang/wg-cargo-std-aware/issues/5 +[wg-cargo-std-aware#64]: https://github.com/rust-lang/wg-cargo-std-aware/issues/64 +[wg-cargo-std-aware#68]: https://github.com/rust-lang/wg-cargo-std-aware/issues/68 +[wg-cargo-std-aware#69]: https://github.com/rust-lang/wg-cargo-std-aware/issues/69 +[wg-cargo-std-aware#6]: https://github.com/rust-lang/wg-cargo-std-aware/issues/6 +[wg-cargo-std-aware#7]: https://github.com/rust-lang/wg-cargo-std-aware/issues/7 +[wg-cargo-std-aware#85]: https://github.com/rust-lang/wg-cargo-std-aware/issues/85 +[wg-cargo-std-aware#86]: https://github.com/rust-lang/wg-cargo-std-aware/issues/86 +[wg-cargo-std-aware#87]: https://github.com/rust-lang/wg-cargo-std-aware/issues/87 +[wg-cargo-std-aware#88]: https://github.com/rust-lang/wg-cargo-std-aware/issues/88 +[wg-cargo-std-aware#8]: https://github.com/rust-lang/wg-cargo-std-aware/issues/8 +[wg-cargo-std-aware#90]: https://github.com/rust-lang/wg-cargo-std-aware/issues/90 +[wg-cargo-std-aware#92]: https://github.com/rust-lang/wg-cargo-std-aware/issues/92 + +[build-std-features]: https://doc.rust-lang.org/cargo/reference/unstable.html#build-std-features +[build-std]: https://doc.rust-lang.org/cargo/reference/unstable.html#build-std +[std-build.rs]: https://github.com/rust-lang/rust/blob/f315e6145802e091ff9fceab6db627a4b4ec2b86/library/std/build.rs#L17 +[std-unsupported]: https://github.com/rust-lang/rust/blob/f768dc01da9a681716724418ccf64ce55bd396c5/library/std/src/sys/pal/mod.rs#L68-L69 + +[Ericson2314]: https://github.com/Ericson2314 +[epage]: https://github.com/epage \ No newline at end of file diff --git a/text/0000-build-std/3-motivation.md b/text/0000-build-std/3-motivation.md new file mode 100644 index 00000000000..82eb3579d3e --- /dev/null +++ b/text/0000-build-std/3-motivation.md @@ -0,0 +1,151 @@ +# Motivation +[motivation]: #motivation + +> [!IMPORTANT] +> +> This section lists all of the motivations that have been associated with +> build-std in its various iterations, but not all of these use cases will be +> addressed by this proposal. +> +> The motivations that will not be addressed are nevertheless mentioned here so +> that reviewers have a more complete context for what has and hasn't been +> desired of build-std over time. + +While the pre-built standard library has been sufficient for the majority of +Rust users, there are a variety of use-cases which require the ability to +re-build the standard library. + +This RFC aims to support the following use cases on stable: + +1. **Building the standard library without relying on unstable escape hatches** + + - While tangential to the core of build-std as a feature, projects like Rust + for Linux want to be able to build crates from the standard library using + a stable toolchain without relying on escape hatches like + `RUSTC_BOOTSTRAP` that the Rust project does not encourage use of + + - It is relatively straightforward to support this, hence its inclusion + + - Cargo's implementation of build-std should be able to re-use whichever + mechanism is designed to address this + +2. **Building standard library crates that are not shipped for a target** + + - Targets which have limited `std` support may wish to use the subsets of + the standard library which could work but are not shipped by the project + (e.g. `std` on `x86_64-unknown-none`) + +3. **Using the standard library with tier three targets** + + - There is no stable mechanism for using the standard library on a tier + three target that does not ship a pre-built std + + - While it is common for these targets to not support the standard library, + they should be able to use `core` + + - These users are forced to use nightly and the unstable `-Zbuild-std` + feature or third-party tools like [cargo-xbuild] (formerly [xargo]) + +Follow-up RFCs will aim to support the following motivations: + +1. **Unblock stabilisation of ABI-modifying compiler flags** + + - Any compiler flags which change the ABI cannot currently be stabilised as + they would immediately mismatch with the pre-built standard library + + - Without an ability to rebuild the standard library using these flags, it + is impossible to use them effectively and safely if stabilised + + - ABI-modifying flags are designated as target modifiers + ([rfcs#3716]/[rust#136966]) and require that the same value for the flag + is passed to all compilation units + + - Flags which need to be set across the entire crate graph to uphold some + property (i.e. enhanced security) are also target modifiers + + - For example: sanitizers, control flow integrity, `-Zfixed-x18`, etc + +1. **Re-building the standard library with different codegen flags or profile** + ([wg-cargo-std-aware#2]) + + - Embedded users need to optimise aggressively for size, due to the limited + space available on their target platforms, which can be achieved in Cargo + by setting `opt-level = s/z` and `panic = "abort"` in their profile. + However, these settings will not apply to the pre-built standard library + + - Similarly, when deploying to known environments, use of `target-cpu` or + `target-feature` can improve the performance of code generation or allow + the use of newer hardware features than the target's baseline provides. As + above, these configuration will not apply to the pre-built standard + library + + - While the pre-built standard library is built to support debugging without + compromising size and performance by setting `debuginfo=1`, this isn't + ideal, and building the standard library with the dev profile would + provide a better experience when debugging + +The following use cases are not supported by this RFC, but could be supported +with follow-up RFCs (and this RFC will attempt to ensure they remain viable as +future possibilities): + +1. **Using the standard library with custom targets** + + - There is no stable mechanism for using the standard library for a custom + target (using target-spec-json) + + - Like tier three targets, these targets often only support `core` and are + forced to use nightly today + +2. **Enabling Cargo features for the standard library** ([wg-cargo-std-aware#4]) + + - There are opportunities to expose Cargo features from the standard library + that would be useful for certain subsets of the Rust users. + + - For example, embedded users may want to enable a feature like + `optimize_for_size` or `panic_immediate_abort` to reduce binary size + +3. **Using miri on a stable toolchain** + + - Using miri requires building the standard library with specific compiler + flags that would not be appropriate for the pre-built standard library, so + is forced to require nightly and build its own sysroot + +Some use cases are unlikely to supported by the project unless a new and +compelling use-case is presented, and so this RFC may make decisions which make +these motivations harder to solve in future: + +1. **Modifying the source code of the standard library** ([wg-cargo-std-aware#7]) + + - Some platforms require a heavily modified standard library that would not + be suitable for upstreaming, such as [Apache's SGX SDK][sgx] which + replaces some standard library and ecosystem crates with forks or custom + crates for a custom `x86_64-unknown-linux-sgx` target + + - Similarly, some tier three targets may wish to patch standard library + dependencies to add or improve support for the target + + - If a stable mechanism were provided to make such changes to the standard + library, then this would constrain future standard library development. + These changes are better attempted by maintaining a fork of the standard + library. + +2. **Retire the concept of the sysroot** + + - Earlier proposals for build-std were motivated in-part by the desire to see + the concept of the sysroot retired. + + - This is challenging while maintaining backwards-compatibility, + especially for users who do not use Cargo and assume rustc can find the + standard library in the sysroot. Removing the sysroot has no advantages + to the end-user of Rust in itself. + +[cargo-xbuild]: https://github.com/rust-osdev/cargo-xbuild +[xargo]: https://github.com/japaric/xargo + +[rfcs#3716]: https://rust-lang.github.io/rfcs/3716-target-modifiers.html +[rust#136966]: https://github.com/rust-lang/rust/issues/136966 +[wg-cargo-std-aware#2]: https://github.com/rust-lang/wg-cargo-std-aware/issues/2 +[wg-cargo-std-aware#4]: https://github.com/rust-lang/wg-cargo-std-aware/issues/4 +[wg-cargo-std-aware#7]: https://github.com/rust-lang/wg-cargo-std-aware/issues/7 + +[sgx]: https://github.com/apache/incubator-teaclave-sgx-sdk \ No newline at end of file diff --git a/text/0000-build-std/4-build-std-always.md b/text/0000-build-std/4-build-std-always.md new file mode 100644 index 00000000000..d1a7d37edbf --- /dev/null +++ b/text/0000-build-std/4-build-std-always.md @@ -0,0 +1,1402 @@ +# `build-std=always` + +This part of the RFC proposes adding a `build-std = "always|never"` option to +the Cargo configuration which will unconditionally rebuild the standard library +crates listed in a new `build-std-crates` option. It also describes how Cargo +(or external tools) should build the standard library crates on stable (i.e., +which flags to pass and features to enable). + +This is aimed at supporting the following [motivations](./3-motivation.md): + +- Building the standard library without relying on unstable escape hatches +- Building standard library crates that are not shipped for a target +- Using the standard library with tier three targets + +# Proposal +[proposal]: #proposal + +Cargo configuration will contain a new key `build-std` under the `[build]` +section ([?][rationale-build-std-in-config]), permitting one of two values - +"never" ([?][rationale-build-std-never]) or "always", defaulting to "never": + +```toml +[build] +build-std = "never" # or `always` +``` + +`build-std` can also be specified in the `[target.]` and +`[target.]` sections ([?][rationale-build-std-target-section]): + +```toml +[target.aarch64-unknown-illumos] +build-std = "never" # or `always` +``` + +The `build-std` configuration locations have the following precedence +([?][rationale-build-std-precedence]): + +1. `[target.]` +2. `[target.]` +3. `[build]` + +As the Cargo configuration is local to the current user (typically in +`.config/cargo.toml` in the project root and/or Cargo home directory), the value +of `build-std` is not influenced by the dependencies of the current crate. + +When `build-std` is set to "always", then the standard library will be +unconditionally recompiled ([?][rationale-unconditional]) in the release profile +defined in its workspace as part of every clean build +([?][rationale-release-profile]). This is primarily useful for users of tier +three targets. As with other dependencies, the standard library's build will +respect the `RUSTFLAGS` environment variable. + +> [!NOTE] +> +> Configuration of the pre-built standard library is split across bootstrap and +> the Cargo packages for the standard library. As much of this configuration as +> possible should be moved to the Cargo profile for these packages so that the +> artifacts produced by build-std match the pre-built standard library as much +> as is feasible. + +Alongside `build-std`, a `build-std-crates` key will be introduced +([?][rationale-build-std-crate]), which can be used to specify which crates from +the standard library should be built. Only "core", "alloc" and "std" are valid +values for `build-std-crates`. + +```toml +[build] +build-std-crates = "std" +``` + +A value of "std" means that every crate in the graph has a direct dependency on +`std`, `alloc` and `core`. Similarly, "alloc" means `alloc` and `core`, and +"core" means just `core`. + +If `std` is to be built and Cargo is building a test or benchmark using the +default test harness then Cargo will also build the `test` crate. + +If [*Standard library dependencies*][deps] are implemented then `builtin` +dependencies will be used if `build-std-crates` is not explicitly set. +Otherwise, `build-std-crates` will default to the crate intended to be supported +by the target (see later +[*Standard library crate stability*][standard-library-crate-stability] section). + +> [!NOTE] +> +> Inspired by the concept of [opaque dependencies][Opaque dependencies], the +> standard library is resolved differently to other dependencies: +> +> - The lockfile included in the standard library source will be used when +> resolving the standard library's dependencies ([?][rationale-lockfile]). +> +> - The dependencies of the standard library crates are entirely opaque to the +> user. Different semver-compatible versions of these dependencies can +> exist in the user's resolve. The user cannot control compilation any of +> the dependencies of the `core`, `alloc` or `std` standard library crates +> individually (via profile overrides, for example). +> +> - The release profile defined by the standard library will be used. +> +> - This profile will be updated to match the current compilation options +> used by the pre-built standard library as much as possible (e.g. using +> `-Cembed-bitcode=yes` to support LTO). +> +> - Standard library crates and their dependencies from `build-std-crates` +> cannot be patched/replaced by the user in the Cargo manifest or config +> (e.g. using source replacement, `[replace]` or `[patch]`) +> +> Cargo will resolves the dependencies of opaque dependencies, such as the +> standard library, separately in their own workspaces. The root of such a +> resolve will be the crates specified in `build-std-crates` or, if +> [*Standard library dependencies*][deps] is implemented, the unified set of +> packages that any crate in the dependency has a direct dependency on. A +> dependency on the relevant roots are added to all crates in the main resolve. +> +> Regardless of which standard library crates are being built, Cargo will build +> the `sysroot` crate of the standard library workspace. `alloc` and `std` will +> be optional dependencies of the `sysroot` crate which will be enabled when the +> user has requested them. Panic runtimes are dependencies of `std` and will be +> enabled depending on the features that Cargo passes to `std` (see +> [*Panic strategies*][panic-strategies]). +> +> rustc loads panic runtimes in a different way to most dependencies, and +> without looking in the sysroot they will fail to load correctly unless passed +> in with `--extern`. rustc will need to be patched to be able to load panic +> runtimes from `-L dependency=` paths in line with other transitive +> dependencies. +> +> The standard library will always be a non-incremental build +> ([?][rationale-incremental]), Cargo's dep-info fingerprint tracking will not +> track the standard library crate sources, Cargo's `.d` dep-info file will not +> include standard library crate sources, and only a `rlib` produced (no +> `dylib`) ([?][rationale-no-dylib]). It will be built in the Cargo `target` +> directory of the crate or workspace like any other dependency. +> +> Standard library crates are provided to the compiler using the `--extern` flag +> with the `noprelude` modifier ([?][rationale-noprelude-with-extern]). + +The host pre-built standard library will always be used for procedural macros +and build scripts ([?][rationale-host-deps-cross], +[?][rationale-host-deps-host]). Multi-target projects (resulting from the +`target` field in artifact dependencies or the use of `per-pkg-target` fields) +may result in the standard library being built multiple times - once for each +target in the project. + +*See the following sections for rationale/alternatives:* + +- [*Why put `build-std` in the Cargo config?*][rationale-build-std-in-config] +- [*Why accept `never` as a value for `build-std`?*][rationale-build-std-never] +- [*Why add `build-std` to the `[target.]` and `[target.]` sections?*][rationale-build-std-target-section] +- [*Why does `[target]` take precedence over `[build]` for `build-std`?*][rationale-build-std-precedence] +- [*Why does "always" rebuild unconditionally?*][rationale-unconditional] +- [*Why does "always" rebuild in release profile?*][rationale-release-profile] +- [*Why add `build-std-crates`?*][rationale-build-std-crate] +- [*Why use the lockfile of the `rust-src` component?*][rationale-lockfile] +- [*Why not build the standard library in incremental?*][rationale-incremental] +- [*Why not produce a `dylib` for the standard library?*][rationale-no-dylib] +- [*Why use `noprelude` with `--extern`?*][rationale-noprelude-with-extern] +- [*Why use the pre-built standard library for procedural macros and build scripts in host mode?*][rationale-host-deps-host] +- [*Why use the pre-built standard library for procedural macros and build scripts in cross-compile mode?*][rationale-host-deps-cross] + +*See the following sections for relevant unresolved questions:* + +- [*What should the `build-std` configuration in `.cargo/config` be named?*][unresolved-config-name] +- [*What should the "always" and "never" values of `build-std` be named?*][unresolved-config-values] +- [*What should `build-std-crates` be named?*][unresolved-build-std-crate-name] +- [*Should the standard library inherit RUSTFLAGS?*][unresolved-inherit-rustflags] + +*See the following sections for future possibilities:* + +- [*Allow reusing sysroot artifacts if available*][future-reuse-sysroot] + +## Standard library crate stability +[standard-library-crate-stability]: #standard-library-crate-stability + +An optional `standard_library_support` field +([?][rationale-why-standard-library-support]) is added to the target +specification ([?][rationale-target-spec-purpose]), replacing the existing +`metadata.std` field. `standard_library_support` has two fields: + +- `supported`, which can be set to either "core", "core and alloc", or + "core, alloc and std" +- `default`, which can be set to either "core", "core and alloc", or + "core, alloc and std" + - `default` cannot be set to a value which is "less than" that of `supported` + (i.e. "core and alloc" when `supported` was only set to "core") + +The `supported` field determines which standard library crates Cargo will permit +to be built for this target on a stable toolchain. On a nightly toolchain, Cargo +will build whichever standard library crates are requested by the user. + +The `default` field determines which crate will be built by Cargo if +`build-std = "always"` and `build-std-crates` is not set. Users can specify +`build-std-crates` to build more crates than included in the `default`, as long +as those crates are included in `supported`. + +The correct value for `standard_library_support` is independent of the tier of +the target and depends on the set of crates that are intended to work for a +given target, according to its maintainers. + +If `standard_library_support` is unset for a target, then Cargo will not permit +any standard library crates to be built for the target on a stable toolchain. It +will be required to use a nightly toolchain to use build-std with that target. + +Cargo's `build-std-crates` field will default to the value of the +`standard_library_support.default` field (`std` for "core, alloc and std", +`alloc` for "core and alloc", and `core` for "core"). This does not prevent +users from building more crates than the default, it is only intended to be a +sensible default for the target that is probably what the user expects. + +The `target-standard-library-support` option will be supported by rustc's +`--print` flag and will be used by Cargo to query this value for a given target: + +```shell-session +$ rustc --print target-standard-library-support --target armv7a-none-eabi +default: core +supported: core, alloc +$ rustc --print target-standard-library-support --target aarch64-unknown-linux-gnu +default: std +supported: core, alloc, std +``` + +*See the following sections for rationale/alternatives:* + +- [*Why introduce `standard_library_support`?*][rationale-why-standard-library-support] +- [*Should target specifications own knowledge of which standard library crates are supported?*][rationale-target-spec-purpose] + +## Interactions with `#![no_std]` +[interactions-with-no_std]: #interactions-with-no_std + +Behaviour of crates using `#![no_std]` will not change whether or not `std` is +rebuilt and passed via `--extern` to rustc, and `#![no_std]` will still be +required in order for `rustc` to not attempt to load `std` and add it to the +extern prelude. [*Standard library dependencies*][deps] describes a future +possibility for how the `no_std` mechanism could be replaced. + +*See the following sections for future possibilities:* + +- [*Replace `#![no_std]` as the source-of-truth for whether a crate depends on `std`*][future-replace-no_std] + +## `restricted_std` +[restricted_std]: #restricted_std + +The existing `restricted_std` mechanism will be removed from `std`'s +[`build.rs`][std-build.rs]. + +*See the following sections for rationale/alternatives:* + +- [*Why remove `restricted_std`?*][rationale-remove-restricted-std] + +## Custom targets +[custom-targets]: #custom-targets + +Cargo will detect when the standard library is to be built for a custom target +and will emit an error ([?][rationale-disallow-custom-targets]). + +> [!NOTE] +> +> Cargo could detect use of a custom target either by comparing it with the list +> of built-in targets that rustc reports knowing about (via `--print target-list`) +> or by checking if a file exists at the path matching the provided target name. +> +> This does not require any changes to rustc. If it is invoked to build the +> standard library then it will continue to do so, as is possible today, it is +> only the build-std functionality in Cargo that will not support custom targets +> initially. + +Furthermore, custom targets will be destabilised in rustc (as in [rust#71009]). +This will not be a significant breaking change as custom targets cannot +effectively be used currently without nightly (needing build-std to have +`core`). + +Custom targets can still be used with build-std on nightly toolchains provided +that `-Zunstable-options` is provided to Cargo. + +*See the following sections for rationale/alternatives:* + +- [*Why disallow custom targets?*][rationale-disallow-custom-targets] + +*See the following sections for future possibilities:* + +- [*Allow custom targets with build-std*][future-custom-targets] + +## Preventing implicit sysroot dependencies +[preventing-implicit-sysroot-dependencies]: #preventing-implicit-sysroot-dependencies + +Cargo will pass a new flag to rustc which will prevent rustc from loading +top-level dependencies from the sysroot ([?][rationale-root-sysroot-deps]). + +> [!NOTE] +> +> rustc could add a `--no-implicit-sysroot-deps` flag with this behaviour. For +> example, writing `extern crate foo` in a crate will not load `foo.rlib` from +> the sysroot if it is present, but if an `--extern noprelude:bar.rlib` is +> provided which depends on a crate `foo`, rustc will look in +> `-L dependency=...` paths and the sysroot for it. + +*See the following sections for rationale/alternatives:* + +- [*Why prevent rustc from loading root dependencies from the sysroot?*][rationale-root-sysroot-deps] + +## Vendored `rust-src` +[vendored-rust-src]: #vendored-rust-src + +When it is necessary to build the standard library, Cargo will look for sources +in a fixed location in the sysroot ([?][rationale-custom-src-path]): +`lib/rustlib/src`. rustup's `rust-src` component downloads standard library +sources to this location and will be made a default component. If the sources +are not found, Cargo will emit an error and recommend the user download +`rust-src` if using rustup. + +`rust-src` will contain the sources for the standard library crates as well as +its vendored dependencies ([?][rationale-vendoring]). As a consequence sources +of standard library dependencies will not need be fetched from crates.io. + +> [!NOTE] +> +> Cargo will not perform any checks to ensure that the sources in `rust-src` +> have been modified ([?][rationale-src-modifications]). It will be documented +> that modifying these sources is not supported. + +*See the following sections for rationale/alternatives:* + +- [*Why not allow the source path for the standard library be customised?*][rationale-custom-src-path] +- [*Why vendor standard library dependencies?*][rationale-vendoring] +- [*Why not check if `rust-src` has been modified?*][rationale-src-modifications] + +*See the following sections for relevant unresolved questions:* + +- [*Should `rust-src` be a default component?*][unresolved-rust-src] + +## Panic strategies +[panic-strategies]: #panic-strategies + +Panic strategies are unlike other profile settings insofar as they influence +which crates are built and which flags are passed to the standard library build. +For example, if `panic = "unwind"` were set in the Cargo profile then the +`panic_unwind` feature would need to be provided to `std` and `-Cpanic=unwind` +passed to suggest that the compiler use that panic runtime. + +If Cargo is not building `std`, then neither of the panic runtimes will be +built. In this circumstance rustc will continue to throw an error when a +unwinding panic strategy is chosen. + +If Cargo would build `std` for a project then Cargo's behaviour depends on +whether or not `panic` is set in the profile: + +- If `panic` is not set in the profile then unwinding may still be the default + for the target and Cargo will need to enable the `panic_unwind` feature to the + `sysroot` crate to build `panic_unwind` just in case it is used + +- If `panic` is set to "unwind" then the `panic_unwind` feature of `sysroot` + will be enabled and `-Cpanic=unwind` will be passed + +- If `panic` is set to "abort" then `-Cpanic=abort` will be passed + + - `panic_abort` is a non-optional dependency of `std` so it will always be + built + +Tests, benchmarks, build scripts and proc macros continue to ignore the "panic" +setting and `panic = "unwind"` is always used - which means the standard library +needs to be recompiled again if the user is using "abort". Once +`panic-abort-tests` is stabilised, the standard library can be built with the +profile's panic strategy even for tests and benchmarks. + +In line with Cargo's stance on not parsing the `RUSTFLAGS` environment variable, +it will not be checked for compilation flags that would require additional +crates to be built for compilation to succeed. + +> [!NOTE] +> +> The `unwind` crate will continue to link to the system's `libunwind` which +> will need to match the target modifiers used by the standard library to avoid +> incompatibilities. Likewise, if `llvm-libunwind`, `-Clink-self-contained=yes` +> or `-Ctarget-feature=+crt-static` are used and the distributed `libunwind` is +> used then it will also need to match the target modifiers of the standard +> library to avoid incompatibilities. + +*See the following sections for future possibilities:* + +- [*Avoid building `panic_unwind` unnecessarily*][future-panic_unwind] + +## Building the standard library on a stable toolchain +[building-the-standard-library-on-a-stable-toolchain]: #building-the-standard-library-on-a-stable-toolchain + +rustc will automatically assume `RUSTC_BOOTSTRAP` when the source path of the +crate being compiled is within the same sysroot as the rustc binary being +invoked ([?][rationale-implied-bootstrap]). Cargo will not need to use +`RUSTC_BOOTSTRAP` when compiling the standard library with a stable toolchain. +The standard library's dependencies will not be permitted to use build probes to +detect whether a nightly version is being used. + +*See the following sections for rationale/alternatives:* + +- [*Why allow building from the sysroot with implied `RUSTC_BOOTSTRAP`?*][rationale-implied-bootstrap] + +## Self-contained objects +[self-contained-objects]: #self-contained-objects + +A handful of targets require linking against special object files, such as +`windows-gnu`, `linux-musl` and `wasi` targets. For example, `linux-musl` +targets require `crt1.o`, `crti.o`, `crtn.o`, etc. + +Since [rust#76158]/[compiler-team#343], the compiler has a stable +`-Clink-self-contained` flag which will look for special object files in +expected locations, typically populated by the `rust-std` components. Its +behaviour can be forced by `-Clink-self-contained=true`, but is force-enabled +for some targets and inferred for others. + +Rust will ship `rust-self-contained` components for any targets which +need it. These components will contain the special object files normally +included in `rust-std`, and will be distributed for all tiers of targets. While +generally these objects are specific to the architecture and C runtime (CRT) +(and so `rust-self-contained-$arch-$crt` could be sufficient and result in fewer +overall components), it's technically possible that Rust could support two +targets with the same architecture and same CRT but different versions of the +CRT, so having target-specific components is most future-proof. These would +replace the `self-contained` directory in existing `rust-std` components. + +Similarly, for any architectures which require it, LLVM's `libunwind` will be +built and shipped in the `rust-self-contained` component. + +As long as these components have been downloaded, as well as any other support +components, such as `rust-mingw`, rustc's `-Clink-self-contained` will be able +to link against the object files and build-std should never fail on account of +missing special object files. rustc will attempt to detect when +`rust-self-contained` components are missing and provide helpful diagnostics in +this case. + +`-Clink-self-contained` also controls whether rustc uses the linker shipped with +Rust. build-std's use of `-Clink-self-contained` will endeavour to ensure that +the whatever the default linker for the current target is (self-contained or +otherwise) will be used. + +*See the following sections for future possibilities:* + +- [*Enable local recompilation of special object files/sanitizer runtimes*][future-recompile-special] + +## `compiler-builtins` +[compiler-builtins]: #compiler-builtins + +`compiler-builtins` is always built with `-Ccodegen-units=10000` to force each +intrinsic into its own object file to avoid symbol clashes with libgcc. This is +currently enforced with a profile override in the standard library's workspace +and is unchanged. + +See [*Allow local builds of `compiler-rt` intrinsics*][future-compiler-builtins-c] +for discussion of the `compiler-builtins-c` feature. + +### `compiler-builtins/mem` +[compiler-builtins-mem]: #compiler-builtinsmem + +It is not possible to use weak linkage to make the symbols provided by +`compiler_builtins/mem` trivially overridable in every case +([?][rationale-no-weak-linkage]). + +The `mem` feature of `compiler_builtins` will be inverted to a new feature named +`external-mem` ([?][rationale-no-mem]). This will not be a default feature, so +`compiler_builtins` will provide mem symbols unless the `external-mem` is +provided. + +`std`, which provides memory symbols via `libc`, will depend on the +`external-mem` feature. Most `no_std` users will use the `compiler_builtins` +implementation of these symbols and will work by default when they do not depend +on `std`. + +Those users providing their own mem symbols can override on weak linkage of the +`compiler_builtins` symbols. or use a nightly toolchain to enable the +`external-mem` feature of an explicit dependency on the standard library (per +[*Standard library dependencies*][deps]). + +*See the following sections for rationale/alternatives:* + +- [*Why not rely on weak linkage for `compiler-builtins/mem` symbols?*][rationale-no-weak-linkage] +- [*Why invert the `mem` feature?*][rationale-no-mem] + +## `profiler-builtins` +[profiler-builtins]: #profiler-builtins + +`profiler-builtins` will not be built by build-std, thus preventing +profile-guided optimisation with a locally-built standard library. +`profiler-builtins` has native dependencies which may fail compilation of the +standard library if missing were `profiler-builtins` to be built by default as +part of the standard library build. + +*See the following sections for future possibilities:* + +- [*Allow building `profiler-builtins`*][future-profiler-builtins] + +## Caching +[caching]: #caching + +Standard library artifacts built by build-std will be reused equivalently to +today's crates/dependencies that are built within a shared target directory. By +default, this limits sharing to a single workspace ([?][rationale-caching]). + +*See the following sections for rationale/alternatives:* + +- [*Why not globally cache builds of the standard library?*][rationale-caching] + +## Generated documentation +[generated-documentation]: #generated-documentation + +When running `cargo doc` for a project to generate documentation and rebuilding +the standard library, the generated documentation for the user's crates will +link to the locally generated documentation for the `core`, `alloc` and `std` +crates, rather than the upstream hosted generation as is typical for non-locally +built standard libraries. + +*See the following sections for rationale/alternatives:* + +- [*Why not link to hosted standard library documentation in generated docs?*][rationale-generated-docs] + +## Cargo subcommands +[cargo-subcommands]: #cargo-subcommands + +Any Cargo command which accepts a package spec with `-p` will not recognise +`core`, `alloc`, `std` or none of their dependencies (unless +[*Standard library dependencies*][deps] is implemented). Many of Cargo's +subcommands will need modification to support build-std: + +[`cargo clean`][cargo-clean] will additionally delete any builds of the standard +library performed by build-std. + +[`cargo fetch`][cargo-fetch] will not fetch the standard library dependencies as +they are already vendored in the `rust-src` component. + +[`cargo miri`][cargo-miri] is not built into Cargo, it is shipped by miri, but +is mentioned in Cargo's documentation. `cargo miri` is unchanged by this RFC, +but build-std is one step towards `cargo miri` requiring less special support. + +> [!NOTE] +> +> `cargo miri` could be re-implemented using build-std to enable a `miri` +> profile and always rebuild. The `miri` profile would be configured in the +> standard library's workspace, setting the flags/options necessary for `miri`. + +[`cargo report`][cargo-report] will not include reports from the standard +library crates or their dependencies. + +[`cargo update`][cargo-update] will not update the dependencies of `std`, +`alloc` and `core`, as these are vendored as part of the distribution of +`rust-src` and resolved separately from the user's dependencies. Neither will +`std`, `alloc` or `core` be updated, as these are unversioned and always match +the current toolchain version. + +[`cargo vendor`][cargo-vendor] will not vendor the standard library crates or +their dependencies. These are pre-vendored as part of the `rust-src` component +([?][rationale-vendoring]). + +The following commands will now build the standard library if required as part +of the compilation of the project, just like any other dependency: + +- [`cargo bench`][cargo-bench] +- [`cargo build`][cargo-build] +- [`cargo check`][cargo-check] +- [`cargo clippy`][cargo-clippy] +- [`cargo doc`][cargo-doc] +- [`cargo fix`][cargo-fix] +- [`cargo run`][cargo-run] +- [`cargo rustc`][cargo-rustc] +- [`cargo rustdoc`][cargo-rustdoc] +- [`cargo test`][cargo-test] + +This part of the RFC has no implications for the following Cargo subcommands: + +- [`cargo add`][cargo-add] +- [`cargo remove`][cargo-remove] +- [`cargo fmt`][cargo-fmt] +- [`cargo generate-lockfile`][cargo-generate-lockfile] +- [`cargo help`][cargo-help] +- [`cargo info`][cargo-info] +- [`cargo init`][cargo-init] +- [`cargo install`][cargo-install] +- [`cargo locate-project`][cargo-locate-project] +- [`cargo login`][cargo-login] +- [`cargo logout`][cargo-logout] +- [`cargo metadata`][cargo-metadata] +- [`cargo new`][cargo-new] +- [`cargo owner`][cargo-owner] +- [`cargo package`][cargo-package] +- [`cargo pkgid`][cargo-pkgid] +- [`cargo publish`][cargo-publish] +- [`cargo search`][cargo-search] +- [`cargo tree`][cargo-tree] +- [`cargo uninstall`][cargo-uninstall] +- [`cargo version`][cargo-version] +- [`cargo yank`][cargo-yank] + +## Stability guarantees +[stability-guarantees]: #stability-guarantees + +build-std enables a much greater array of configurations of the standard library +to exist and be produced by stable toolchains than the single configuration that +is distributed today. + +It is not feasible for the Rust project to test every combination of profile +configuration, Cargo feature, target and standard library crate. As such, the +stability of build-std as a mechanism must be separated from the stability +guarantees which apply to configurations of the standard library it enables. + +For example, while a stable build-std mechanism may permit the standard library +to be built for a tier three target, the Rust project continues to make no +commitments or guarantees that the standard library for that target will +function correctly or build at all. Even on a tier one target, the Rust project +cannot test every possible variation of the standard library that build-std +enables. + +The tier of a target no longer determines the possibility of using the standard +library, but rather the level of support provided for the standard library on +the target. + +Cargo and Rust project documentation will clearly document the configurations +which are tested upstream and are guaranteed to work. Any other configurations +are supported on a strictly best-effort basis. The Rust project may later choose +to provide more guarantees for some well-tested configurations (e.g. enabling +sanitisers). This documentation need not go into detail about the exact +compilation flags used in a configuration - for example, "the release profile +with the address sanitizer is tested to work" would be sufficient. + +There are also no guarantees about the exact configuration of the standard +library. Over time, the standard library built by build-std could be changed to +be closer to that of the pre-built standard library. + +Additionally, there are no guarantees that the build environment required for +the standard library will not change over time (e.g. new minimum versions of +system packages or C toolchains, etc). + +Building the standard library crates in the sysroot without requiring +`RUSTC_BOOTSTRAP` is intended for enabling the standard library to be built with +a stable toolchain and stable compiler flags, despite that the standard library +uses unstable features in its source code, not as a general mechanism for +bypassing Rust's stability mechanisms. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This section aims to justify all of the decisions made in the proposed design +from [*Proposal*][proposal] and discuss why alternatives were not chosen. + +## Why put `build-std` in the Cargo config? +[rationale-build-std-in-config]: #why-put-build-std-in-the-cargo-config + +There are various alternatives to putting `build-std` in the Cargo configuration: + +1. Cargo could continue to use an explicit command-line flag to enable + build-std, such as the current `-Zbuild-std` (stabilised as `--build-std`). + + This approach is proven to work, as per the current unstable implementation, + but has a poor user experience, requiring an extra argument to every + invocation of Cargo with almost every subcommand of Cargo. + + However, this approach does not lend itself to use with other future and + current Cargo features. Additional flags would be required to enable Cargo + features (like today's `-Zbuild-std-features`) and would still necessarily be + less fine-grained than being able to enable features on individual standard + library crates. Similarly for public/private dependencies or customising the + profile for the standard library crates. + +2. build-std could be enabled or disabled in the `Cargo.toml`. However, under + which conditions the standard library is rebuilt is better determined by the + user of Cargo, rather than the package being built. + + A user may want to never rebuild the standard library so as to avoid + invalidating the guarantees of their qualified toolchain, or may want to + rebuild unconditionally to further optimise the standard library for their + known deployment platform, or may only want to rebuild as necessary to ensure + the build will succeed. All of these rationale can apply to the same crate in + different circumstances, so it doesn't make sense for a crate to decide this + once in its `Cargo.toml`. + + It would be a waste of resources if a dependency declared that it must always + rebuild the standard library when the pre-built crate would be sufficient and + this could not be overridden. It is also unclear how to aggregate different + configurations of the `build-std` key from different crates in the dependency + graph into a single value. + +While using `build-std` key in the Cargo configuration shares some of the +downsides of using an explicit flag - not having a natural extension point for +other Cargo options exposed to dependencies - +[*Standard library dependencies*][deps] addresses these concerns. + +↩ [*Proposal*][proposal] + +## Why accept `never` as a value for `build-std`? +[rationale-build-std-never]: #why-accept-never-as-a-value-for-build-std + +The user can specify `never` (the default value) if they prefer which will never +rebuild the standard library. rustc will still return an error when the user's +target-modifiers do not match the pre-built standard library. + +The `never` value is useful particularly for qualified toolchains where +rebuilding the standard library may invalidate the testing that the qualified +toolchain has undergone. + +↩ [*Proposal*][proposal] + +## Why add `build-std` to the `[target.]` and `[target.]` sections? +[rationale-build-std-target-section]: #why-add-build-std-to-the-targettriple-and-targetcfg-sections + +Supporting `build-std` as a key of both `[build]` and `[target]` sections allows +the greatest flexibility for the user. The overhead of rebuilding the standard +library may not be desirable in general but would be required when building on +targets which do not ship a pre-built standard library. + +↩ [*Proposal*][proposal] + +## Why does `[target]` take precedence over `[build]` for `build-std`? +[rationale-build-std-precedence]: #why-does-target-take-precedence-over-build-for-build-std + +`[target]` configuration is necessarily more narrowly scoped so it makes sense +for it to override a global default in `[build]`. + +↩ [*Proposal*][proposal] + +## Why have a manual "always" option instead of a "when-needed" mode? +[rationale-unconditional]: #why-have-a-manual-always-option-instead-of-a-when-needed-mode + +Always using a locally-built standard library avoids the complexity associated +with an automatic build-std mechanism while still being useful for users of tier +three targets. By leaving an automatic mechanism for a later RFC, fewer of the +technical challenges of build-std need to be addressed all at once. + +Having an opt-in mechanism initially, such as `build-std = "always"`, allows for +early issues with build-std to be ironed out without potentially affecting more +users like an automatic mechanism. Later proposals will extend the `build-std` +option with an automatic mechanism. + +↩ [*Proposal*][proposal] + +## Why does "always" rebuild in release profile? +[rationale-release-profile]: #why-does-always-rebuild-in-release-profile + +The release profile most closely matches the existing pre-built standard +library, which has proven itself suitable for a majority of use cases. + +By minimising the differences between a newly-built std and a pre-built std, +there is less chance of the user experiencing bugs or unexpected behaviour from +the well-tested and supported pre-built std. Later proposals will extend the +`build-std` option with customised standard library builds that use the user's +profile. + +↩ [*Proposal*][proposal] + +## Why add `build-std-crates`? +[rationale-build-std-crate]: #why-add-build-std-crates + +Not all standard library crates will build on all targets. In a `no_std` project +for a tier three target, `build-std-crates` gives the user the ability to limit +which crates are built to those they know they need and will build successfully. + +*See [Standard library dependencies*][deps] for an alternative to +`build-std-crates`.* + +↩ [*Proposal*][proposal] + +## Why use the lockfile of the `rust-src` component? +[rationale-lockfile]: #why-use-the-lockfile-of-the-rust-src-component + +Using different dependency versions for the standard library would invalidate +the upstream testing of the standard library. In particular, some crates use +unstable APIs when included as a dependency of the standard library meaning that +there is a high risk of build breakage if any package version is changed. + +Using the lockfile included in the `rust-src` component guarantees that the same +dependency versions are used as in the pre-built standard library. As the +standard library does not re-export types from its dependencies, this will not +affect interoperability with the same dependencies of different versions used by +the user's crate. + +Using the lockfile does prevent Cargo from resolving the standard library +dependencies to newer patch versions that may contain security fixes. However, +this is already impossible with the pre-built standard library. + +See +[*Why vendor the standard library's dependencies?*][rationale-vendoring] + +↩ [*Proposal*][proposal] + +## Why not build the standard library in incremental? +[rationale-incremental]: #why-not-build-the-standard-library-in-incremental + +The standard library sources are not intended to be modified locally, similarly +to those Cargo fetches from `registry` or `git` sources. Incremental compilation +would only add a compilation time overhead for any package sources which do not +change. + +↩ [*Proposal*][proposal] + +## Why not produce a `dylib` for the standard library? +[rationale-no-dylib]: #why-not-produce-a-dylib-for-the-standard-library + +The standard library supports being built as both a `rlib` and a `dylib` and +both are shipped as part of the `rust-std` component. As the `dylib` does not +contain a metadata hash, it can be rebuilt unnecessarily when toolchain versions +change (e.g. switching between stable and nightly and back). The `dylib` is only +linked against when `-Cprefer-dynamic` is used. build-std will initially be +conservative and not include the `dylib` and `-Cprefer-dynamic` would fail +compilation. + +*See the following sections for future possibilities:* + +- [*Build both `dylib` and `rlib` variants of the standard library*][future-crate-type] + +↩ [*Proposal*][proposal] + +## Why use the pre-built standard library for procedural macros and build scripts in cross-compile mode? +[rationale-host-deps-cross]: #why-use-the-pre-built-standard-library-for-procedural-macros-and-build-scripts-in-cross-compile-mode + +Procedural macros always run on the host and need to be built with a +configuration that are compatible with the host toolchain's Cargo and rustc, +limiting the potential customisations of the standard library that would be +valid. There is little advantage to using a custom standard library with +procedural macros, as they are not part of the final output artifact and +anywhere they can run already have a toolchain with host tools and a pre-built +standard library. + +Build scripts similarly always run on the host and thus would require building +the standard library again for the host. There is little advantage to doing this +as build scripts are not part of the final output artifact. Build scripts do not +respect `RUSTFLAGS` which could result in target modifier mismatches if +rebuilding the standard library does respect `RUSTFLAGS`. + +↩ [*Proposal*][proposal] + +## Why use the pre-built standard library for procedural macros and build scripts in host mode? +[rationale-host-deps-host]: #why-use-the-pre-built-standard-library-for-procedural-macros-and-build-scripts-in-host-mode + +Unlike when in cross-compile mode, if Cargo is in host mode (i.e. `--target` is +not provided), the standard library built by build-std could hypothetically be +used for procedural macros and build scripts without additional recompilations +of the standard library. + +However, as with [cross-compile mode][rationale-host-deps-cross], there is +little advantage to using a customised standard library for procedural macros or +build scripts, and both would require limitations on the customisations possible +with build-std in order to guarantee compatibility with the compiler or build +script, respectively. + +↩ [*Proposal*][proposal] + +## Should target specifications own knowledge of which standard library crates are supported? +[rationale-target-spec-purpose]: #should-target-specifications-own-knowledge-of-which-standard-library-crates-are-supported + +It is much simpler to record this information in a target's specification than +build this information into Cargo or to try and match on the target's cfg values +in the standard library's `build.rs` and set a cfg that Cargo could read. + +Target specifications have typically been considered part of the compiler and +there has been hesitation to have target specs be the source of truth for +information like standard library support, as this is the domain of the library +team and ought to be owned by the standard library (such as in the standard +library's `build.rs`). However, with appropriate processes and sync points, +there is no reason why the target specification could not be primarily +maintained by the compiler team but in close coordination with library and other +relevant teams. + +↩ [*Standard library crate stability*][standard-library-crate-stability] + +## Why introduce `standard_library_support`? +[rationale-why-standard-library-support]: #why-introduce-standard_library_support + +Attempting to compile the standard library crates may fail for some targets +depending on which standard library crates that target intends to support. When +enabled, build-std should default to only building those crates that are +expected to succeed, and should prevent the user from attempting to build those +crates that are expected to fail. This will provide a much improved user +experience than attempting to build standard library crates and encountering +complex and unexpected compilation failures. + +For example, `no_std` targets often do not support `std` and so should inform +the error with a helpful error message that `std` cannot be built for the target +rather than attempt to build it and fail with confusing and unexpected errors. +Similarly, many `no_std` targets do support `alloc` if a global allocator is +provided, but if build-std built `alloc` by default for these targets then it +would often be unnecessary and could often fail. + +It is not sufficient to determine which crates should be supported for a target +based on its the tier. For example, targets like `aarch64-apple-tvos` are tier +three while intending to fully support the standard library. It would be +needlessly limiting to prevent build-std from building `std` for this target. +However, build-std does provide a stable mechanism to build `std` for this +target that did not previously exist, so there must be clarity about what +guarantees and level of support is provided by the Rust project: + +1. Whether a standard library crate is part of the stable interface of + the standard library as a whole is determined by the library team and the set + of crates that comprise this interface is the same for all targets + +2. Whether any given standard library crate can be built with build-std is + determined on a per-target basis depending on whether it is intended that the + target be able to support that crate + +3. Whether the Rust project provide guarantees or support for the standard + library on a target is determined by the tier of the target + +4. Whether the pre-built standard library is distributed for a target is + determined by the tier of the target and which crates it intends to support + +5. Which crate is built by default by build-std is determined on a per-target + basis + +For example, consider the following targets: + +- `armv7a-none-eabihf` + + 1. As with any other target, the `std`, `alloc` and `core` crates are stable + interfaces to the standard library + + 2. It intends to support the `core` and `alloc` crates, which build-std will + permit to be built. `std` cannot be built by build-std for this target (on + stable) + + 3. It is a tier three target, so no support or guarantees are provided for the + standard library crates + + 4. It is a tier three target, so no standard library crates are distributed + + 5. `alloc` would not build without a global allocator crate being provided by + the user and may not be required by all users, so only `core` will be built + by default + +- `aarch64-apple-tvos` + + 1. As with any other target, the `std`, `alloc` and `core` crates are stable + interfaces to the standard library + + 2. It intends to support `core`, `alloc` and `std` crates, which build-std + will permit to be built + + 3. It is a tier three target, so no support or guarantees are provided for the + standard library crates + + 4. It is a tier three target, so no standard library crates are distributed + + 5. All of `core`, `alloc` and `std` will be built by default + +- `armv7a-none-eabi` + + 1. As with any other target, the `std`, `alloc` and `core` crates are stable + interfaces to the standard library + + 2. It intends to support the `core` and `alloc` crates, which build-std will + permit to be built. `std` cannot be built by build-std for this target (on + stable) + + 3. It is a tier two target, so the project guarantees that the `core` and + `alloc` crates will build + + 4. It is a tier two target, so there are distributed artefacts for the `core` + and `alloc` crates + + 5. `alloc` would not build without a global allocator crate being provided by + the user and may not be required by all users, so only `core` will be built + by default + +- `aarch64-unknown-linux-gnu` + + 1. As with any other target, the `std`, `alloc` and `core` crates are stable + interfaces to the standard library + + 2. It intends to support the `core`, `alloc` and `std` crates, which build-std + will permit to be built + + 3. It is a tier one target, so the project guarantees that the `core`, `alloc` + and `std` will build and that they have been tested + + 4. It is a tier one target, so there are distributed artefacts for the `core`, + `alloc` and `std` crates + + 5. All of `core`, `alloc` and `std` will be built by default + +↩ [*Standard library crate stability*][standard-library-crate-stability] + +## Why remove `restricted_std`? +[rationale-remove-restricted-std]: #why-remove-restricted_std + +`restricted_std` was originally added as part of a mechanism to enable the +standard library to build on all targets (just with stubbed out functionality), +however stability is not an ideal match for this use case. rustc will still try +to compile unstable code, so this doesn't help ensure the standard library builds +on all targets. + +Furthermore, when `restricted_std` applies, users must add +`#![feature(restricted_std)]` to opt-in to using the standard library anyway +(conditionally, only for affected targets), and have no mechanism for opting-in +on behalf of their dependencies (including first-party crates like `libtest`). + +It is still valuable for the standard library to be able to compile on as many +targets as possible using the `unsupported` module in its platform abstraction +layer, but this mechanism does not use `restricted_std`. + +↩ [*`restricted_std`*][restricted_std] + +## Why disallow custom targets? +[rationale-disallow-custom-targets]: #why-disallow-custom-targets + +While custom targets can be used on stable today, in practice, they are only +used on nightly as `-Zbuild-std` would need to be used to build at least `core`. +As such, if build-std were to be stabilised, custom targets would become much +more usable on stable toolchains. This is undesirable as there are many open +questions surrounding the [unstable target-spec-json][rust#71009] for custom +targets and how they ought to be supported. + +In order to avoid users relying on the unstable format with a stable toolchain, +using custom targets with build-std on a stable toolchain is disallowed by Cargo +until another RFC can consider all the implications of this thoroughly. + +Similarly, custom targets are destabilised in rustc, as the changes in +[*Building the standard library on a stable toolchain*][building-the-standard-library-on-a-stable-toolchain] +could allow the unstable format to be relied upon even with Cargo's prohibition +of custom targets. + +↩ [*Custom targets*][custom-targets] + +## Why prevent rustc from loading root dependencies from the sysroot? +[rationale-root-sysroot-deps]: #why-prevent-rustc-from-loading-root-dependencies-from-the-sysroot + +Loading root dependencies from the sysroot could be a source of bugs. + +For example, if a crate has an explicit dependency on `core` which is newly +built, then there will be no `alloc` or `std` builds present. A user could still +write `extern crate alloc` and accidentally load `alloc` from the sysroot +(compiled with the default profile settings) and consequently `core` from the +sysroot, conflicting with the newly build `core`. `extern crate alloc` should +only be able to load the `alloc` crate if the crate depends on it in its +`Cargo.toml`. A similar circumstance can occur with dependencies like +`panic_unwind` that the compiler tries to load itself. + +Dependencies of packages can still be loaded from the sysroot, even with +`--no-implicit-sysroot-deps`, to support the circumstance where Cargo uses a +pre-built standard library crate (e.g. +`$sysroot/lib/rustlib/$target/lib/std.rlib`) and needs to load the dependencies +of that crate which are also in the sysroot. + +`--no-implicit-sysroot-deps` is a flag rather than default behaviour to preserve +rustc's usability when invoked outside of Cargo. For example, by compiler +developers when working on rustc. + +`--sysroot=''` is an existing mechanism for disabling the sysroot - this is not +used as it remains desirable to load dependencies from the sysroot as a +fallback. In addition, rustc uses the sysroot path to find `rust-lld` and +similar tools and would not be able to do so if the sysroot were disabled by +providing an empty path. + +↩ [*Preventing implicit sysroot dependencies*][preventing-implicit-sysroot-dependencies] + +## Why use `noprelude` with `--extern`? +[rationale-noprelude-with-extern]: #why-use-noprelude-with---extern + +Using `noprelude` allows `build-std` to closer match rustc's behaviour when it +loads crates from the sysroot. Without `noprelude`, rustc adds crates provided +with `--extern` flags to the extern prelude. As a consequence, if a newly-built +`alloc` were passed using `--extern alloc=alloc.rlib` then `extern crate alloc` +would not be required to use the locally-built `alloc`, but it would be to use +the pre-built `alloc` when `--extern alloc=alloc.rlib` is not provided. This +difference in how a crate is made available to rustc should not be observable to +the user as they have not opted into the migration. + +Passing crates without `noprelude` with the existing prelude behaviour has also +been a source of [bugs][wg-cargo-std-aware#40] in previous `-Zbuild-std` +implementations. + +↩ [*Preventing implicit sysroot dependencies*][preventing-implicit-sysroot-dependencies] + +## Why not allow the source path for the standard library be customised? +[rationale-custom-src-path]: #why-not-allow-the-source-path-for-the-standard-library-be-customised + +It is not a goal of this proposal to enable or improve the usability of custom +or modified standard libraries. + +↩ [*Vendored `rust-src`*][vendored-rust-src] + +## Why vendor the standard library's dependencies? +[rationale-vendoring]: #why-vendor-the-standard-librarys-dependencies + +Vendoring the standard library is possible since it currently has its own +workspace, allowing the dependencies of just the standard library crates (and +not the compiler or associated tools in `rust-lang/rust`) to be easily packaged. +Doing so has multiple advantages.. + +- Avoid needing to support standard library dependencies in `cargo vendor` +- Avoid needing to support standard library dependencies in `cargo fetch` +- Re-building the standard library does not require an internet connection +- Standard library dependency versions are fixed to those in the `Cargo.lock` + anyway, so initial builds with `build-std` start quicker with these + dependencies already available +- Allow build-std to continue functioning if a `crates.io` dependency is + "yanked" + - This leaves the consequences of a toolchain version using yanked + dependencies the same as without this RFC + +..and few disadvantages: + +- A larger `rust-src` component takes up more disk space and takes longer to + download + - If using build-std, these dependencies would have to be downloaded at build + time, so this is only an issue if build-std is not used and `rust-src` is + downloaded. + - `rustc-src` is currently 3.5 MiB archived and 44 MiB extracted, and if + dependencies of the standard library were vendored, then it would be 9.1 MiB + archived and 131 MiB extracted. +- Vendored dependencies can't be updated with the latest security fixes + - This is no different than the pre-built standard library + +How this affects `crates.io`/`rustup` bandwidth usage or user time spent +downloading these crates is unclear and depends on user patterns. If not +vendored, Cargo will "lazily" download them the first time `build-std` is used +but this may happen multiple times if they are cleaned from its cache without +upgrading the toolchain version. + +See +[*Why use the lockfile of the `rust-src` component?*][rationale-lockfile] + +↩ [*Vendored `rust-src`*][vendored-rust-src] + +## Why not check if `rust-src` has been modified? +[rationale-src-modifications]: #why-not-check-if-rust-src-has-been-modified + +This is in line with other immutable dependency sources (like registry or git). +It is also likely that any protections implemented to check that the sources in +`rust-src` have not been modified could be trivially bypassed. + +Any crate that depends on `rust-src` having been modified would not be usable +when published to crates.io as the required modifications will obviously not be +included. + +↩ [*Vendored `rust-src`*][vendored-rust-src] + +## Why allow building from the sysroot with implied `RUSTC_BOOTSTRAP`? +[rationale-implied-bootstrap]: #why-allow-building-from-the-sysroot-with-implied-rustc_bootstrap + +Cargo needs to be able to build the standard library crates, which inherently +require unstable features. It could set `RUSTC_BOOTSTRAP` internally to do this +with a stable toolchain, but this is a bypass mechanism that the project do not +want to encourage use of, and as this is a shared requirement with other build +systems that wish to build an unmodified standard library and want to work on +stable toolchains, it is worth establishing a narrow general mechanism. + +For example, Rust's project goal to enable Rust for Linux to build using only a +stable toolchain would require that it be possible to build `core` without +nightly. + +It is not sufficient for rustc to special-case the `core`, `alloc` and `std` +crate names as, when being built as part of the standard library, dependencies +of the standard library also use unstable features and it is not practical to +special-case all of these crates. + +↩ [*Building the standard library on a stable toolchain*][building-the-standard-library-on-a-stable-toolchain] + +## Why invert the `mem` feature? +[rationale-no-mem]: #why-invert-the-mem-feature + +While "negative" features are typically discouraged due to how features unify +(e.g. `std` features are preferred to `no_std`): the `mem` feature's current +behaviour is the opposite of what is optimal. + +Ideally, a crate should be able to provide alternate memory symbols and disable +`compiler_builtins`' symbols for the entire crate graph by enabling a feature +(e.g. `std`/`libc` could do this) - this is what an `external-mem` feature +enables. + +↩ [*`compiler-builtins-mem`*][compiler-builtins-mem] + +## Why not use weak linkage for `compiler-builtins/mem` symbols? +[rationale-no-weak-linkage]: #why-not-use-weak-linkage-for-compiler-builtinsmem-symbols + +Since [compiler-builtins#411], the relevant symbols in `compiler_builtins` +already have weak linkage. However, it is nevertheless not possible to simply +remove the `mem` feature and have the symbols always be present: + +- Some targets, such as those based on MinGW, do not have sufficient support for + weak definitions (at least with the default linker). +- Weak linkage has precedence over shared libraries and the symbols of a + dynamically-linked `libc` should be preferred over `compiler_builtins`'s + symbols. + +↩ [*`compiler-builtins-mem`*][compiler-builtins-mem] + +## Why not globally cache builds of the standard library? +[rationale-caching]: #why-not-globally-cache-builds-of-the-standard-library + +The standard library is no different than regular dependencies in being able to +benefit from global caching of dependency builds. It is out-of-scope of this +proposal to propose a special-cased mechanism for this that applies only to the +standard library. [cargo#5931] tracks the feature request of intermediate +artifact caching in Cargo. + +↩ [*Caching*][caching] + +## Why not link to hosted standard library documentation in generated docs? +[rationale-generated-docs]: #why-not-link-to-hosted-standard-library-documentation-in-generated-docs + +Cargo would need to pass `-Zcrate-attr="doc(html_root_url=..)"` to the standard +library crates when building them but doesn't have the required information to +know what url to provide. Cargo would require knowledge of the current toolchain +channel to build the correct url and doesn't know this. + +↩ [*Generated documentation*][generated-documentation] + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +The following are aspects of the proposal which warrant further discussion or +small details are likely to be bikeshed prior to this part of the RFC's +acceptance or stabilisation and aren't pertinent to the overall design: + +## What should the `build-std` configuration in `.cargo/config` be named? +[unresolved-config-name]: #what-should-the-build-std-configuration-in-cargoconfig-be-named + +What should this configuration option be named? `build-std`? +`rebuild-standard-library`? + +↩ [*Proposal*][proposal] + +## What should the "always" and "never" values of `build-std` be named? +[unresolved-config-values]: #what-should-the-always-and-never-values-of-build-std-be-named + +What is the most intuitive name for the values of the `build-std` setting? +`always`? `manual`? `unconditional`? + +`always` combined with the configuration option being named `build-std` - +`build-std = "always"` - is imperfect as it reads as if the standard library +will be re-built every time, when it actually just avoids use of the pre-built +standard library and caches the newly-built standard library. + +↩ [*Proposal*][proposal] + +## What should `build-std-crates` be named? +[unresolved-build-std-crate-name]: #what-should-build-std-crates-be-named + +What should this configuration option be named? + +↩ [*Proposal*][proposal] + +## Should the standard library inherit RUSTFLAGS? +[unresolved-inherit-rustflags]: #should-the-standard-library-inherit-rustflags + +Existing designs for *[Opaque dependencies]* intended that `RUSTFLAGS` would not +apply to the opaque dependency. However, if a target modifier were set using +`RUSTFLAGS` and build-std ignored the variable, then rustc would fail to build +the user's project due to incompatible target modifiers. This would necessitate +that every stable target modifier be exposed via Cargo to be usable in practice. + +↩ [*Proposal*][proposal] + +## Should `rust-src` be a default component? +[unresolved-rust-src]: #should-rust-src-be-a-default-component + +Ensuring `rust-src` is a default component reduces friction for users, and CI, +who have to otherwise need to install the component manually the first time they +use `build-std`. + +On the other hand this increases their storage and bandwidth costs, plus +bandwidth costs for the project. The impact on usability is limited for the user +to once per toolchain as the component persists through updates. + +↩ [*Vendored rust-src*][vendored-rust-src] + +# Future possibilities +[future-possibilities]: #future-possibilities + +There are many possible follow-ups to this part of the RFC: + +## Allow reusing sysroot artifacts if available +[future-reuse-sysroot]: #allow-reusing-sysroot-artifacts-if-available + +This part of the RFC proposes rebuilding all required crates unconditionally as +this fits Cargo's existing compilation model better. However, just building a +crate equivalent to one already in the sysroot is inefficient. Cargo could learn +when to reuse artifacts in the sysroot when equivalent to ones it intends to +build, but this is complex enough to warrant its own proposal if desired. + +↩ [*Proposal*][proposal] + +## Replace `#![no_std]` as the source-of-truth for whether a crate depends on `std` +[future-replace-no_std]: #replace-no_std-as-the-source-of-truth-for-whether-a-crate-depends-on-std + +Crates can currently use the crate attribute `#![no_std]` to indicate a lack of +dependency on `std`. Introducing `build-std-crates` or explicit dependencies (as +in [*Standard library dependencies*][deps]) would add a second way for the user +to indicate a lack of dependency on the standard library. It could therefore be +desirable to deprecate `#![no_std]` so that there remains only a single way to +express a dependency on the standard library. + +`#![no_std]` serves two purposes - it stops the compiler from adding `std` to +the extern prelude and it prevents the user from depending on anything from +`std` accidentally. rustc's default behaviour of loading `std` when not +explicitly provided the crate via an `--extern` flag should be preserved for +backwards-compatibility with existing direct invocations of rustc. + +`#![no_std]` could instead become a compiler flag which would indicate to the +compiler that `std` should not be loaded by default and that `core`'s prelude +should be used instead. Cargo would use this flag when driving rustc, providing +explicit paths to the newly-built or pre-built standard library crates, just as +with any other dependency. + +In addition, `#![no_std]` could be migrated to denying a lint which would +prevent use of items from `std`. + +↩ [*Interactions with `#![no_std]`*][interactions-with-no_std] + +## Allow custom targets with build-std +[future-custom-targets]: #allow-custom-targets-with-build-std + +This would require a decision from the relevant teams on the exact stability +guarantees of the target-spec-json format and whether any large changes to +the format are desirable prior to broader use. + +↩ [*Custom targets*][custom-targets] + +## Avoid building `panic_unwind` unnecessarily +[future-panic_unwind]: #avoid-building-panic_unwind-unnecessarily + +This would require adding a `--print default-unwind-strategy` flag to rustc and +using that to avoid building `panic_unwind` if the default is abort for any +given target and `panic` is not set in the profile. + +↩ [*Panic strategies*][panic-strategies] + +## Enable local recompilation of special object files/sanitizer runtimes +[future-recompile-special]: #enable-local-recompilation-of-special-object-filessanitizer-runtimes + +These files are shipped pre-compiled for relevant targets and are not compiled +locally. If a user wishes to customise the compilation of these files like the +standard library, then there is no mechanism to do so. + +↩ [*Self-contained objects*][self-contained-objects] + +## Allow building `profiler-builtins` +[future-profiler-builtins]: #allow-building-profiler-builtins + +It may be possible to ship a rustup component with pre-compiled native +dependencies of `profiler-builtins` so that build-std can reliably compile the +`profiler-builtins` crate regardless of the environment. Alternatively, +stability guarantees could be adjusted to set expectations that some parts of +the standard library may not build without external system dependencies. + +If `profiler-builtins` can be reliably built, then it should be unconditionally +included in part of the standard library build. + +↩ [*profiler-builtins*][profiler-builtins] + +## Build both `dylib` and `rlib` variants of the standard library +[future-crate-type]: #build-both-dylib-and-rlib-variants-of-the-standard-library + +build-std could build both the `dylib` and `rlib` of the standard library. + +↩ [*Why not produce a `dylib` for the standard library?*][rationale-no-dylib] + +[deps]: ./5-standard-library-dependencies.md +[future-compiler-builtins-c]: ./5-standard-library-dependencies.md#allow-local-builds-of-compiler-rt-intrinsics + +[Opaque dependencies]: https://hackmd.io/@epage/ByGfPtRell + +[cargo#5931]: https://github.com/rust-lang/cargo/issues/5931 +[compiler-builtins#411]: https://github.com/rust-lang/compiler-builtins/pull/411 +[compiler-team#343]: https://github.com/rust-lang/compiler-team/issues/343 +[rust#76158]: https://github.com/rust-lang/rust/pull/76158 +[rust#71009]: https://github.com/rust-lang/rust/pull/71009 +[rust#135395]: https://github.com/rust-lang/rust/pull/135395 + +[std-build.rs]: https://github.com/rust-lang/rust/blob/f315e6145802e091ff9fceab6db627a4b4ec2b86/library/std/build.rs#L17 + +[cargo-add]: https://doc.rust-lang.org/cargo/commands/cargo-add.html +[cargo-bench]: https://doc.rust-lang.org/cargo/commands/cargo-bench.html +[cargo-build]: https://doc.rust-lang.org/cargo/commands/cargo-build.html +[cargo-check]: https://doc.rust-lang.org/cargo/commands/cargo-check.html +[cargo-clean]: https://doc.rust-lang.org/cargo/commands/cargo-clean.html +[cargo-clippy]: https://doc.rust-lang.org/cargo/commands/cargo-clippy.html +[cargo-doc]: https://doc.rust-lang.org/cargo/commands/cargo-doc.html +[cargo-fetch]: https://doc.rust-lang.org/cargo/commands/cargo-fetch.html +[cargo-fix]: https://doc.rust-lang.org/cargo/commands/cargo-fix.html +[cargo-fmt]: https://doc.rust-lang.org/cargo/commands/cargo-fmt.html +[cargo-generate-lockfile]: https://doc.rust-lang.org/cargo/commands/cargo-generate-lockfile.html +[cargo-help]: https://doc.rust-lang.org/cargo/commands/cargo-help.html +[cargo-info]: https://doc.rust-lang.org/cargo/commands/cargo-info.html +[cargo-init]: https://doc.rust-lang.org/cargo/commands/cargo-init.html +[cargo-install]: https://doc.rust-lang.org/cargo/commands/cargo-install.html +[cargo-locate-project]: https://doc.rust-lang.org/cargo/commands/cargo-locate-project.html +[cargo-login]: https://doc.rust-lang.org/cargo/commands/cargo-login.html +[cargo-logout]: https://doc.rust-lang.org/cargo/commands/cargo-login.html +[cargo-metadata]: https://doc.rust-lang.org/cargo/commands/cargo-metadata.html +[cargo-miri]: https://doc.rust-lang.org/cargo/commands/cargo-miri.html +[cargo-new]: https://doc.rust-lang.org/cargo/commands/cargo-new.html +[cargo-owner]: https://doc.rust-lang.org/cargo/commands/cargo-owner.html +[cargo-package]: https://doc.rust-lang.org/cargo/commands/cargo-package.html +[cargo-pkgid]: https://doc.rust-lang.org/cargo/commands/cargo-pkgid.html +[cargo-publish]: https://doc.rust-lang.org/cargo/commands/cargo-publish.html +[cargo-remove]: https://doc.rust-lang.org/cargo/commands/cargo-remove.html +[cargo-report]: https://doc.rust-lang.org/cargo/commands/cargo-report.html +[cargo-run]: https://doc.rust-lang.org/cargo/commands/cargo-run.html +[cargo-rustc]: https://doc.rust-lang.org/cargo/commands/cargo-rustc.html +[cargo-rustdoc]: https://doc.rust-lang.org/cargo/commands/cargo-rustdoc.html +[cargo-search]: https://doc.rust-lang.org/cargo/commands/cargo-search.html +[cargo-test]: https://doc.rust-lang.org/cargo/commands/cargo-test.html +[cargo-tree]: https://doc.rust-lang.org/cargo/commands/cargo-tree.html +[cargo-uninstall]: https://doc.rust-lang.org/cargo/commands/cargo-uninstall.html +[cargo-update]: https://doc.rust-lang.org/cargo/commands/cargo-update.html +[cargo-vendor]: https://doc.rust-lang.org/cargo/commands/cargo-vendor.html +[cargo-version]: https://doc.rust-lang.org/cargo/commands/cargo-version.html +[cargo-yank]: https://doc.rust-lang.org/cargo/commands/cargo-yank.html +[wg-cargo-std-aware#40]: https://github.com/rust-lang/wg-cargo-std-aware/issues/40 \ No newline at end of file diff --git a/text/0000-build-std/5-standard-library-dependencies.md b/text/0000-build-std/5-standard-library-dependencies.md new file mode 100644 index 00000000000..f8d4099061d --- /dev/null +++ b/text/0000-build-std/5-standard-library-dependencies.md @@ -0,0 +1,1057 @@ +# Explicit dependencies + +This part of the RFC proposes supporting explicit dependencies on the standard +library crates in `Cargo.toml`. This enables Cargo to determine which standard +library crates are required by the crate graph without `build-std-crates` being +set and for different crates to require different standard library crates. + +While not directly necessary for our core listed motivations, this allows future +extensions which support public/private standard library dependencies or +enabling features of the standard library. Allowing the standard library to +behave similarly to other dependencies reduces user friction and can improve +build times. + +# Proposal +[proposal]: #proposal + +Users can now optionally declare explicit dependencies on the standard library +in their `Cargo.toml` files ([?][rationale-why-explicit-deps]): + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true } +``` + +`builtin` is a new source of dependency, like registry dependencies (with the +`version` key and optionally the `registry` key), `path` dependencies or `git` +dependencies. `builtin` can only be set to `true` and cannot be combined with +any other dependency source for a given dependency +([?][rationale-builtin-other-sources]). + +`builtin` can only be used with crates named `core`, `alloc` or `std` +([?][rationale-no-builtin-other-crates]) on stable. This set could be expanded +with new crates in future. + +Use with any other crate name is gated on a perma-unstable `cargo-feature` +([?][rationale-unstable-builtin-crates]). If a builtin dependency on a unstable +crate name exists but is not used due to cfgs, then Cargo will still require the +Cargo feature. + +> [!NOTE] +> +> Explicit dependencies are passed to rustc without the `noprelude` modifier +> ([?][rationale-explicit-noprelude]). When adding an explicit dependency, users +> may need to adjust their code (removing extraneous `extern crate` statements +> or root-relative paths, like `::std`). + +Crates without an explicit dependency on the standard library now have a +implicit dependency ([?][rationale-no-migration]) on `std`, `alloc` and `core`. +Any explicit `builtin` dependency present in any dependency table will disable +the implicit dependencies. + +> [!NOTE] +> +> Implicit dependencies are passed to rustc with the `noprelude` modifier to +> ensure backwards compatibility as in [`build-std=always`][always-noprelude]. + +When a `std` dependency is present an additional implicit dependency on the +`test` crate is added for crates that are being tested with the default test +harness. The `test` crate's name, but not its interface, will be stabilised so +Cargo can refer to it. + +crates.io will accept crates published which have `builtin` dependencies. + +Standard library dependencies can be marked as `optional` and be enabled +conditionally by a feature in the crate: + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true, optional = true } +core = { builtin = true } + +[features] +default = ["std"] +std = ["dep:std"] +``` + +If there is an optional dependency on the standard library then Cargo will +validate that there is at least one non-optional dependency on the standard +library (e.g. an optional `std` and non-optional `core` or `alloc`, or an +optional `alloc` and non-optional `core`). `core` cannot be optional. For +example, the following example will error as it could result in a build without +`core` (if the `std` feature were disabled): + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true, optional = true } +# error: must have a non-optional dependency on core + +[features] +default = ["std"] +std = ["dep:std"] +``` + +However, in this example, a build for the `x86-64-pc-windows-gnu` target would +have an explicit dependency on `alloc` (and indirectly on `core`), while a build +for any other target would have implicit dependencies on `std`, `alloc` and +`core`: + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +# implicit deps on `core`, `alloc` and `std` unless target='x86_64-pc-windows-gnu' + +[target.x86_64-pc-windows-gnu.dependencies] +alloc.builtin = true +``` + +Dependencies with `builtin = true` cannot be renamed with the `package` key +([?][rationale-package-key]). It is not possible to perform source replacement +on the `builtin` source using the `[source]` Cargo config table +([?][rationale-source-replacement]), and nor is it possible to override +`builtin` dependencies with the `[replace]` sections or `paths` overrides +([?][rationale-overriding-builtins]), though [patching][patches] is permitted. + +Dependencies with `builtin = true` can be specified as platform-specific +dependencies: + +```toml +[target.'cfg(unix)'.dependencies] +std = { builtin = true} +``` + +Implicit and explicit standard library dependencies are added to `Cargo.lock` +files ([?][rationale-cargo-lock]). + +> [!NOTE] +> +> A new version of the `Cargo.lock` file will be introduced to add support for +> packages with a `builtin` source: +> +> ```toml +> [[package]] +> name = "std" +> version = "0.0.0" +> source = "builtin" +> ``` +> +> The package version of `std`, `alloc` and `core` will be fixed at `0.0.0`. The +> optional lockfile fields `dependencies` and `checksum` will not be present for +> `builtin` dependencies. + +*See the following sections for rationale/alternatives:* + +- [*Why explicitly declare dependencies on the standard library in `Cargo.toml`?*][rationale-why-explicit-deps] +- [*Why disallow builtin dependencies to be combined with other sources?*][rationale-builtin-other-sources] +- [*Why disallow builtin dependencies on other crates?*][rationale-no-builtin-other-crates] +- [*Why unstably allow all names for `builtin` crates?*][rationale-unstable-builtin-crates] +- [*Why not use `noprelude` for explicit `builtin` dependencies?*][rationale-explicit-noprelude] +- [*Why not require builtin dependencies instead of supporting implicit ones?*][rationale-no-migration] +- [*Why disallow renaming standard library dependencies?*][rationale-package-key] +- [*Why disallow source replacement on `builtin` packages?*][rationale-source-replacement] +- [*Why add standard library dependencies to Cargo.lock?*][rationale-cargo-lock] + +*See the following sections for relevant unresolved questions:* + +- [*What syntax is used to identify dependencies on the standard library in `Cargo.toml`?*][unresolved-dep-syntax] +- [*What is the format for builtin dependencies in `Cargo.lock`?*][unresolved-lockfile] + +*See the following sections for future possibilities:* + +- [*Allow unstable crate names to be referenced behind cfgs without requiring nightly*][future-cfg-unstable-crate-name] +- [*Allow `builtin` source replacement*][future-source-replacement] +- [*Remove `rustc_dep_of_std`*][future-rustc_dep_of_std] + +## Non-`builtin` standard library dependencies +[non-builtin-standard-library-dependencies]: #non-builtin-standard-library-dependencies + +Cargo already supports `path` and `git` dependencies for crates named `core`, +`alloc` and `std` which continue to be supported and work: + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { path = "../my_std" } # already supported by Cargo +``` + +A `core`/`alloc`/`std` dependency with a `path`/`git` source can be combined +with `builtin` dependencies: + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { path = "../my_std" } +core = { builtin = true } +``` + +Crates with these dependency sources will remain unable to be published to +crates.io. + +## Patches +[patches]: #patches + +Under a perma-unstable feature it is permitted to patch standard library +dependencies with `path` and `git` sources (or any other source) +([?][rationale-patching]): + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true } + +[patch.builtin] # permitted on nightly +std = { .. } + +[patch.builtin] # permitted on nightly +std = { path = "../libstd" } +``` + +As with dependencies, crates with `path`/`git` patches for `core`, `alloc` or +`std` are not accepted by crates.io. + +*See the following sections for rationale/alternatives:* + +- [*Why unstably permit patching of standard library dependencies?*][rationale-patching] + +*See the following sections for relevant unresolved questions:* + +- [*What syntax is used to patch dependencies on the standard library in `Cargo.toml`?*][unresolved-patch-syntax] + +## Features +[features]: #features + +On a stable toolchain, it is not permitted to enable or disable features of +explicit standard library dependencies ([?][rationale-features]), as in the +below example: + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true, features = [ "foo" ] } # not permitted +# ..or.. +std = { builtin = true, default-features = false } # not permitted +``` + +*See the following sections for rationale/alternatives:* + +- [*Why limit enabling standard library features to an unstable feature?*][rationale-features] + +*See the following sections for future possibilities:* + +- [*Allow enabling/disabling features with build-std*][future-features] + +## Public and private dependencies +[public-and-private-dependencies]: #public-and-private-dependencies + +Implicit and explicit dependencies on the standard library default to being +public dependencies ([?][rationale-default-public]). + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +``` + +..is equivalent to the following explicit dependency on `std`: + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true, public = true } +``` + +*See the following sections for relevant unresolved questions:* + +- [*Should standard library dependencies default to public?*][unresolved-std-default-public] + +*See the following sections for rationale/alternatives:* + +- [*Why default to public for standard library dependencies?*][rationale-default-public] + +## `dev-dependencies` and `build-dependencies` +[dev-dependencies-and-build-dependencies]: #dev-dependencies-and-build-dependencies + +Explicit dependencies on the standard library can be specified in +`dev-dependencies` in the same way as regular `dependencies`. Any explicit +`builtin` dependency present in `dev-dependencies` table will disable the +implicit dependencies. It is possible for `dev-dependencies` to have additional +`builtin` dependencies that the `dependencies` section does not have (e.g. +requiring `std` when the regular dependencies only require `core`). + +Build scripts and proc macros continue to use the prebuilt standard library as +in [`build-std=always`][always], and so explicit dependencies on the standard +library are not supported in `build-dependencies`. + +*See the following sections for relevant unresolved questions:* + +- [*Should we support `build-dependencies`?*][unresolved-build-deps] + +## Registries +[registries]: #registries + +Standard library dependencies will be present in the registry index +([?][rationale-cargo-index]). A `builtin_deps` key is added to the +[index's JSON schema][cargo-json-schema] ([?][rationale-cargo-builtindeps]). +`builtin_deps` is similar to the existing `deps` key and contains a list of JSON +objects, each representing a dependency that is "builtin" to the Rust toolchain +and cannot otherwise be found in the registry. The +["publish" endpoint][cargo-registry-web-publish] of the Registry Web API will +similarly be updated to support `builtin_deps`. + +> [!NOTE] +> +> It is expected that the keys of these objects will be: +> +> - `name` +> - String containing name of the `builtin` package. Can shadow the names of +> other packages in the registry (except those packages in the `deps` key +> of the current package) ([?][rationale-cargo-index-shadowing]) +> +> - `features`: +> - An array of strings containing enabled features in order to support +> changing the standard library features on nightly. Optional, empty by +> default. +> +> - `optional`, `default_features`, `target`, `kind`: +> - These keys have the same definition as in the `deps` key +> +> The keys `req`, `registry` and `package` from `deps` are not required per the +> limitations on builtin dependencies. +> +> The `builtin_deps` key is optional and if not present its default value will +> be the implicit builtin dependencies: +> +> ```json +> "builtin_deps" : [ +> { +> "name": "std", +> "features": [], +> "optional": false, +> "default_features": true, +> "target": null, +> "kind": "normal", +> }, +> { +> "name": "alloc", +> ... # as above +> }, +> { +> "name": "core", +> ... # as above +> } +> ] +> ``` +> +> When producing a registry index entry for a package Cargo will not serialise +> any `builtin` dependencies it inferred. This allows the set of inferred +> packages to change in the future if needed. Similarly, the published +> `Cargo.toml` will not explicitly declare any inferred dependencies. + +*See the following sections for rationale/alternatives:* + +- [*Why add standard library crates to Cargo's index?*][rationale-cargo-index] +- [*Why add a new key to Cargo's registry index JSON schema?*][rationale-cargo-builtindeps] +- [*Why can `builtin_deps` shadow other packages in the registry?*][rationale-cargo-index-shadowing] + +## Cargo subcommands +[cargo-subcommands]: #cargo-subcommands + +Any Cargo command which accepts a package spec with `-p` will now additionally +recognise `core`, `alloc` and `std` and none of their dependencies. Many of +Cargo's subcommands will need modification to support build-std: + +[`cargo add`][cargo-add]'s heuristics will include adding `std`, `alloc` or +`core` as builtin dependencies if these crate names are provided. `cargo add` +will additionally have a `--builtin` flag to allow for adding crates with a +`builtin` source explicitly: + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true } # <-- this would be added +``` + +If attempting to add a crate name outside of `core`, `alloc` or `std` this will +fail unless the required `cargo-feature` is added to allow other `builtin` crate +names as described in [the rationale][rationale-unstable-builtin-crates]. + +If attempting to add a `builtin` crate with features then this will fail unless +the required `cargo-feature` is enabled as described in [*Features*][features]. + +[`cargo info`][cargo-info] will learn how to print information for the built-in +`std`, `alloc` and `core` dependencies: + +```shell-session +$ cargo info std +std +rust standard library +license: Apache 2.0 + MIT +rust-version: 1.86.0 +documentation: https://doc.rust-lang.org/1.86.0/std/index.html +``` + +```shell-session +$ cargo info alloc +alloc +rust standard library +license: Apache 2.0 + MIT +rust-version: 1.86.0 +documentation: https://doc.rust-lang.org/1.86.0/alloc/index.html +``` + +```shell-session +$ cargo info core +core +rust standard library +license: Apache 2.0 + MIT +rust-version: 1.86.0 +documentation: https://doc.rust-lang.org/1.86.0/core/index.html +``` + +[`cargo metadata`][cargo-metadata] will emit `std`, `alloc` and `core` +dependencies to the metadata emitted by `cargo metadata` (when those crates are +explicit dependencies). None of the standard library's dependencies will be +included. `source` would be set to `builtin` and the remaining fields would be +set like any other dependency. + +> [!NOTE] +> +> `cargo metadata` output could look as follows: +> +> ```json +> { +> "packages": [ +> { +> /* ... */ +> "dependencies": [ +> { +> "name": "std", +> "source": "builtin", +> "req": "*", +> "kind": null, +> "rename": null, +> "optional": false, +> "uses_default_features": true, +> "features": ["compiler-builtins-mem"], +> "target": null, +> "public": true +> } +> ], +> /* ... */ +> } +> ] +> } +> ``` + +[`cargo pkgid`][cargo-pkgid] when passed `-p core` would print +`builtin://.#core` as the source, likewise with `alloc` and `std`. This format +complies with [Cargo's spec for Package IDs][cargo-pkgid-spec]. + +[`cargo remove`][cargo-remove] will remove `core`, `alloc` or `std` explicitly +from the manifest if invoked with those crate names (using the same heuristics +as those described above for `cargo add`): + +```toml +[package] +name = "hello_world" +version = "0.1.0" +edition = "2024" + +[dependencies] +std = { builtin = true } # <-- this would be removed +``` + +[`cargo tree`][cargo-tree] will show `std`, `alloc` and `core` at appropriate +places in the tree of dependencies. As opaque dependencies, none of the other +dependencies of `std`, `alloc` or `core` will be shown. Neither `std`, `alloc` +or `core` will have a version number. + +> [!NOTE] +> +> `cargo tree` output could look as follows: +> +> ```shell-session +> $ cargo tree +> myproject v0.1.0 (/myproject) +> ├── rand v0.7.3 +> │ ├── getrandom v0.1.14 +> │ │ ├── cfg-if v0.1.10 +> │ │ │ └── core v0.0.0 +> │ │ ├── libc v0.2.68 +> │ │ │ └── core v0.0.0 +> │ │ └── core v0.0.0 +> │ ├── libc v0.2.68 (*) +> │ │ └── core v0.0.0 +> │ ├── rand_chacha v0.2.2 +> │ │ ├── ppv-lite86 v0.2.6 +> │ │ │ └── core v0.0.0 +> │ │ ├── rand_core v0.5.1 +> │ │ │ ├── getrandom v0.1.14 (*) +> │ │ │ └── core v0.0.0 +> │ │ └── std v0.0.0 +> │ │ └── alloc v0.0.0 +> │ │ └── core v0.0.0 +> │ ├── rand_core v0.5.1 (*) +> │ └── std v0.0.0 (*) +> └── std v0.0.0 (*) +> ``` + +This part of the RFC has no implications for the following Cargo subcommands: + +- [`cargo bench`][cargo-bench] +- [`cargo build`][cargo-build] +- [`cargo check`][cargo-check] +- [`cargo clean`][cargo-clean] +- [`cargo clippy`][cargo-clippy] +- [`cargo doc`][cargo-doc] +- [`cargo fetch`][cargo-fetch] +- [`cargo fix`][cargo-fix] +- [`cargo fmt`][cargo-fmt] +- [`cargo generate-lockfile`][cargo-generate-lockfile] +- [`cargo help`][cargo-help] +- [`cargo init`][cargo-init] +- [`cargo install`][cargo-install] +- [`cargo locate-project`][cargo-locate-project] +- [`cargo login`][cargo-login] +- [`cargo logout`][cargo-logout] +- [`cargo miri`][cargo-miri] +- [`cargo new`][cargo-new] +- [`cargo owner`][cargo-owner] +- [`cargo package`][cargo-package] +- [`cargo publish`][cargo-publish] +- [`cargo report`][cargo-report] +- [`cargo run`][cargo-run] +- [`cargo rustc`][cargo-rustc] +- [`cargo rustdoc`][cargo-rustdoc] +- [`cargo search`][cargo-search] +- [`cargo test`][cargo-test] +- [`cargo uninstall`][cargo-uninstall] +- [`cargo update`][cargo-update] +- [`cargo vendor`][cargo-vendor] +- [`cargo version`][cargo-version] +- [`cargo yank`][cargo-yank] + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This section aims to justify all of the decisions made in the proposed design +from [*Proposal*][proposal] and discuss why alternatives were not chosen. + +## Why explicitly declare dependencies on the standard library in `Cargo.toml`? +[rationale-why-explicit-deps]: #why-explicitly-declare-dependencies-on-the-standard-library-in-cargotoml + +If there are no explicit dependencies on standard library crates, Cargo would +need to be able to determine which standard library crates to build when this is +required: + +- Cargo could unconditionally build `std`, `alloc` and `core`. Not only would + this be unnecessary and wasteful for `no_std` crates in the embedded + ecosystem, but sometimes a target may not support building `std` at all and + this would cause the build to fail. + +- rustc could support a `--print` value that would print whether the crate + declares itself as `#![no_std]` crate, and based on this, Cargo could build + `std` or only `core`. This would require asking rustc to parse crates' + sources while resolving dependencies, slowing build times. Alternatively, + Cargo can already read Rust source to detect frontmatter (for `cargo script`) + so it could additionally look for `#![no_std]` itself. Regardless of how it + determines a crate is no-std, Cargo would also need to know whether to build + `alloc` too, which checking for `#![no_std]` does not help with. Cargo could + go further and ask rustc whether a crate (or its dependencies) used `alloc`, + but this seems needlessly complicated. + +- Cargo could allow the user to specify which crates are required to be built, + such as with the existing options to the `-Zbuild-std=` flag. + [`build-std=always`][always] proposes a `build-std-crates` flag to enable + explicit dependencies to be a separate part of this RFC. + +Furthermore, supporting explicit dependencies on standard library crates enables +use of other Cargo features that apply to dependencies in a natural and +intuitive way. If there were not explicit standard library dependencies and +enabling features on the `std` crate was desirable, then a mechanism other than +the standard syntax for this would be necessary, such as a flag (e.g. +`-Zbuild-std-features`) or option in Cargo's configuration. This also applies to +optional dependencies, public/private features, etc. + +Users already use Cargo features to toggle `#![no_std]` in crates which support +building without the standard library. When dependencies on the standard library +are exposed in `Cargo.toml` then they can be made optional and enabled by the +existing Cargo features that crates already have. + +↩ [*Proposal*][proposal] + +## Why disallow builtin dependencies to be combined with other sources? +[rationale-builtin-other-sources]: #why-disallow-builtin-dependencies-to-be-combined-with-other-sources + +If using `path`/`git` sources with `builtin` dependencies worked in the same way +as using `path`/`git` sources with `version` sources, then: crates with +`path`/`git` standard library dependencies could be pushed to crates.io. + +This is not desirable as it is unclear that supporting `path`/`git` sources +which shadow standard library crates was a deliberate choice and so enabling +that pattern to be used more widely when not necessary is needlessly permissive. + +In addition, when combined with a `git`/`path` source, the `version` constraint +also applies to package from the `git`/`path` source. If `version` were used +alongside `builtin`, then this behaviour would be a poor fit as.. + +- ..the `std`, `alloc` and `core` crates all currently have a version of `0.0.0` + +- ..choosing different version requirements for different `builtin` crates is + confusing when a single version of these crates is provided by the toolchain + +Hypothetically, choosing a different version for `builtin` crates could be a way +of supporting per-target/per-profile MSRVs, but this has limited utility. + +↩ [*Proposal*][proposal] + +## Why disallow builtin dependencies on other crates? +[rationale-no-builtin-other-crates]: #why-disallow-builtin-dependencies-on-other-crates + +`builtin` dependencies could be accepted on two other crates - dependencies of +the standard library, like `compiler_builtins`, or other crates in the sysroot +added manually by users, however: + +- The standard library's dependencies are not part of the stable interface of + the standard library and it is not desirable that users can observe their + existence or depend on them directly + +- Other crates in the sysroot added by users are not something that can + reasonably be supported by build-std and these crates should become regular + dependencies + +↩ [*Proposal*][proposal] + +## Why unstably allow all names for `builtin` crates? +[rationale-unstable-builtin-crates]: #why-unstably-allow-all-names-for-builtin-crates + +For any crate shipped with the standard library in the sysroot, the user can +already write an `extern crate` declaration to use it. Most are marked unstable +either explicitly or implicitly with the use of `-Zforce-unstable-if-unmarked` +so this does not allow items from these crates to be used on stable. + +For example, some users write benchmarks using `libtest` and have written +`extern crate test` without the `#[cfg(test)]` attribute to load the crate. +There may be other niche uses of unstable sysroot crates that this enables to +continue on nightly toolchains. + +An allowlist of `builtin` crate names isn't used here to avoid Cargo needing to +hardcode the names of many crates in the sysroot which are inherently unstable. + +↩ [*Proposal*][proposal] + +## Why not use `noprelude` for explicit `builtin` dependencies? +[rationale-explicit-noprelude]: #why-not-use-noprelude-for-explicit-builtin-dependencies + +Explicit builtin dependencies without the `noprelude` modifier behave more +consistently with other dependencies specified in the Cargo manifest. + +This is a trade-off, trading consistency of user experience with special-casing +in Cargo. Cargo would have to handle implicit vs explicit dependencies +differently. An explicit dependency on the standard library will behave +similarly to other dependencies in their manifest, but the behaviour will be +subtly different than with implicit builtin dependencies (where `extern crate` +is required). + +↩ [*Proposal*][proposal] + +## Why not require builtin dependencies instead of supporting implicit ones? +[rationale-no-migration]: #why-not-require-builtin-dependencies-instead-of-supporting-implicit-ones + +Requiring explicit `builtin` dependencies over an edition would increase the +boilerplate required for users of Cargo and make the minimal `Cargo.toml` file +larger. + +Supporting implicit dependencies allows the majority of the Rust ecosystem from +having to make any changes - `no_std` crates (or crates with a `std` feature) +will still benefit from adding explicit dependencies as allow them to be easily +used with `no_std` targets but users can still work around any legacy crates in +the graph with [`build-std-crates`][always]. + +↩ [*Proposal*][proposal] + +## Why disallow renaming standard library dependencies? +[rationale-package-key]: #why-disallow-renaming-standard-library-dependencies + +Cargo allows [renaming dependencies][cargo-docs-renaming] with the `package` +key, which allows user code to refer to dependencies by names which do not +match their `package` name in their respective `Cargo.toml` files. + +However, rustc expects the standard library crates to be present with their +existing names - for example, `core` is always added to the +[extern prelude][rust-extern-prelude]. + +Alternatively, a mechanism could be added to rustc so that it could be informed +of the user's names for `builtin` crates. + +↩ [*Proposal*][proposal] + +## Why disallow source replacement on `builtin` packages? +[rationale-source-replacement]: #why-disallow-source-replacement-on-builtin-packages + +Modifying the source code of the standard library in the `rust-src` component is +not supported. Source replacement of the `builtin` source could be a way to +support this in future but this is out-of-scope for this proposal. + +See [*Allow `builtin` source replacement*][future-source-replacement]. + +↩ [*Proposal*][proposal] + +## Why not permit overriding dependencies with `replace` or `paths`? +[rationale-overriding-builtins]: #why-not-permit-overriding-dependencies-with-replace-or-paths + +Similarly to [source replacement][rationale-source-replacement], easing +modification of the standard library sources is out-of-scope for this proposal. + +↩ [*Proposal*][proposal] + +## Why add standard library dependencies to `Cargo.lock`? +[rationale-cargo-lock]: #why-add-standard-library-dependencies-to-cargolock + +`Cargo.lock` is a direct serialisation of a resolve and that must be a two-way +non-lossy process in order to make the `Cargo.lock` useful without doing further +resolution to fill in missing `builtin` packages. + +↩ [*Proposal*][proposal] + +## Why unstably permit patching of the standard library dependencies? +[rationale-patching]: #why-unstably-permit-patching-of-the-standard-library-dependencies + +Being able to patch `builtin = true` dependencies and replace their source with +a `path` dependency is required to be able to replace `rustc_dep_of_std`. As +crates which use these sources cannot be published to crates.io, this would not +enable a usable general-purpose mechanism for crates to modify the standard +library sources. This capability is restricted to nightly toolchains as that is +all that is required for it to be used in replacing `rustc_dep_of_std`. + +↩ [*Patches*][patches] + +## Why limit enabling standard library features to an unstable feature? +[rationale-features]: #why-limit-enabling-standard-library-features-to-an-unstable-feature + +If it were possible to enable features of the standard library crates on stable +then all of the standard library's current features would immediately be held to +the same stability guarantees as the rest of the standard library, which is not +desirable. See +[*Allow enabling/disabling features with build-std*][future-features] + +↩ [*Features*][features] + +## Why default to public for standard library dependencies? +[rationale-default-public]: #why-default-to-public-for-standard-library-dependencies + +There are crates building on stable which re-export from the standard library. +If the standard library dependencies were not public then these crates would +start to trigger the `exported_private_dependencies` lint when upgrading to a +version of Cargo with a standard library dependency. + +↩ [*Public and private dependencies*][public-and-private-dependencies] + +## Why follow the default privacy of explicit standard library dependencies? +[rationale-explicit-private]: #why-follow-the-default-privacy-of-explicit-standard-library-dependencies + +This may be unintuitive when a user first writes an explicit standard library +dependency, triggering the `exported_private_dependency` lint, but this would be +caught immediately by the user. However, it is also unintuitive that the default +for privacy of a explicitly written dependency would depend on which crate the +dependency was (i.e. the standard library has a different default than +everything else). + +↩ [*Public and private dependencies*][public-and-private-dependencies] + +## Why add standard library crates to Cargo's index? +[rationale-cargo-index]: #why-add-standard-library-crates-to-cargos-index + +When Cargo builds the dependency graph, it is driven by the index (not +`Cargo.toml`), so builtin dependencies need to be included in the index. + +↩ [*Registries*][registries] + +## Why add a new key to Cargo's registry index JSON schema? +[rationale-cargo-builtindeps]: #why-add-a-new-key-to-cargos-registry-index-json-schema + +Cargo's [registry index schema][cargo-json-schema] is versioned and making a +behaviour-of-Cargo-modifying change to the existing `deps` keys would be a +breaking change. Each package is published under one particular version of the +schema, meaning that older versions of Cargo cannot use newer versions of +packages which are defined using a schema it does not have knowledge of. + +Cargo ignores packages published under an unsupported schema version, so older +versions of Cargo cannot use newer versions of packages relying on these +features (though this would be true because of an incompatible Cargo manifest +anyway). New schema versions are disruptive to users on older toolchains, as the +resolver will act as if a package does not exist. Recent Cargo versions have +improved error reporting for this circumstance. + +Some new fields, including `rust-version`, were added to all versions of the +schema. Cargo ignores fields it does not have knowledge of, so older versions of +Cargo will simply not use `rust-version` and its presence does not change their +behaviour. + +Existing versions of Cargo already function correctly without knowledge of +crate's standard library dependencies. A new top-level key will be ignored by +older versions of Cargo, while newer versions will understand it. This is a +different approach to that taken when artifact dependencies were added to the +schema, as those do not have a suitable representation in older versions of +Cargo. + +The obvious alternative to a `builtin_deps` key is to modify `deps` entries with +a new `builtin: bool` field and to increment the version of the schema. However, +these entries would not be understood by older versions of Cargo which would +look in the registry to find these packages and fail to do so. + +That approach could be made to work if dummy packages for `core`/`alloc`/`std` +were added to registries. Older versions of Cargo would pass these to rustc +via `--extern` and shadow the real standard library dependencies in the sysroot, +so these packages would need to contain `extern crate std; pub use std::*;` (and +similar for `alloc`/`core`) to try and load the pre-built libraries from the +sysroot (this is the same approach as packages like [embed-rs][embed-rs-source] +take today, using `path` dependencies for the standard library to shadow it). + +↩ [*Registries*][registries] + +## Why can `builtin_deps` shadow other packages in the registry? +[rationale-cargo-index-shadowing]: #why-can-builtin_deps-shadow-other-packages-in-the-registry + +While `crates.io` forbids certain crate names including `std`, `alloc` and +`core`, third party registries may allow them without a warning. The schema +needs a way to refer to packages with the same name either in the registry or +builtin, which `builtin_deps` allows. + +`builtin_deps` names are not allowed to shadow names of packages in `deps` as +these would conflict when passed to rustc via `--extern`. + +↩ [*Registries*][registries] + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +The following small details are likely to be bikeshed prior to this part of the +RFC's acceptance or stabilisation and aren't pertinent to the overall design: + +## What syntax is used to identify dependencies on the standard library in `Cargo.toml`? +[unresolved-dep-syntax]: #what-syntax-is-used-to-identify-dependencies-on-the-standard-library-in-cargotoml + +What syntax should be used for the explicit standard library dependencies? +`builtin = true`? `sysroot = true`? + +↩ [*Proposal*][proposal] + +## What is the format for builtin dependencies in `Cargo.lock`? +[unresolved-lockfile]: #what-is-the-format-for-builtin-dependencies-in-cargolock + +How should `builtin` deps be represented in lockfiles? Is `builtin = true` +appropriate? Could the `source` field be reused with the string "builtin" or +should it stay only as a URL+scheme? + +↩ [*Proposal*][proposal] + +## What syntax is used to patch dependencies on the standard library in `Cargo.toml`? +[unresolved-patch-syntax]: #what-syntax-is-used-to-patch-dependencies-on-the-standard-library-in-cargotoml + +`[patch.builtin]` is the natural syntax given `builtin` is a new source, but may +be needlessly different to existing packages. + +↩ [*Patches*][patches] + +## Should standard library dependencies default to public? +[unresolved-std-default-public]: #should-standard-library-dependencies-default-to-public + +Standard library dependencies defaulting to public is a trade-off between +special-casing in Cargo and requiring that any user with a dependency on the +standard library who re-exports from the standard library manually declare their +dependency as public. + +It is also inconsistent with +[*Why not use `noprelude` for explicit `builtin` dependencies?*][rationale-explicit-noprelude] +which aims to make builtin dependencies consistent with other dependencies in +the manifest. + +↩ [*Public and private dependencies*][public-and-private-dependencies] + +## Should we support `build-dependencies`? +[unresolved-build-deps]: #should-we-support-build-dependencies + +Allowing `builtin` dependencies to be used in `dependencies` and +`dev-dependencies` but not in `build-dependencies` is an inconsistency. + +However, supporting `builtin` dependencies in `build-dependencies` would permit +no-std build scripts. It is unclear whether supporting no-std build scripts +would be desirable. + +↩ [*`dev-dependencies` and `build-dependencies`*][dev-dependencies-and-build-dependencies] + +# Future possibilities +[future-possibilities]: #future-possibilities + +There are many possible follow-ups to this part of the RFC: + +## Allow unstable crate names to be referenced behind cfgs without requiring nightly +[future-cfg-unstable-crate-name]: #allow-unstable-crate-names-to-be-referenced-behind-cfgs-without-requiring-nightly + +It is possible to allow builtin dependencies on unstable crate names to exist +behind cfgs and for the crate to be compiled on a stable toolchain as long as +the cfgs are not active. This is a trade-off - it adds a large constraint on +when Cargo can validate the set of crate names, but would enable users to avoid +using nightly or doing MSRV bumps. + +↩ [*Proposal*][proposal] + +## Allow `builtin` source replacement +[future-source-replacement]: #allow-builtin-source-replacement + +This involves allowing the user to blanket-override the standard library sources +with a `[source.builtin]` section of the Cargo configuration. + +As [rationale-source-replacement] details it is unclear if users need to do this +or if it's even something the Rust project wishes to support. + +↩ [*Proposal*][proposal] + +## Remove `rustc_dep_of_std` +[future-rustc_dep_of_std]: #remove-rustc_dep_of_std + +With first-class explicit dependencies on the standard library, +`rustc_dep_of_std` is rendered unnecessary and explicit dependencies on the +standard library can always be present in the `Cargo.toml` of the standard +library's dependencies. + +The `core`, `alloc` and `std` dependencies can be patched in the standard +library's workspace to point to the local copy of the crates. This avoids +`crates.io` dependencies needing to add support for `rustc_dep_of_std` before +the standard library can depend on them. + +↩ [*Proposal*][proposal] + +## Allow enabling/disabling features with build-std +[future-features]: #allow-enablingdisabling-features-with-build-std + +This would require the library team be comfortable with the features declared on +the standard library being part of the stable interface of the standard library. + +The behaviour of disabling default features has been highlighted as a potential +cause of breaking changes. + +Alternatively, this could be enabled alongside another proposal which would +allow the standard library to define some features as stable and others as +unstable. + +As there are some features that Cargo will set itself when appropriate (e.g. to +enable or disable [panic runtimes][panic-strategies] or +[`compiler-builtins/mem`][compiler-builtins-mem]), Cargo may need to always +prevent some otherwise stable features from being toggled as it controls those. + +↩ [*Features*][features] + +## Allow local builds of `compiler-rt` intrinsics +[future-compiler-builtins-c]: #allow-local-builds-of-compiler-rt-intrinsics + +The [`c` feature][background-dependencies] of `compiler_builtins` (which is also +exposed by `core`, `alloc` and `std` through `compiler-builtins-c`) causes its +`build.rs` file to build and link in more optimised C versions of intrinsics. + +It will not be enabled by default because it is possible that the target +platform does not have a suitable C compiler available. The user being able to +enable this manually will be enabled through work on features (see +[*Allow enabling/disabling features with build-std*][future-features]). Once the +user can enable `compiler-builtins/c`, they will need to manually configure +`CFLAGS` to ensure that the C components will link with Rust code. + +[always]: ./4-build-std-always.md + +[background-dependencies]: ./1-background.md#dependencies +[cargo-docs-renaming]: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml +[cargo-json-schema]: https://doc.rust-lang.org/cargo/reference/registry-index.html#json-schema +[cargo-registry-web-publish]: https://doc.rust-lang.org/cargo/reference/registry-web-api.html#publish +[cargo-pkgid-spec]: https://doc.rust-lang.org/cargo/reference/pkgid-spec.html +[embed-rs-source]: https://github.com/embed-rs/stm32f7-discovery/blob/e2bf713263791c028c2a897f2eb1830d7f09eceb/core/src/lib.rs#L7 +[rust-extern-prelude]: https://doc.rust-lang.org/reference/names/preludes.html#extern-prelude + +[panic-strategies]: ./4-build-std-always.md#panic-strategies +[compiler-builtins-mem]: ./4-build-std-always.md#compiler-builtinsmem +[always-noprelude]: ./4-build-std-always.md#why-use-noprelude-with---extern + +[cargo-add]: https://doc.rust-lang.org/cargo/commands/cargo-add.html +[cargo-bench]: https://doc.rust-lang.org/cargo/commands/cargo-bench.html +[cargo-build]: https://doc.rust-lang.org/cargo/commands/cargo-build.html +[cargo-check]: https://doc.rust-lang.org/cargo/commands/cargo-check.html +[cargo-clean]: https://doc.rust-lang.org/cargo/commands/cargo-clean.html +[cargo-clippy]: https://doc.rust-lang.org/cargo/commands/cargo-clippy.html +[cargo-doc]: https://doc.rust-lang.org/cargo/commands/cargo-doc.html +[cargo-fetch]: https://doc.rust-lang.org/cargo/commands/cargo-fetch.html +[cargo-fix]: https://doc.rust-lang.org/cargo/commands/cargo-fix.html +[cargo-fmt]: https://doc.rust-lang.org/cargo/commands/cargo-fmt.html +[cargo-generate-lockfile]: https://doc.rust-lang.org/cargo/commands/cargo-generate-lockfile.html +[cargo-help]: https://doc.rust-lang.org/cargo/commands/cargo-help.html +[cargo-info]: https://doc.rust-lang.org/cargo/commands/cargo-info.html +[cargo-init]: https://doc.rust-lang.org/cargo/commands/cargo-init.html +[cargo-install]: https://doc.rust-lang.org/cargo/commands/cargo-install.html +[cargo-locate-project]: https://doc.rust-lang.org/cargo/commands/cargo-locate-project.html +[cargo-login]: https://doc.rust-lang.org/cargo/commands/cargo-login.html +[cargo-logout]: https://doc.rust-lang.org/cargo/commands/cargo-login.html +[cargo-metadata]: https://doc.rust-lang.org/cargo/commands/cargo-metadata.html +[cargo-miri]: https://doc.rust-lang.org/cargo/commands/cargo-miri.html +[cargo-new]: https://doc.rust-lang.org/cargo/commands/cargo-new.html +[cargo-owner]: https://doc.rust-lang.org/cargo/commands/cargo-owner.html +[cargo-package]: https://doc.rust-lang.org/cargo/commands/cargo-package.html +[cargo-pkgid]: https://doc.rust-lang.org/cargo/commands/cargo-pkgid.html +[cargo-publish]: https://doc.rust-lang.org/cargo/commands/cargo-publish.html +[cargo-remove]: https://doc.rust-lang.org/cargo/commands/cargo-remove.html +[cargo-report]: https://doc.rust-lang.org/cargo/commands/cargo-report.html +[cargo-run]: https://doc.rust-lang.org/cargo/commands/cargo-run.html +[cargo-rustc]: https://doc.rust-lang.org/cargo/commands/cargo-rustc.html +[cargo-rustdoc]: https://doc.rust-lang.org/cargo/commands/cargo-rustdoc.html +[cargo-search]: https://doc.rust-lang.org/cargo/commands/cargo-search.html +[cargo-test]: https://doc.rust-lang.org/cargo/commands/cargo-test.html +[cargo-tree]: https://doc.rust-lang.org/cargo/commands/cargo-tree.html +[cargo-uninstall]: https://doc.rust-lang.org/cargo/commands/cargo-uninstall.html +[cargo-update]: https://doc.rust-lang.org/cargo/commands/cargo-update.html +[cargo-vendor]: https://doc.rust-lang.org/cargo/commands/cargo-vendor.html +[cargo-version]: https://doc.rust-lang.org/cargo/commands/cargo-version.html +[cargo-yank]: https://doc.rust-lang.org/cargo/commands/cargo-yank.html \ No newline at end of file diff --git a/text/0000-build-std/6-appendix-summary-of-changes.md b/text/0000-build-std/6-appendix-summary-of-changes.md new file mode 100644 index 00000000000..395f16cc023 --- /dev/null +++ b/text/0000-build-std/6-appendix-summary-of-changes.md @@ -0,0 +1,97 @@ +# Appendix I: Summary of changes +[appendix-i]: #appendix-i-summary-of-changes + +There are many features proposed in this RFC for different parts of the project: + +- Bootstrap/infra/release + - [Vendoring standard library sources into `rust-src`](./4-build-std-always.md#vendored-rust-src) (`build-std=always`) + - [`rust-src` is a default component](./4-build-std-always.md#vendored-rust-src) (`build-std=always`) + - [`rust-self-contained` components](./4-build-std-always.md#self-contained-objects) (`build-std=always`) + - [Testing build-std in rust-lang/rust CI][constraints-on-the-standard-library] +- Cargo + - [`build-std = "always"`](./4-build-std-always.md) (`build-std=always`) + - [Extending Cargo subcommmands](./4-build-std-always.md#cargo-subcommands) + - [Prohibiting custom targets](./4-build-std-always.md#custom-targets) (`build-std=always`) + - [Explicit dependencies](./5-standard-library-dependencies.md#proposal) (*Standard library dependencies*) + - [Lockfile changes](./5-standard-library-dependencies.md#proposal) + - [Registry changes](./5-standard-library-dependencies.md#registries) + - [Extending Cargo subcommmands](./5-standard-library-dependencies.md#cargo-subcommands) +- Compiler + - [Loading `panic_unwind` from `-L dependency=`](./4-build-std-always.md#proposal) (`build-std=always`) + - [`--no-implicit-sysroot-deps`](./4-build-std-always.md#preventing-implicit-sysroot-dependencies) (`build-std=always`) + - [Destabilise custom targets](./4-build-std-always.md#custom-targets) (`build-std=always`) + - [Assuming `RUSTC_BOOTSTRAP` for sysroot builds](./4-build-std-always.md#building-the-standard-library-on-a-stable-toolchain) (`build-std=always`) + - [Detect missing `rust-self-contained` components and provide diagnostics](./4-build-std-always.md#self-contained-objects) (`build-std=always`) + - [Forcing many codegen-units for `compiler-builtins`](./4-build-std-always.md#compiler-builtins) (`build-std=always`) +- Project-wide + - [Documenting build-std stability guarantees](./4-build-std-always.md#stability-guarantees) (`build-std=always`) +- Standard library + - [Removing `restricted_std`](./4-build-std-always.md#restricted_std) (`build-std=always`) + - [Moving configuration into the standard library's profile](./4-build-std-always.md) (`build-std=always`) + +## Constraints on the standard library, compiler and bootstrap +[constraints-on-the-standard-library]: #constraints-on-the-standard-library-compiler-and-bootstrap + +A stable mechanism for building the standard library imposes some constraints on +the rest of the toolchain that would need to be upheld: + +- No further required customisation of the pre-built standard library through + any means other than the profile in `Cargo.toml` +- Avoid mandatory C dependencies on the standard library + - At the very least, new dependencies on the standard library will impact + whether the standard library can be successfully built by users with varying + environments and this impact will need to be considered going forward + - New C dependencies will need to be careful not to cause symbol conflicts + with user crates that pull in the same dependency (e.g. using + [`links =...`][links]) + - If this did come up, it might be possible to work around it with + postprocessing that renames C symbols used by the standard library but + that would be better avoided +- The standard library continues to exist in its own workspace, with its own + lockfile +- The name of the `test` crate becomes stable (but not its interface) +- The `panic-unwind` and `compiler-builtins-mem` `sysroot` features become + stable so Cargo can refer to them + - This should not necessitate a "stable/unstable features" mechanism, rather a + guarantee from the library team that they're happy for these to stay +- Dependencies of the standard library cannot use build probes to detect whether nightly features can be used + - With + [*Assuming `RUSTC_BOOTSTRAP` for sysroot builds*](./4-build-std-always.md#building-the-standard-library-on-a-stable-toolchain), + these build probes would always assume the crate is being built on nightly + +> [!NOTE] +> +> Cargo will likely be made a [JOSH] subtree of the [rust-lang/rust] so that all +> relevant parts of the toolchain can be updated in tandem when this is +> necessary. + +## Unresolved questions +[unresolved-questions]: #unresolved-questions + +The following are all of the unresolved questions from all stages of the RFC: + +- [*What should the `build-std` configuration in `.cargo/config` be named?*][unresolved-config-name] +- [*What should the "always" and "never" values of `build-std` be named?*][unresolved-config-values] +- [*What should `build-std-crates` be named?*][unresolved-build-std-crate-name] + +## Future possibilities +[future-possibilities]: #future-possibilities + +The following are all of the future possibilities from all stages of the RFC: + +- [*Allow custom targets with build-std*][future-custom-targets] +- [*Avoid building `panic_unwind` unnecessarily*][future-panic_unwind] +- [*Enable local recompilation of special object files/sanitizer runtimes*][future-recompile-special] +- [*Allow choosing the crate type of the standard library?*][future-crate-type] + +[future-crate-type]: ./4-build-std-always.md#build-both-dylib-and-rlib-variants-of-the-standard-library +[future-custom-targets]: ./4-build-std-always.md#allow-custom-targets-with-build-std +[future-panic_unwind]: ./4-build-std-always.md#avoid-building-panic_unwind-unnecessarily +[future-recompile-special]: ./4-build-std-always.md#enable-local-recompilation-of-special-object-filessanitizer-runtimes +[unresolved-build-std-crate-name]: ./4-build-std-always.md#what-should-build-std-crates-be-named +[unresolved-config-name]: ./4-build-std-always.md#what-should-the-build-std-configuration-in-cargoconfig-be-named +[unresolved-config-values]: ./4-build-std-always.md#what-should-the-always-and-never-values-of-build-std-be-named + +[JOSH]: https://josh-project.github.io/josh/intro.html +[rust-lang/rust]: https://github.com/rust-lang/rust +[links]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-links-field \ No newline at end of file diff --git a/text/0000-build-std/7-appendix-literature-review.md b/text/0000-build-std/7-appendix-literature-review.md new file mode 100644 index 00000000000..33215663228 --- /dev/null +++ b/text/0000-build-std/7-appendix-literature-review.md @@ -0,0 +1,1614 @@ +# Appendix II: Exhaustive literature review +[appendix-ii]: #appendix-ii-exhaustive-literature-review + +This section will attempt to summarize every issue, pull request, RFC and +discussion related to the design and implementation of build-std since its +conception in May 2015. If anything has been omitted then that's just an +oversight and it can be added. The summaries may not reflect current up-to-date +information if those updates weren't in the discussion being summarized. +Up-to-date information should be present when these issues are referenced in the +previous sections. + +This section's level of detail is not strictly necessary to understand this RFC, +the summary at the start of the [History][history] section should be +sufficient, but this section should help if more detail is desired on some +referenced material. + +## [rfcs#1133] (2015) +[a1-rfcs-1133-2015]: #rfcs1133-2015 + +This section contains all of the sources related to [rfcs#1133]. + +- **[rfcs#1133]: Make Cargo aware of standard library dependencies**, + [Ericson2314], May 2015 + - This is the first RFC that proposed making Cargo aware of the standard + library + - It was motivated by.. + - ..better support for cross-compilation to targets that cannot have + pre-built std due to strange configuration requirements + - ..building std with different configurations (e.g. panic + strategies/features/etc) + - ..simplifying `rustbuild` + - The RFC proposes both that the standard library should be explicitly listed + as a dependency in `Cargo.toml` and that it should be rebuilt when necessary + - `std = { version = "1.10", stdlib = true }` is the proposed syntax for a + `std` dependency in `Cargo.toml` + - `version = "*"` is also acceptable. As today, a package specifying a + version for the standard library will be rejected by crates.io + - `stdlib = true` defines the source of the dependency (similarly to `git` + or `path`) + - As the compiler/language version is tied to the standard library + version, this is effectively declaring an MSRV + - This dependency would be implicitly added to all crates unless a + standard library dependency is explicitly written + - i.e. writing `core = ..` would prevent the default `std = ..` + dependency from being added implicitly + - An `implicit-dependencies` key would also be added and to determine + whether implicit standard library dependencies are added. This is to + be able to write the `Cargo.toml` of the `core` crate + - `no_std` was not stable at the time of this RFC being written, so + the RFC does not address the circumstance where an implicit + dependency may make an existing `no_std` crate fail to build for a + tier three target that does not support the standard library + - The RFC is written prior to the introduction of the `rust-src` component + and assumes that other implementations of the compiler may exist that will + have their own standard library implementations and that the feature must + support that + - A description of how the feature could be implemented in Cargo is + provided: + - Add implicit dependencies as early as possible so the majority of Cargo + requires fewer changes + - `stdlib = true` generates a "source id" mapping to either a compiler + source directory (e.g. `rust-src` today) or a "sysroot binary mock + source" + - A "sysroot binary mock source" is the source containing the pre-built + standard library and is found by searching the sysroot for the + standard library rlib + - It always builds the source if present, falling back to the sysroot + rlibs otherwise + - The RFC argues that as the configuration of the standard library to + build is not known, the implementation must be conservative (e.g. no + features are enabled) + - It is intended that rustc would never use the sysroot (by passing + `--sysroot=''`) + - The standard library would not be added to lockfiles, the implementation + argues that there would be no value in adding it + - The RFC aims to be forward-compatible with the standard library source + being replaced by users and that parts of the standard library could be on + crates.io + - The RFC also suggests adding a mechanism for this to be introduced to + Cargo as an unstable feature, as Cargo had no mechanism for this when the + RFC was written + - There was varied feedback over a period of three years, initially: + - The advantage of replacing the standard library was not clear, as + `std = { path = "..." }` is already possible with Cargo (and used by some + projects to wrap the standard library) + - This doesn't work if a user wants to replace `std` over the entire + dependency graph + - Declaring a standard library dependency explicitly in all crates was seen + as unfortunate by some as it was redundant in many cases + - [xargo] was mentioned for the first time in May 2016. + - [cargo#2768] was opened as an initial implementation of the RFC. + - After core/std was being built using bootstrap and Cargo, there was another + burst of activity on the RFC: + - Supporting `compiler-rt` was raised as a challenge, but it was argued that + this wasn't a blocker as they are not necessary for the very bare metal + use-case intended by the author + - Lots of possibilities that the RFC enabled were raised on the discussion + thread + - There was discussion about what would trigger a build of std versus re-use + of std from the sysroot: + - The RFC wasn't exceptionally clear on this, but it appears that the + intent was for the `Cargo.toml` to be the source-of-truth for what a + package needs, and that the environment/profile settings determine + whether a build needs to happen + - Cargo would pass `--sysroot-whitelist` with a list of crate names to + the compiler when it determines that rustc should be permitted to use + the pre-built artifacts from the sysroot + - It was repeatedly expressed by the author and advocates of the RFC that a + ideal final state would be that the sysroot as a concept not exist (at + least for loading dependencies) + - Contemporary project members discussed the RFC and shared their + conclusions: + - Sysroot crates being specified with a special syntax is reasonable + - Putting std on crates.io is a non-goal + - The sysroot exists and won't go away anytime soon + - A known location in the sysroot for the std sources is desirable/good + - The std facade should be a normal part of the crate DAG + - This was not true when the comment was written in July 2016 + - It should be possible to override features of the sysroot crates + - Features should be used to avoid C dependencies + - Sysroot replacement poses stability concerns so must be restricted to + nightly + - By August 2016, the participating project members seemed quite happy with + the RFC and shared some more feedback: + - There was interest in requiring semver versions for the standard library, + as a pseudo-MSRV + - `build-dependencies` and `dev-dependencies` shouldn't have explicit + standard library dependencies + - Didn't want a `--no-resolve-sysroot` flag (to prevent loading crates from + the sysroot) as this would break people who use unstable crates from the + sysroot + - There were specific implications identified with respect to + versioning/dependency specification: + - Crates in the sysroot would need to have the same version as the + compiler + - Standard library dependencies effectively allow language version pinning + - A `^1.0` version would add constraints on Rust 2 + - It would stabilise the name of `test` + - A mechanism for building unstable code in the sysroot on the stable + compiler would be necessary + - This is the current use case desired by Rust for Linux + - There were many concerns about avoiding abuse of this and discussion + of mechanisms to confirm that only the intended unstable crates were + built on a stable compiler + - The RFC stalled for a few months and then was blocked by a mechanism for + unstable Cargo features in July 2017 + - In February 2018, the RFC was closed in favour of [cargo#4959] +- **[cargo#2768]: Phase 1 of stdlib dependencies**, [Ericson2314], Jun 2016 + - An initial implementation of [rfcs#1133]. + - This PR received almost no feedback. +- **[cargo#5002]: Explicit standard library dependencies**, [Ericson2314], Feb + 2018 + - A issue split-off from [rfcs#1133] with the goal of discussing how explicit + dependencies on the standard library could work + - This issue received no feedback +- **[cargo#5003]: Implicit standard library dependencies**, [Ericson2314], Feb + 2018 + - A issue split-off from [rfcs#1133] with the goal of discussing how implicit + dependencies on the standard library could work + - This issue received no feedback + +## [xargo] and [cargo#4959] (2016) +[a1-xargo-and-cargo-4959-2016]: #xargo-and-cargo4959-2016 + +This section contains all of the sources related to [xargo] and [cargo#4959]: + +- **[xargo]**, [japaric], Mar 2016 + - A developer tool that wraps Cargo to provide build-std-like functionality + - Built while [rfcs#1133] was still being discussed + - Builds custom sysroots based on the configuration in a `Xargo.toml` + - Requires nightly as it builds the standard library and gets standard + library sources from the `rust-src` component + - Supports multi-stage builds, features, flags, etc. + - Xargo is used instead of Cargo and has the same command line interface, but + ensures that the locally-built sysroots are used + - In maintenance mode since January 2018 + - Cargo team were interested in incorporating its functionality at the time +- **[cargo#4959]: Sysroot building functionality**, [japaric], Jan 2018 + - Successor of [rfcs#1133], inspired by [xargo] + - Xargo was fairly widely used but had issues due to being a wrapper around + Cargo rather than a part of Cargo + - It didn't replicate Cargo's fingerprinting so would unnecessarily perform + rebuilds + - It didn't track changes to sysroot sources to trigger rebuilds when they + changed + - It didn't have a rustup shim so users couldn't use the toolchain selection + arguments typical of rustup shims (e.g. `+nightly`) + - There were deviations from built-in Cargo commands + - It suggested adding a `[sysroot]` section in the `.cargo/config` + - Could specify features/flags/stages for + `core`/`compiler_builtins`/`alloc`/`std`/etc + - Can also specify `rust-src` path + - Optional, defaults to `rust-src` component path + - If set, Cargo would rebuild sysroot crates for targets, put them in the + `target` directory and use them for rustc/rustdoc/etc + - These would be rebuilt when profiles change, using Cargo's usual + fingerprinting mechanism + - These crates would always be built using the release profile + - Like xargo, would support multi-stage builds + - e.g. `std` built with default sysroot, then `test` built in sysroot w/ + new `std`, then a new sysroot made with both the new `test` and new + `std` + - None of it would apply to `build.rs` + - There was a variety of feedback: + - There should be restrictions on what crates could be put in the sysroot + should be added so that this mechanism could not be used for third-party + crates + - Procedural macros would also need to use the host sysroot + - Syntax was not user-friendly enough + - The ability to customise std and not require the `rust-src` component be + used was requested + - Similarly, requests for `[patch]` sections + - There was a mention of a imminent merging of rustup and Cargo which could + be relevant + - Spoiler: this didn't happen + - Exposing bootstrap stages is cumbersome, error-prone and brittle and Cargo + should just figure that part out automatically + - There were arguments against further reliance on the sysroot as a concept + - Incremental/IDE support is allegedly more challenging with the sysroot + - Some proposed features in the ether circa March 2018 related to + module-system namespacing/lighter `extern crate` would be made more + challenging if the entire sysroot were automatically imported + - Many of the other arguments against the sysroot were actually arguments + against a pre-built standard library that cannot possibly satisfy all + users +- **[cargo-xbuild]**, [rust-osdev], May 2018 + - A simplified fork of `xargo` + - Wrapper around `cargo build` that uses a custom sysroot according to the + configuration in `package.metadata.cargo-xbuild` in `Cargo.toml` + - Now recommends using `--build-std` in Cargo instead + +## [rfcs#2663] (2019) +[a1-rfcs-2663-2019]: #rfcs2663-2019 + +This section contains all of the sources related to [rfcs#2663]: + +- **[rfcs#2663]: `std` Aware Cargo**, [jamesmunns], Mar 2019 + - Re-building the standard library in this proposal is motivated by: + - Being able to use the standard library with tier three and custom targets + that do not have a pre-built standard library + - Customising the standard library with different feature flags + - Applying different codegen flags to the standard library + - This proposal largely focuses on what the user experience of build-std + should be and has many unresolved questions and details left for the + implementers to work out + - Unlike [rfcs#1133], this RFC only focuses on when the standard library + should be rebuilt, rather than how a dependency on the standard library + should be declared + - It is primarily inspired by [xargo] and also cites [rfcs#1133], + [cargo#5002] and [cargo#5003] + - There are four objectives of the RFC: + - Allow core to be recompiled when a target isn't available from rustup + - This should be possible on a stable toolchain, but will require nightly + to.. + - ..set feature flags + - ..modify core + - Cargo should recompile core when a custom target spec is used, feature + flags are modified, profile configuration is changed or the sysroot is + patched + - As above, on a stable toolchain, only the profile configuration will + be able to be changed to trigger a rebuild + - Only the root crate's feature flags and profiles would be respected, + not those of dependencies + - Unless opting into Cargo features, core does not need to be explicitly + specified as a dependency in `Cargo.toml` + - Sources for core are from the `rust-src` component + - The proposal lists three implications of the above feature: + - Target specification format would need to be stabilised + - It isn't explained why this is a necessary implication + - Rust implementation of `compiler_builtins` would need to be used for + custom targets + - `RUSTC_BOOTSTRAP` needs to be set for core + - Allow use of "stable" Cargo features from Cargo + - This section proposes that a mechanism exist to declare Cargo features + as stable and unstable and that only stable features be usable on the + stable channel (i.e. resolving [cargo#10881]) + - There are no specifics on how these features would be declared, but + the syntax for using a feature is described as being identical for + stable and unstable features + - Unstable feature flags in the standard library could be stabilised by + following the same stabilisation process as anything else in the + standard library + - The proposal does not mention anything about the unification of the + features of the dependencies of the standard with those same + dependencies of the user's crate + - Allow alloc/std to be recompiled when a target isn't available and allow + use of "stable" Cargo features from alloc/std + - Exactly as above + - Doesn't address complexities with alloc/std - when does Cargo need to + build alloc or std and not just core? How does Cargo know this? + - Allow users to provide their own versions of core/alloc/std + - Would always require nightly + - Suggests using regular patch syntax under `patch.sysroot` key + - There are many unresolved questions in the RFC: + - How to specify a dependency on the standard library in a crate? + - How does a `no_std` crate tell Cargo it doesn't need to build std? + - Should std be rebuilt if core is? + - Should there be tamper-detection for the standard library? + - Should the standard library be built locally (per project) or globally + (shared between projects)? + - What to do with the standard library's `Cargo.lock`? + - Should profile overrides *always* rebuild std? + - Should providing a custom standard library require nightly? + - Should customising the standard library be permitted? + - There are also a handful of future possibilities mentioned: + - core/std could be unified into a single crate with different features to + represent the differences that exist between the crates + - The project could choose to stop shipping a precompiled standard library + - This RFC was sufficiently vague that it was unlikely to be accepted as + written - to accept it is largely to assert that something like build-std is + desired, which is not controversial, but the tricky details which make + build-std difficult would still need to be designed and discussed + - A variety of feedback was received: + - Enabling customisation of std may not be practical because it would + require all combinations of Cargo feature flags be tested + - It is asserted that this testing would be necessary without considering + alternatives (such as a documented stability policy for the standard + library's feature flags) + - It is suggested that only feature flags of omission (that remove parts + of the standard library) be used to avoid this + - It was later relayed that Cargo features do not work this way + - Another suggestion was only no flags enabled and all flags enabled could + be tested and that this would be sufficient + - There is disagreement over whether specifying a custom standard library + should require nightly + - It is already possible to specify a standard library dependency in + `Cargo.toml` with a path and have it override that crate + - At the time of the conversation, this was used by the `embed-rs` crate + to override some of the standard library's future types + ([source][embed-rs-source]/[`Cargo.toml`][embed-rs-cargo-toml]) + - Some argue that it should be possible to do this on stable because any + alternative implementation would necessarily only build on nightly + anyway + - It is argued both that the target-spec-json format *would* and *would not* + need to be stabilised if it were supported by build-std + - As build-std would make it more practical to use these custom targets on + a stable toolchain, some argue that the format would need to be made + stable + - The host sysroot would need to be used for `build.rs` and procedural + macros + - There are C dependencies of the standard library that are difficult to + build and would need some consideration of this to ensure the usability of + build-std + - `libbacktrace` has been replaced by `gimli` so this is less of a concern + now + - There are concerns that rebuilding the standard library whenever the + profile is changed would be too disruptive + - e.g. changing the optimisation level triggering a rebuilding of std + would add to compilation times + - It is suggested that the standard library only be rebuilt if a crate + adds an explicit dependency on them + - Cargo's default profiles may not match the configuration of the + pre-built standard library + - Research is required to ensure that the default configuration of the + standard library is known to Cargo and not just bootstrap + - Alternatively, only rebuild for ABI-modifying flags + - Sanitizers require sanitizer runtimes to be present and these are not + configured by Cargo features + - Stable/unstable Cargo features is an RFC of its own + - Only considering the root crate would break dependencies that enable + specific standard library features + - Cargo needs to know what crates to build when a rebuild is triggered + - e.g. only `core`, `alloc` too, `std` as well? + - There are various arguments that the standard library and its crates + should not be special-cased in any way + - There is pushback to this arguing that the standard library is + inherently special + - Trying to make the standard library a regular crate that is versioned, + on crates.io and does not exist in the sysroot is often raised in + these prior art and makes build-std much more complicated and less + likely to succeed + - Users should have the option of using the C implementation of + `compiler_builtins` + - Crates will need to be able to specify a lack of dependency on the + standard library (e.g. `no_std` crates) + - It is important that a pre-compiled standard library and locally-compiled + standard library have identical behaviour + - t-lang [discussed the RFC in a meeting][rfcs#2663-t-lang] + - There weren't any concerns, except: + - Niko raised that putting something behind a Cargo feature that was + previously not gated behind a Cargo feature breaks users who use + `default-features = false` and this should be resolved as it has + implications for the standard library's development + - It was suggested that `default-features = false` could be prohibited + for standard library dependencies + - Ultimately closed with interested parties directed to the + [wg-cargo-std-aware] repository + - There was additional feedback in the draft version of the RFC which was + shared with some project members - [jamesmunns/rfcs#1] + - Initially the RFC did not clarify when Cargo would trigger a rebuild + - Cargo needs to know how not to do any of the proposed machinery for the + `core` and `compiler_builtins` crates itself + - `rustc_inherit_overflow_checks` could be removed + - Cargo features enabled in the standard library cannot just be decided by + the root crate + - There was disagreement about whether this is accurate + - Interactions with `rustc-std-workspace-core` are unclear + - `patch.sysroot.$crate` key rather than re-using `patch.$crate` makes the + standard library special + +## [wg-cargo-std-aware] (2019-) +[a1-wg-cargo-std-aware-2019-]: #wg-cargo-std-aware-2019- + +After the above issues, pull requests, RFCs and discussions, the +[wg-cargo-std-aware] was created to host issues related to build-std and the +effort which resulted in the current unstable implementation of build-std. + +Unlike prior art which predates [wg-cargo-std-aware], [wg-cargo-std-aware]'s +issues capture the majority of the tricky details involved in build-std but +often do not have much discussion to summarise. + +Issues in [wg-cargo-std-aware] are very varied and so are split into eight +categories in this proposal: + +### Use cases +[use-cases]: #use-cases + +These issues collect and elaborate on the use cases that users have for +build-std: + +- **[wg-cargo-std-aware#2]: Build the standard library with custom profile + settings**, [ehuss], Jul 2019 + - Currently, the standard library is only available with the profile settings + chosen when Rust is distributed + - "release" profile with `codegen-units=1` and `opt-level=2` + - Using different settings is desirable + - When build-std builds Cargo, it should build using the current profile + - Profile overrides can be used to use different settings for the standard + library specifically + - Users currently do not expect the standard library to be rebuilt when + changing the profile + - e.g. setting `opt-level=3` will not currently rebuild std and if it + started to do so then this could significantly increase build times for + small projects + - At time of writing, build-std was intended to be strictly opt-in so this + would not be an issue + - It was later clarified that the concern was not "whether configured + profile settings would apply to the standard library", but rather "whether + that always triggers a rebuild" + - build-std should always re-use pre-built artifacts if such artifacts + exist and match the desired profile + - Risk that profile overrides could expose internal details about the standard + library, like the `compiler_builtins` crate +- **[wg-cargo-std-aware#3]: Build the standard library for an unsupported + target**, [ehuss], Jul 2019 + - build-std is desired to make it easier to build the standard library for + targets that do not have a pre-built standard library, including unsupported + targets (i.e. `no_std` environments) and custom targets (using + target-spec-json) + - Tools like [xargo] and [cargo-xbuild] are used to do this + - Custom targets are already supported by Cargo, so work for + [wg-cargo-std-aware#2] should overlap + - `no_std` binaries often require nightly features + - `core` is not maximally portable + - It may be worth exploring how to lower the barrier to porting the standard + library to a new platform using build-std +- **[wg-cargo-std-aware#4]: Build the standard library with different cargo + features**, [ehuss], Jul 2019 + - Some users want to enable or disable Cargo features to remove or modify + parts of the standard library + - e.g. to change the formatting machinery to emphasise code size, or remove + panic machinery when unused + - There was agreement that this customisation should be limited to code behind + `cfg(feature = "..")` attributes, not arbitrary `cfg`s + - [cargo#8490] added `-Zbuild-std-features=` to support this experimentally + - An example provided was wanting to be able to toggle whether `RefCell` could + provide backtraces when panicking for outstanding borrows +- **[wg-cargo-std-aware#7]: Custom source standard library**, [ehuss], Jul 2019 + - Some users want to be able to replace the source of the standard library + with a custom implementation + - The issue notes that this is unlikely to be stabilised in the foreseeable + future + - It was suggested that it would be better to allow the user to supply a + pre-built artifact for any dependency in the crate graph + - As of 2022, miri needed its own sysroot build and used [xargo] to do that +- **[wg-cargo-std-aware#19]: Use in rustbuild**, [Ericson2314], Jul 2019 + - build-std could be used as part of the [rust-lang/rust] bootstrap, this + would.. + - ..simplify bootstrap + - ..reduce gap between official and user builds + - ..demonstrate that Cargo is expressive enough + - Most discussion took place in internals thread + - See *Dogfooding -Z build-std in rustbuild* below +- **[wg-cargo-std-aware#36]: Better understand the no_std binary use case**, + [ehuss], Sep 2019 + - `no_std` binaries can be tricky (e.g. may require manually linking libc) and + it would be nice if this were smoother + - This use-case seems adjacent to build-std +- **[wg-cargo-std-aware#42]: metaprogramming and bootstrapping**, + [Ericson2314], Sep 2019 + - `build.rs` and `proc_macro` ought to be able to leverage build-std too + - There was no further relevant discussion +- **[wg-cargo-std-aware#61]: Can we tailor the compiler-builtins package that is + compiled and linked together with the core crate?**, [parraman], Nov 2020 + - User wants to change the `compiler_builtins` version using build-std + - Closed as duplicate of [wg-cargo-std-aware#7] +- **internals.r-l.o: [Dogfooding -Z build-std in rustbuild][wg-cargo-std-aware#19-internals]**, [Ericson2314], Jan 2021 + - Proposes using `-Zbuild-std` in rustbuild in order to dogfood it to more + users + - Suggests that this would lead to Cargo needing to learn some of bootstrap's + current behaviour and that Cargo would need to be changed more often to + accommodate changes that bootstrap receives + - Suggests that changing the Cargo version contained within a beta release + would make this easier + - More detail is provided on how this could be implemented + - This would require building rustc with the beta standard library + - At the time this was seen as unlikely, but has since been implemented + ([rust#119899]) + - This implication was rejected as it was argued that `-Zbuild-std` would + eventually allow a different source to be used + - `-Zbuild-std` *might* be useful for bootstrap but using it in bootstrap is + unlikely to help advance the stabilisation of build-std + - There was detailed discussion about the degree to which `x.py` is just a + wrapper around Cargo and whether the sysroot is necessary + - Rust for Linux developers add that it is important for their use case that + there is a simple way to build the standard library without requiring Cargo + - Ideally with a mechanism for doing this using a stable toolchain +- **[wg-cargo-std-aware#77]: Cannot test bug fixes of dependencies of + `libcore`/`libstd` on targets where `-Zbuild-std` is required**, [cr1901], Nov + 2021 + - User wants to change the `compiler_builtins` version to test upcoming + bugfixes + - Closed as duplicate of [wg-cargo-std-aware#7] + +### Support for build-std in Cargo's subcommands +[support-for-build-std-in-cargo-subcommands]: #support-for-build-std-in-cargos-subcommands + +These issues discuss the complexity involved in supporting build-std in specific +subcommands of Cargo: + +- **[wg-cargo-std-aware#20]: Support `cargo metadata`**, [ehuss], Sep 2019 + - `cargo metadata` outputs machine-readable JSON containing an array of all of + the packages in the workspace + - The format has some stability guarantees + - build-std would need to be supported in `cargo metadata` but the issue does + not describe how that should work + - There was no discussion on the issue +- **[wg-cargo-std-aware#21]: Support `cargo clean`**, [ehuss], Sep 2019 + - `cargo clean -p std` doesn't work but should + - There was no discussion on the issue +- **[wg-cargo-std-aware#22]: Support `cargo fetch`**, [ehuss], Sep 2019 + - Implemented by [jyn514] in [cargo#10129] + - Fetches crates.io dependencies of the standard library and should continue + to do this if the dependencies are not vendored +- **[wg-cargo-std-aware#23]: Support `cargo vendor`**, [ehuss], Sep 2019 + - `cargo vendor` doesn't understand build-std + - Vendoring the standard library and its dependencies would lock the user to a + specific toolchain version, which would not be desirable + - If `rust-src` contained a vendored standard library then `cargo vendor` + would not need changed + - Makes source replacement or `path` overrides challenging to implement + - At the time of writing, vendoring the standard library was difficult as + `cargo vendor` does not support vendoring a subset of the packages in a + workspace, and the standard library did not have its own workspace at the + time + - Implemented by [Gankra] in [cargo#8834] and [rust#78790] by adding `patch` + entries to all members to use the vendored crates + - Later reverted in [cargo#8968] and [rust#79838] due to regressions + - [cargo#8962]: Cargo was always updating the registry index with + `-Zbuild-std` + - [cargo#8963]: Cargo was producing unused patch warnings with + `-Zbuild-std-features=compiler-builtins-mem` + - [cargo#8945]: Custom targets were broken as Cargo didn't expect + `rustc-std-workspace` in the vendored dependencies + - Using patches to emulate vendoring (by changing `path`) is not correct and + proper vendoring should be used + - [rust#128534] moved the standard library to its own workspace which should + make vendoring possible + - There were some suggestions that using a vendored standard library should + be optional as it prevents patching the standard library +- **[wg-cargo-std-aware#24]: Support `cargo pkgid`**, [ehuss], Sep 2019 + - `cargo pkgid` doesn't support build-std but should + - There was no discussion on the issue +- **[wg-cargo-std-aware#26]: Possibly support `-p` for standard library + dependencies**, [ehuss], Sep 2019 + - `-p` normally works for any dependency + - There was discussion of how `-p` could work if `-Zbuild-std` were the final + interface for build-std + - There is a risk of leaking the dependencies of the standard library + depending on how this support is implemented +- **[wg-cargo-std-aware#45]: Support --build-plan**, [ehuss], Sep 2019 + - `--build-plan` does not include anything from build-std + - [cargo#7614] is a long and still-actively-discussed thread discussing the + future of `--build-plan` with mixed opinions + - `--build-plan` is still unstable so support for build-std can be a blocker + for it if build-std lands first or vice-versa +- **[wg-cargo-std-aware#83]: Allow std to be specified as package spec**, + [dullbananas], Apr 2023 + - User wants to use `-p` with build-std, closed as duplicate of + [wg-cargo-std-aware#26] + +### Dependencies of the standard library +[appendix-dependencies-of-the-standard-library]: #dependencies-of-the-standard-library + +These issues discuss the challenges involved in building some of the +dependencies of the standard library: + +- **[wg-cargo-std-aware#15]: Deps compiler_builtin**, [ehuss], Jul 2019 + - There's a `c` feature of this crate that uses optimised C versions for some + of the intrinsics + - This is used by default + - The `mem` feature of this crate provides some demangled symbols used in + `no_std` builds of `alloc` when there is not a `libc` to provide those + symbols + - Built using a large number of codegen units to force each function to into + its own unit + - [rust#135395] attempted to fix compiler-builtins' codegen unit + partitioning in the compiler + - It forced the compiler to use a large number of codegen units for the + compiler-builtins crate + - There is a tension between the embedded use-case for compiler-builtins' + intrinsics and what Rust for Linux wants + - RfL depends on compiler-builtins being compiled into a single object + file + - It was mentioned that almost all intrinsics now have a Rust implementation + and that could be made the default for build-std eventually + - Some concerns that this would result in divergence between locally-built + and distributed standard libraries +- **[wg-cargo-std-aware#16]: Deps: backtrace**, [ehuss], Jul 2019 + - `libbacktrace` previously required a C compiler but has since been replaced + by `gimli` in the standard library and so this is no longer an issue + ([rust#46439]) +- **[wg-cargo-std-aware#17]: Deps: sanitizers**, [ehuss], Jul 2019 + - At the time of writing, sanitizers are a dependency of the standard library + and require LLVM components (located using `LLVM_CONFIG`) + - It is suggested that the implementation of sanitizers could be revisited so + that they are distributed as plain libraries alongside rustc and linking + against them rather than rebuilding them as part of build-std + - There would be no dependency on LLVM sources outside of building rustc. + [rust#31605] could be resurrected to do this. +- **[wg-cargo-std-aware#18]: Deps: proc-macro**, [ehuss], Jul 2019 + - Procedural macros are already built with the host sysroot so wouldn't use a + customised standard library +- **[wg-cargo-std-aware#65]: Does not work with vendoring**, [raphaelcohn], Feb + 2021 + - Closed as duplicate of [wg-cargo-std-aware#23] + +### Design considerations +[design-considerations]: #design-considerations + +These issues document open design questions for build-std: + +- **[wg-cargo-std-aware#5]: Cargo standard library dependencies**, [ehuss], Jul + 2019 + - How will a dependency on the standard library be expressed in `Cargo.toml`? + - There are various requirements laid about by the issue: + - Users should have to opt-in to build the standard library + - It should be able to support alternative standard library implementations + - Cargo needs to be able to pass `--extern name` to specify explicit + dependencies to add to the extern prelude, even for pre-build artifacts + - References [rfcs#1133], [rfcs#2663], [cargo#5002], [cargo#5003], [rust#57288] + - [rust#57288] tracks the effort to stop requiring `extern crate`. As of + October 2023: + - `--extern proc_macro` passed by Cargo to avoid needing + `extern crate proc_macro` + - `extern crate test` is still required, but `test` is unstable + - `extern crate std`/`extern crate alloc` still required for `no_std` - + Cargo has no way of knowing that a no_std crate uses std and that + `--extern crate std` ought to be used + - `extern crate core` as above for `no_core` + - How does Cargo handle multiple crates in the graph declaring dependencies + on the standard library? + - Unify them, there is no concept of a "version" for the standard library + dependencies + - e.g. a union of all features + - How to balance implicit vs explicit dependencies on the standard library? + - Existing stable crates depend on the standard library implicitly + - Tempting to declare every crate does unless opting out + - Existing stable `no_std` crates do not have an explicit opt-out but are + compatible with targets that work without a standard library at all + - How to express a dependency on a pre-built artifact vs building from source? + - Users could have no direct control, rebuilds are triggered based on other + factors (e.g. profile settings, target, feature flags, etc) if a pre-built + artifact does not exist for that configuration + - Any differences between a locally-built artifact and pre-built artifact is + a bug + - It was suggested that the *Pre-Pre-RFC: making `std`-dependent Cargo + features features a first-class concept* proposal be adopted (see below) +- **[wg-cargo-std-aware#6]: Target specification?**, [ehuss], Jul 2019 + - build-std makes target-spec-json more usable on stable and project teams + will need to decide if they are comfortable with the current format or if it + needs changed + - Custom targets are more likely to require nightly as not having a standard + library, they will need nightly to use build-std + - If is opined that if build-std allows the standard library to be built for + custom targets then that should require an explicit decision to stabilise + the format + - e.g. if `cargo +stable build --target=foo.json` works then the format is + de-facto stable + - The standard library's `build.rs` checks for whole target string which + doesn't support custom targets very well + - There are a handful of general requests of the target-spec-json format: + - Switch to TOML + - Clean up LLVM-specific details + - Allow inherited specs + - Use them in rustc rather than defining them in code +- **[wg-cargo-std-aware#8]: Standard library portability**, [ehuss], Jul 2019 + - Discusses needs for making the standard library more portable and configurable + - i.e. not strictly build-std related but build-std benefits from a more + portable standard library + - References a bunch of previous/related efforts: + - [rfcs#1502] + - [internals.r-l.o: Fleshing out libstd scenarios] + - [internals.r-l.o: Refactoring libstd for ultimate portability] + - [rfcs#1868] + - [A vision for portability in Rust] + - [portability-wg] + - [embedded-wg] + - Somewhat out-of-scope for wg-cargo-std-aware + - The mechanism for increasing portability may need to be exposed via + `Cargo.toml` (e.g. features) + - It is an impediment of making it easier to build std for unsupported targets +- **[wg-cargo-std-aware#11]: Downloading source**, [ehuss], Jul 2019 + - rustup can download `rust-src` which doesn't include the dependencies of the + standard library, but does include the `Cargo.lock` + - Preference that rustup not be required + - Acquiring dependencies may be challenging + - build-std needs to be transparent to be a first-class feature + - Cargo has support for downloading various things but needs to know where + the source is + - rustup knows toolchain and commit but Cargo doesn't + - Could have a reasonable default probing location (e.g. for distros) and + could query rustup + - [cargo#2768] allowed the user to specify a path in their configuration + - Could publish the source for each standard library version to crates.io +- **[wg-cargo-std-aware#12]: `Cargo.lock` and dependency resolution**, [ehuss], Jul 2019 + - Very likely that the standard library will need to be built with the same + dependency versions as distributed version + - May end up with different versions of dependencies as the user's crate + - Locking dependencies may be good to guarantee determinism + - Closed assuming that this is a settled question + - Current implementation uses a separate resolve to keep dependency versions + separate from user's dependency versions +- **[wg-cargo-std-aware#13]: Default feature specification**, [ehuss], Jul 2019 + - How to determine the default set of cfg values for the standard library? + - Currently set by bootstrap + - Also sets environment variables used by build scripts (e.g. `LLVM_CONFIG`) + - Various possibilities + - Hardcode in Cargo, not idea long-term + - Some configuration in source distribution + - Make user explicitly declare them + - Similar to [wg-cargo-std-aware#4] +- **[wg-cargo-std-aware#14]: Default flags and environment variables**, + [ehuss], Jul 2019 + - There are additional rustc flags passed to standard library crates which + could be duplicated initially but not long-term + - There are suggestions that a config file may be necessary as in + [wg-cargo-std-aware#13] + - There is overlap with [wg-cargo-std-aware#28] + - There is a summary of the differences in flags between bootstrap and + build-std in + [wg-cargo-std-aware#14 (comment)][wg-cargo-std-aware#14-review] +- **[Pre-Pre-RFC: making `std`-dependent Cargo features features a first-class + concept][wg-cargo-std-aware#5-internals]**, [bascule], Aug 2019 + - API guidelines say that Cargo features should be strictly additive and that + gating std support should be behind a `std` feature + - `no_std` users end up always using `default-features = false` and opting + into everything except `std` + - Unintentionally enabled `std` features can prevent linking, often deep in + dependency hierarchies + - Proposes top-level `std-features = true` that automatically enables or + disables the `std` features of dependencies +- **[wg-cargo-std-aware#29]: Support different panic strategies**, [ehuss], Sep 2019 + - Current implementation hardcoded for `unwind` + - How does Cargo know what to use? Inspecting the profile? + - Always build both `unwind`/`abort`? + - `-Cpanic=abort` needs to be passed for `abort` crate + - Some targets default to `abort` + - `abort` crates currently rebuild w/ `unwind` when built with `libtest` + - Cargo should work transparently - user sets `panic` in their profile and Cargo + respects that and handles everything else + - As everything is compiled by Cargo, can lift restrictions like `-Cpanic=abort` + being necessary for `panic_abort` crate + - Cargo should be able to take a more "pure" stance relative to `libtest` + - [rust#64158] later merged supporting `panic=abort` with `libtest` + - It is not yet stable ([rust#67650]) + - Ideally only compile one panic strategy crate + - Target-specific settings are tricky + - Almost all cases of building the standard library from source are `panic=abort` + targets w/ no unwinding support + - Is profile sufficient to build `panic_abort`/`panic_unwind`? + - Some crates don't want any panic strategy - when not using the standard library, + not defining a panic strategy may make sense + - `std` has `#![needs_panic_runtime]` + - build-std may require `-Zpanic-abort-tests` +- **[wg-cargo-std-aware#30]: Figure out how to handle target-specific + settings**, [ehuss], Sep 2019 + - Some targets have special logic and it isn't clear how this can be handled + by Cargo + - Often target-specific and toolchain-related + - Can potentially whitelist or blacklist targets +- **[wg-cargo-std-aware#38]: Consider doing a forced lock of the standard + library**, [ehuss], Sep 2019 + - Ensure that the in-memory `Cargo.lock` is not modified + - Hard to guarantee in practice + - Could add a test to assert this + - [cargo#13916] attempted to add a check to verify the virtual workspace + for the standard library post-resolve is a strict subset of it + pre-resolve + - [rust#128534]/[cargo#14358] meant that the standard library has its own + workspace and build-std does not need to generate one + - On-disk lockfile is now used so do not need to worry about it changing + - Still need to implement `--locked` behaviour + - Doing so would break `[patch]` with build-std +- **[wg-cargo-std-aware#39]: Figure out interaction with public/private dependencies**, [ehuss], Sep 2019 + - Public/private status of the standard library is hardcoded to public in MVP - should + it be? + - If defined in `Cargo.toml`, it will likely inherit visibility from that, which defaults + to private. +- **[wg-cargo-std-aware#43]: What will the UX be for enabling "build libstd" + logic?**, [alexcrichton], Sep 2019 + - How will build-std eventually be enabled? + - At time of writing, `-Zbuild-std=$crates`, but what should the eventual + syntax be? + - e.g. `build.std` option in `Cargo.toml` or `.config/cargo` + - Opting into build-std as an unstable feature should be a `-Z` flag or entry + in `cargo-features` + - Eventually not a flag at all, and Cargo should try to compile the standard + library automatically if a compiled standard library is not available + - Concern that the pre-compiled standard library may not match by default + and there would be too many false-positive rebuilds + - For missing targets, need to distinguish between forgetting to run + `rustup target add` and needing to compile the standard library + - `[profile]` could not affect the standard library by default + - There are cases where implicit rebuilding could be desirable + - e.g. ABI-modifying flags + - Passing a flag on every invocation is not a good user experience + - [cargo#10308] is a prototype with explicit standard library dependencies + - Idea posted on internals, see *Build-std and the standard library* below + - If enabled manually, [cargo#8733] expressed the need to enable build-std + only for certain targets, in particular when cross-compiling to no-std + targets +- **[wg-cargo-std-aware#46]: How to handle special object files?**, [ehuss], Sep 2019 + - How to handle pre-built object files needed to link on a target? + - e.g. `crt1.o`, `crti.o`, `crtn.o`, etc for musl or `dllcrt2.o`, + `crt2.o`, etc for Windows GNU + - Cargo is target agnostic and does not want to hardcode logic for particular + targets + - The target definitions define which files they expect to find and link + against + - [rust#68887] adds a `self-contained` directory which could help +- **[wg-cargo-std-aware#47]: how to handle pre-built linkers?**, [ehuss], Sep + 2019 + - Some pre-built targets ship with a copy of rust-lld or gcc to assist with + linking + - `rust-lld` could be shipped as a rustup component or the user could be + forced to install these components + - Since this issue was filed, `rust-lld` is now always shipped + - rustc finds `rust-lld` via the sysroot and if the sysroot is not provided + then another mechanism will need to be used to find `rust-lld` +- **[wg-cargo-std-aware#50]: Impact on build scripts that invoke rustc**, [jdm], Oct 2019 + - Need to make sure that build scripts that invoke rustc are able to do this with the correct + standard library + - Closed as t-cargo do not want to encourage or support build probes +- **[wg-cargo-std-aware#51]: Plan for removal of `rustc-dep-of-std`**, [ehuss], Nov 2019 + - `rustc-dep-of-std` is a feature that packages used as dependencies of `std` have that, + when enabled, use to declare explicit dependencies of `core`/`compiler_builtins`/etc + - May not be necessary once these packages are always declaring dependencies on + `core`/etc + - Cargo would still need to work out what to do when seeing explicit dependencies + - These dependencies could always be built with build-std but that would involve + using build-std in bootstrap which may not be desirable + - Alternatively, have some mechanism for telling Cargo that the explicit dependencies + in this instance aren't from the sysroot but an rlib from bootstrap + - Could be automatic or variation on patch syntax +- **[wg-cargo-std-aware#57]: Support building workspace with separate build-std + parameters**, [jschwe], Jun 2020 + - User wants to have one package depend on `core` and `alloc` and another + package depend on `core`, `alloc` and `std` but this isn't possible with the + current experimental flag + - Closed as duplicate of [wg-cargo-std-aware#5] +- **[wg-cargo-std-aware#68]: Support Profile Guided Optimisation (PGO)**, [errantmind], Mar 2021 + - `profiler_builtins` isn't compatible with `no_core` + - i.e. `profiler_builtins` requires `core` so if building with profiling, need to + build `core` first w/out the profiling instrumentation + - [rust#79958] improves the error message for this +- **internals.r-l.o: [Build-std and the standard library][wg-cargo-std-aware#43-internals]**, [ehuss], Dec 2019 + - Proposes changing the standard library so all of its crates can build on all + targets + - It may not work and not all APIs will be available, but there will be no + compilation errors + - i.e. add runtime errors or cfgs + - Makes things simpler because build-std can become a simple toggle + - Building an empty standard library is preferred over not building at all as + this avoids breaking `no_std` crates that build for tier three targets with + no standard library support at all + - Suggestion that the standard library use Cargo features more so that most + functionality is gated by default + - Fixes this issue as most of the standard library would be absent by + default + - Some prefer that the standard library be listed in the `Cargo.toml` + eventually but that this would work initially + - Explicit standard library dependencies are being considered + - It could be a inconvenience to need to specify the standard library + dependencies in most cases + - Most packages don't need it + - Need default for packages that do not specify a standard library + dependency + - References to the "portability lint" + - Lint that triggers when calling an available platform-specific code so + that users can avoid making code less portable + - Lint was later deemed infeasible + - Described as necessary step towards long-standing goal to move from a + multiple-crates model of the standard library to a cfg-flag model + - Cargo could learn what targets a package supports + - e.g. libstd could declare this + - Doesn't work for custom targets (i.e. target-spec-json) + - Could accidentally allow the standard library to be used in `no_std` + projects + - Adds noise to `Cargo.toml` +- **[wg-cargo-std-aware#85]: Figure out rust-lang/rust testing strategy**, [ehuss], Mar 2023 + - How to test build-std when it is tightly coupled to the standard library source tree? + - Could use rust-toolstate but wasn't very good + - Could use git subtrees (or [JOSH]) + - Could aim to reduce coupling between the standard library and Cargo + - Which targets get tested and how? +- **[wg-cargo-std-aware#86]: Consider limiting supported targets in initial + stabilization**, [ehuss], Mar 2023 + - Supporting build-std on all targets is a large effort so consider + supporting fewer targets initially + - There could be a concept of stability for targets w/ build-std + - i.e. this target works with build-std on nightly only + - Generally supportive reception but every contributor wanted their target to be + one of the supported ones +- **[wg-cargo-std-aware#88]: `cargo doc -Zbuild-std` doesn't generate. links to the standard library**, [jyn514], Jun 2023 + - Cargo doesn't treat the standard library as coming from crates.io and the standard + library doesn't have `html_root_url` set, so local documentation doesn't get links to + the standard library + - Bootstrap passes `-Zcrate-attr="doc(html_root_url=..)"` +- **[cargo#12375]: artifact-dependencies doesn't compose well with build-std**, [tamird], Jul 2023 + - The current unstable implementation of build-std is required when building + for tier three targets but it is impossible to enable it for an artifact + dependency. +- **[wg-cargo-std-aware#90]: restricted_std applicability to custom JSON targets**, [Mark-Simulacrum], Feb 2024 + - The standard library's `build.rs` changed to use `TARGET_OS` rather than an complete + target name so JSON targets don't end up enabling `restricted_std` + - Need to work out what guarantees are desired + - Should std-supported be a property of the target, not `build.rs`? + - Idea behind the current approach: + - Slightly modified target specs should continue to work and permit usage of the + standard library + - Different target specs that are very different from existing targets ought not + permit usage of the standard library + - [rust#71009] needs to be resolved first + - Aims to de-stabilise target specifications, making it prohibited to pass one to + the compiler + - cc [wg-cargo-std-aware#6] + - If target-spec-json is unstable then their behaviour with build-std is also unstable +- **[wg-cargo-std-aware#92]: Consider flag to mark unstable targets**, [madsmtm], Feb 2024 + - Tier three targets are required to use nightly because of build-std + - Stable build-std effectively stabilises these targets, so users would stop using + nightly and get fixes less frequently + - Is a notion of unstable or incomplete targets desirable? +- **[wg-cargo-std-aware#95]: Use `build-std=core` on a custom target + automatically**, [nazar-pc], Mar 2024 + - Closed as duplicate of [wg-cargo-std-aware#43] + +### Implementation +[implementation]: #implementation + +These issues include bug reports for the current unstable implementation of +build-std, as well as unresolved questions for the implementation, both from +[wg-cargo-std-aware]. In addition, this section will list the pull requests in +[rust-lang/cargo] and [rust-lang/rust] that directly contributed to the +implementation. + +- **[cargo#7216]: Basic standard library support.**, [ehuss], Aug 2019 + - Initial implementation of `-Zbuild-std` + - Constructed a synthetic workspace for the standard library and used + `--extern` to provide the new dependencies to rustc. + - Required `--target` to be passed so that the host sysroot was used for build + scripts and procedural macros. +- **[cargo#7336]: Add `alloc` and `proc_macro` to libstd crates**, [alexcrichton], Sep 2019 + - Ensures `alloc` and `proc_macro` are also built when `std` is +- **[wg-cargo-std-aware#25]: Remove requirement for `--target`**, [ehuss], Sep 2019 + - `--target` must be specified to require Cargo run in cross-compilation mode (as + the `proc_macro` crate needs the host sysroot) + - Fixed in [cargo#14317] +- **[wg-cargo-std-aware#27]: Possibly publish a synthetic `Cargo.toml`**, [ehuss], Sep 2019 + - build-std initially had to create a false `Cargo.toml` for the standard library as it + was part of a workspace with the rest of rust-lang/rust + - Fixed by [cargo#14358] (after [rust#128534]) +- **[wg-cargo-std-aware#28]: Fixup some crate flags**, [ehuss], Sep 2019 + - Some crates require special compiler flags which are scattered throughout bootstrap + - e.g. `compiler_builtins` uses `debug-assertions=no`, `codegen-units=1`, `panic=abort` + - cc [wg-cargo-std-aware#15] + - There will always be some implicit contract between the standard library and build-std + - Move these to `Cargo.toml` wherever possible + - Progress made in [rust#64316] + - Bootstrap has been improved so there is less of this - target-specific and + sanitizer parts remain +- **[wg-cargo-std-aware#31]: Possibly add a way to disable the sysroot on + `rustc`**, [ehuss], Sep 2019 + - Implementation uses `--extern` to tell rustc where to find the standard library but it + will look in the sysroot for any crate that was not provided with an `--extern` + argument + - This can end up loading sysroot versions of the standard library and with + duplicate language item errors + - Was closed when build-std used `--sysroot` in [cargo#7421] and then re-opened when + it changed back to `--extern` in [cargo#7699] +- **[wg-cargo-std-aware#33]: Consider better testing strategy**, [ehuss], Sep 2019 + - build-std's testing in Cargo after the initial implementation wasn't very good and has + since been improved +- **[wg-cargo-std-aware#34]: Consider mixing `__CARGO_DEFAULT_LIB_METADATA` into the + hash**, [ehuss], Sep 2019 + - Environment variable is used for embedding the release channel into the metadata + hash but does not appear to be necessary for build-std +- **[wg-cargo-std-aware#35]: Consider not building std as a dylib**, [ehuss], Sep 2019 + - The standard library's manifest builds it as both a rlib and a dylib + - build-std only needs the rlib + - Don't want to produce the dylib and have anyone depend on it always being built + - Fixed by [cargo#7353] +- **[wg-cargo-std-aware#37]: Consider setting `require_optional_deps` to `false` for + standard library**, [ehuss], Sep 2019 + - Workspace created for the standard library sets `require_optional_deps` to `true` + but probably doesn't need to + - Fixed in [cargo#7337] +- **[cargo#7337]: Don't resolve std's optional dependencies**, [alexcrichton], Sep 2019 + - Unrequested optional dependencies are typically the `dev-dependencies` of + `std` and so don't need to be built + - Fixes [wg-cargo-std-aware#37] +- **[cargo#7350]: Improve test suite for -Zbuild-std**, [alexcrichton], Sep 2019 + - Many improvements to build-std testing, primarily the introduction of a + "mock-std" workspace which mimics the standard library's structure and + allows tests to run much quicker + - Fixes [wg-cargo-std-aware#33] +- **[rust#64316]: Delete most of `src/bootstrap/bin/rustc.rs`**, [alexcrichton], Sep 2019 + - Moved most of the standard library's configuration from the rustc shim in + bootstrap to the standard library using Cargo features, so it could be + leveraged by `-Zbuild-std` +- **[wg-cargo-std-aware#40]: Using `--extern` is apparently not equivalent to + `--sysroot`**, [alexcrichton], Sep 2019 + - Using `--extern alloc=alloc.rlib` meant that writing `extern crate alloc` was + not required, which is different than putting `alloc.rlib` in the sysroot where + `extern crate alloc` would still be required + - A consequence of this is that a locally compiled standard library would not + require `extern crate alloc` while the pre-compiled standard library would + - build-std switched to using the sysroot in [cargo#7421] and then switched back to + using `--extern` in [cargo#7699] (once `--extern noprelude:alloc=alloc.rlib` was + added) +- **[cargo#7421]: Change build-std to use --sysroot**, [ehuss], Sep 2019 + - The initial implementation used `--extern` to provide rustc the newly-built + standard library artifacts to later rustc invocations. This did not have + identical behaviour to the existing pre-built artefacts in the sysroot + ([wg-cargo-std-aware#40]) + - Negated the need to prevent rustc from using the sysroot + ([wg-cargo-std-aware#31]) + - Instead, this PR constructed a sysroot in Cargo's `target` directory and passed + that to rustc to replace the default sysroot + - It was found that this sysroot approach could still allow users to depend on + sysroot crates without declaring a dependency on + it([wg-cargo-std-aware#49]) +- **[wg-cargo-std-aware#41]: Documentation on how to use the new cargo-std support in + place of cargo xbuild**, [alex], Sep 2019 + - Fixed by adding documentation on build-std to Cargo's unstable feature documentation +- **[wg-cargo-std-aware#44]: Disable incremental for std crates**, [ehuss], Sep 2019 + - Incremental is not necessary for build-std + - Fixed in [cargo#8177] +- **[wg-cargo-std-aware#48]: Investigate custom libdir setting**, [ehuss], Sep 2019 + - Cargo expects a specific sysroot layout that can be changed with `bootstrap.toml` + - Intended to switch to `--print=target-libdir` from [rust#69608] + - Later made irrelevant by [cargo#7699] +- **[wg-cargo-std-aware#49]: Usage of `--sysroot` may still be racy and/or allow false + dependencies**, [alexcrichton], Oct 2019 + - Later made irrelevant by [cargo#7699] where `--extern` is used again +- **[cargo#7699]: Switch build-std to use --extern**, [ehuss], Dec 2019 + - A revert of [cargo#7421], but uses new `--extern` options `priv` and + `noprelude` from [rust#67074] + - Adding standard library crates to the extern prelude was the cause of + [wg-cargo-std-aware#40] + - Re-opened [wg-cargo-std-aware#31] but fixes [wg-cargo-std-aware#49] +- **[wg-cargo-std-aware#53]: `compiler_builtins` seems to be missing + symbols**, [tomaak], Jan 2020 + - `compiler_builtins` provides symbols through the `mem` feature + - It is not enabled by default as the standard library typically gets + these symbols through the platform `libc` + - `libc`'s implementation is typically better optimized + - Some `no_std` targets need compiler_builtins' implementations + - A concept of "target-specific features" could be beneficial + - Supporting custom targets makes this tricky + - `compiler-builtins-mem` feature is forwarded through standard library + crates to `compiler_builtins` + - [compiler-builtins#411] added weak linkage to the mem functions in compiler-builtins +- **[cargo#7931]: build-std: remove sysroot probe**, [ehuss], Feb 2020 + - An optimisation to remove an unnecessary `rustc --print` invocation +- **[wg-cargo-std-aware#54]: Adjust libstd to make non-Rust dependencies + optional**, [nagisa], Apr 2020 + - Cross-compiling the standard library's `C` dependencies is difficult + - The standard library now uses `gimli` not backtrace which makes this easier + - `libunwind` can be linked with `-Clink-self-contained` + - linux-musl and fortanix-sgx both may still require a C compiler + - Unclear if this can be avoided +- **[wg-cargo-std-aware#55]: Persistent unused attribute warnings when recompiling + libcore after linker flags are changed**, [cr1901], Apr 2020 + - Issue with incremental compilation, fixed by [cargo#8177] (cc [wg-cargo-std-aware#44]) +- **[cargo#8177]: build-std: Don't treat std like a "local" package**, [ehuss], Apr 2020 + - Adds the concept of a "local" package (not controlled by the user) + - `build-std` no longer uses incremental or dep-info fingerprint tracking and + will not show warnings in standard library crates + - Closed [wg-cargo-std-aware#44] and [wg-cargo-std-aware#55] +- **[wg-cargo-std-aware#62]: Linker can't find `core::panicking::panic` when lto is turned + on**, [hnj2], Nov 2020 + - `panic` can't be found with LTO turned on + - compiler-builtins wasn't being built with `overflow-checks=false` and + `debug-assertions=false` +- **[cargo#8490]: Add a `-Zbuild-std-features` flag**, [alexcrichton], Jul 2020 + - Allows users to enable Cargo features from the standard library +- **[rust#77086]: Include libunwind in the rust-src component**, [ehuss], Sep 2020 + - Includes `src/llvm-project/libunwind` in `rust-src` which is needed by some + targets, such as musl targets, to build the unwind crate +- **[cargo#8834]: Patch in vendored dependencies in `rust-src`**, [Gankra], November 2020 + - A step towards supporting `cargo vendor` ([wg-cargo-std-aware#23]) + - Allow for the `rust-src` component to include vendored dependencies of the standard + library + - Reverted in [cargo#8968] due to: + - [cargo#8962]: `-Zbuild-std` always updates the registry index + - [cargo#8963]: unused patch warnings when using + `-Zbuild-std-features=compiler-builtins-mem` + - [cargo#8945]: `-Zbuild-std` with custom targets was broken +- **[rust#78790]: Vendor libtest's dependencies in the rust-src component**, [Gankra], Nov 2020 + - [rust-lang/rust] half of [cargo#8834] + - Reverted in [rust#80082] as it caused `x.py dist` to always require + network access ([rust#79218]) +- **[wg-cargo-std-aware#63]: Support code-coverage**, [catenacyber], Dec 2020 + - Finding duplicate language item when building with `-Zinstrument-coverage` and build-std + - Works with `-Zno-profiler-runtime` + - Presumably profiler runtime is being loaded from the sysroot and that is loading other + sysroot crates and conflicting with the locally built crates +- **[wg-cargo-std-aware#64]: -Z build-std with unified workspace**, [Ericson2314], Jan 2021 + - In the current build-std implementation, the standard library is resolved + separately then combined with the user's crate graph + - It is argued that this is undesirable as it makes the standard library special + - Better to keep them separate as the standard library wants to have fixed dependency + versions matching the distributed version +- **[wg-cargo-std-aware#66]: Cross-compilation (of libc) on MacOS fails**, [raphaelcohn], Feb 2021 + - Unclear exactly what the root cause of this issue is +- **[wg-cargo-std-aware#67]: Update README.md**, [ghost], Mar 2021 + - Merged into [wg-cargo-std-aware] +- **[wg-cargo-std-aware#69]: "use of unstable library feature 'restricted_std'" can't be + fixed for deps**, [Manishearth], Jun 2021 + - User wants to use crates which use the standard library but where `restricted_std` applies + and that can't be fixed for the dependencies other than by patching +- **[wg-cargo-std-aware#70]: error: could not find native static library `c`, perhaps a `-L` + flag is missing?**, [mkb2091], Jun 2021 + - Duplicate of [wg-cargo-std-aware#66] +- **[wg-cargo-std-aware#72]: Figure out how to deal with `cargo test` with a + `no-std` target**, [phip1611], Jul 2021 + - `cargo test` doesn't work with `no_std` targets as `restricted_std` triggers + on `libtest` + - There were no comments on this issue +- **[cargo#10129]: Add support for `-Zbuild-std` to `cargo fetch`**, [jyn514], Nov 2021 + - Enables `cargo fetch -Zbuild-std` to fetch standard library crates +- **[wg-cargo-std-aware#76]: Unable to build executable for musl target**, [HenryJk], Nov 2021 + - It is unclear what fixed this issue, potentially a libc version bump + or linking `self-contained` +- **[cargo#10308]: Move build-std to Cargo.toml**, [fee1-dead], Jan 2022 + - Attempts to fix issues with build-std and per-package-target ([cargo#9451]) + - build-std's user interface is a large and open question that this patch + didn't have all the answers for so this was later closed + ([wg-cargo-std-aware#43]) +- **[cargo#10330]: Support per pkg target for -Zbuild-std**, [fee1-dead], Jan 2022 + - Another attempt to fix build-std and per-package-target ([cargo#9451]) + - Attempted to remove `--target` restriction but was told that this wasn't possible + - It probably was possible, thanks to various refactorings to Cargo between + 2019 and 2022, as [cargo#14317] faced no difficulties in removing the + restriction +- **[wg-cargo-std-aware#81]: -lunwind despite build-std=\["panic_abort", "std"\] on + powerpc-unknown-linux-musl**, [george-hopkins], Oct 2022 + - It is unclear what this issue is, there was very little detail and nobody commented +- **[rust#108924]: panic_immediate_abort requires abort as a panic strategy**, [tmiasko], Mar 2023 + - Adds a compile error when the `panic_immediate_abort` feature isn't used with `-Cpanic=abort` + - This could be triggered by build-std, as per [rust#107016] +- **[cargo#12088]: hack around `libsysroot` instead of `libtest`**, [weihanglo], May 2023 + - Cargo previously resolved features from the `test` crate, now it does so from + the `sysroot` crate, which is the canonical "head" of the standard library + - This approach to resolving features of standard library crates is still + considered a hack +- **[wg-cargo-std-aware#87]: The restricted_std error message is confusing**, [ehuss], May 2023 + - If you build a `no_std` target and forget to include `#![no_std]` then the standard library + is loaded and there's an "unstable library feature" error, this is confusing + - Improved error added in [rust#123360] and checking if a target supports the standard library + in [cargo#14183] +- **[cargo#13065]: fix: reorder `--remap-path-prefix` flags for `-Zbuild-std`**, [weihanglo], Nov 2023 + - Changing the order these flags are passed improves the source path in + diagnostics +- **[rust#120232]: Add support for JSON targets when using build-std**, [c272], Jan 2024 + - Updates the `restricted_std` filtering in `std`'s + [`build.rs`][std-build.rs] to check the target os rather than the target + triple (which is just set to the filename for JSON targets). + - Custom targets that "look similar" to builtin targets do not need to use + `restricted_std` ([wg-cargo-std-aware#90]) +- **[cargo#13404]: Verify build-std crate graph against library lock file**, [c272], Feb 2024 + - Added a new Cargo test, as an alternative to doing a forced lock + ([wg-cargo-std-aware#38]), to ensure that the resolved standard library unit + graph is a subset of the distributed `Cargo.lock` + - Closed in preference of a check at runtime +- **[rust#123360]: Document restricted_std**, [adamgemmell], Apr 2024 + - Improves the error message encountered when attempting to use `std` on a + target where `restricted_std` is set + - Proposed closing issues around `restricted_std` but a more comprehensive + solution was desired +- **[cargo#13916]: Verify build-std resolve against original lockfile**, [adamgemmell], May 2024 + - Same as [cargo#13404] but during Cargo execution + - Superseded by changes from [rust#128534] +- **[cargo#14183]: Check build target supports std when building with -Zbuild-std=std**, [harmou01], Jul 2024 + - Disallows building `std` when `metadata.std` field in the unstable + `target-spec-json` is `false` + - Aims to improve user experience compared to the `restricted_std` error + - `target-spec-json`'s `metadata.std` was added to support generation of the + compiler documentation, rather than as a source-of-truth for this + information +- **[cargo#14317]: Remove requirement for --target when invoking Cargo with -Zbuild-std**, [harmou01], Jul 2024 + - Cargo now defaults to "cross-compile" mode and unifies units when in + "host-only" mode, so host dependencies do not use the build-std standard + library and this restriction can be removed +- **[rust#128534]: Move the standard library to a separate workspace**, [bjorn3], Aug 2024 + - `rust-src` now has its own lockfile, enabling simplifications in build-std + implementation +- **[cargo#14358]: Remove hack on creating virtual std workspace**, [weihanglo], Aug 2024 + - Following [rust#128534], Cargo does not need to construct a synthetic + workspace and can load the workspace from disk + - Also enables `build-std` to use the configuration present in the workspace + manifest +- **[cargo#14370]: fix: std Cargo.lock moved to `library` dir**, [weihanglo], Aug 2024 + - Fix for [cargo#14358] which use the new `library/Cargo.lock` +- **[wg-cargo-std-aware#91]: rust-lld: undefined symbol: memchr when compiling + with nightly-aarch64-unknown-linux-musl**, [yogh333], Aug 2024 + - Undefined `memchr` symbol on `aarch64-unknown-linux-musl` when using `compiler-builtins-mem` + feature + - Suggested that the issue is a missing aarch64 musl libc +- **[cargo#14589]: Implement `--locked` for build-std**, [adamgemmell], Sep 2024 + - Alternative to [cargo#13916] + - Reuses Cargo's `--locked` machinery now that the standard library has its + own workspace and lockfile + - Concerns raised about resolving with optional dependencies and breaking the + future ability to patch the standard library workspace + - Closed pending a more comprehensive plan from the build-std project goal +- **[cargo#14850]: always link to std when testing proc-macros**, [weihanglo], Nov 2024 + - A small fix for testing proc-macros with build-std when `libstd.so` stopped + being shipped +- **[cargo#14899]: determine root crates by target spec `std:bool`**, [weihanglo], Dec 2024 + - This removes the hard error from [cargo#14183] and instead uses `std` as the + default crate for build-std if `metadata.std` is true, and + `core`/`compiler_builtins` otherwise + - `std` can be built on some targets even if they don't officially support + std, and rustdoc was relying on this behaviour +- **[cargo#14938]: make Resolve align to what to build**, [weihanglo], Dec 2024 + - Reverted part of [cargo#14899] which meant that `panic_unwind` would not + build if the `panic-unwind` feature was not present +- **[cargo#14951]: Do not hash absolute sysroot path into stdlib crates metadata**, [Dirbaio], Dec 2024 + - Improves reproducibility of `build-std` builds by only hashing paths of + standard library sources relative to the sysroot +- **[cargo#15065]: parse as comma-separated list**, [weihanglo], Jan 2025 + - Fixes a minor regression when providing multiple crates via + `CARGO_UNSTABLE_BUILD_STD` +- **[rust#135395]: Enforce the compiler-builtins partitioning scheme**, [saethlin], Jan 2025 (closed) + - Removes the profile override for `compiler_builtins`' codegen-units and + implements it in the compiler instead + - One of the use cases is build-std, which does not use the profile override + - Closed due to build issues with Rust for Linux unrelated to build-std +- **[wg-cargo-std-aware#93]: Stack trace for duplicate lang item?**, [illuzen], Feb 2025 + - Missing `panic_abort` and so loading it from the sysroot and hitting a duplicate language + item error + - Downstream of [wg-cargo-std-aware#31] +- **[wg-cargo-std-aware#94]: `panic_immediate_abort` and `no_std`**, [nazar-pc], Mar 2025 + - Suggests that a panic handler crate shouldn't be necessary when + `panic_immediate_abort` is enabled + +### Bugs in the compiler or standard library +[bugs-in-the-compiler-or-standard-library]: #bugs-in-the-compiler-or-standard-library + +These issues were bug reports for build-std that ultimately ended up being +issues resolved in the standard library or compiler: + +- **[wg-cargo-std-aware#32]: Figure out why profile override causes linker + errors**, [ehuss], Sep 2019 + - Ended up being a bug in symbol mangling ([rust#64319]) +- **[wg-cargo-std-aware#52]: cannot produce proc-macro on musl host toolchain**, + [12101111], Nov 2019 + - Issue was entirely unrelated to build-std +- **[wg-cargo-std-aware#56]: duplicate item in crate `core`**, [chaozju], Jun + 2020 + - User's dependency was using `std` (forgot to disable `std` feature) +- **[wg-cargo-std-aware#58]: It is not possible to use `-Zbuild-std` with a + Development-Channel Rust**, [cr1901], Aug 2020 + - User's toolchain version did not match local checkout of [rust-lang/rust] +- **[wg-cargo-std-aware#59]: Can't build executables for musl**, [vi], Sep 2020 + - `libunwind`'s source was missing in `rust-src` for targets that need it + - Fixed in [rust#77086] +- **[wg-cargo-std-aware#60]: Can't build std if I specify target json file: std + does not see networking**, [vi], Sep 2020 + - The standard library's `build.rs` was matching on entire target names rather + than just components like `target_os` + - Fixed in [rust#120232] +- **[wg-cargo-std-aware#71]: "duplicate lang item in crate `core`" when + building**, [TheBlueMatt], Jul 2021 + - Duplicate of [wg-cargo-std-aware#56] +- **[wg-cargo-std-aware#73]: Build on Windows fails to select a version of + `libc` for package `test`**, [MauriceKayser], Oct 2021 + - Duplicate of [cargo#9976], ultimately unrelated to build-std +- **[wg-cargo-std-aware#74]: undefined reference errors on aarch64**, + [SparrowLii], Nov 2021 + - `core`'s implementation briefly depended on libc after [rust#83655] + - `compiler_builtins` had no implementation of the symbols that the + `outline-atomics` feature was using from `libc` + - `compiler_builtins` gained implementation in [compiler-builtins#532] +- **[wg-cargo-std-aware#75]: Code won't compile with panic="abort" option**, + [HenryJk], Nov 2021 + - User was missing `panic_abort` crate in `-Zbuild-std=` +- **[wg-cargo-std-aware#78]: Rust compiler workspace patches are being ignored + when compiling with `-Zbuild-std`**, [raoulstrackx], Nov 2021 + - Adding a `[patch]` for the standard library to the workspace in the + [rust-lang/rust] did not work + - Fixed when the standard library gained its own workspace ([rust#128534]) +- **[wg-cargo-std-aware#79]: Building std with support for certain lang items**, + [AZMCode], Jan 2022 + - User wanted to define their own language item for `Error` in a `no_std` + project + - Ultimately `Error` was moved to `core` +- **[wg-cargo-std-aware#80]: duplicate lang item if `#![feature(test)]` is + enabled**, [skyzh], Mar 2022 + - `std` wasn't in the list of crates to `-Zbuild-std` and was being pulled from sysroot, + so `core` was being loaded twice +- **[wg-cargo-std-aware#82]: Hidden symbol isn't defined**, [wcampbell0x2a], Jan + 2023 + - Duplicate of [rust#107016], fixed in [rust#108924] + +### Cargo feature requests narrowly applied to build-std +[cargo-feature-requests-narrowly-applied-to-build-std]: #cargo-feature-requests-narrowly-applied-to-build-std + +These issues were feature requests for build-std that could have been a more +general feature for Cargo that could then apply to build-std too: + +- **[wg-cargo-std-aware#84]: Cache libstd artifacts between projects**, + [jyn514], May 2023 + - Cargo rebuilds the standard library for each project used with `-Zbuild-std` + (in the `target` directory) but it could be cached globally + - It was the opinion of team members that this was hard to implement + - It also could apply to any dependency of specific version and configuration + and so this could be resolved by proposing the addition of a global caching + mechanism for dependencies +- **[wg-cargo-std-aware#89]: Allow scoping of unstable features to specific + targets**, [ketsuban], Oct 2023 + - User wants unstable `build-std` to only be enabled for one target and + nightly not be required when building for other targets + - This is a consequence of how Cargo's unstable features work, and would be + fixed by a change to that mechanism, rather than anything specific to + build-std + +## Other issues +[a1-other-issues]: #other-issues + +There are a handful of other issues that are not directly referencing build-std, +but which affect related features in the toolchain, such as `no_std` crates: + +- **[keyword nostd vs no_std vs category no-std][crates.io#7306]**, [gnzlbg], Feb 2018 + - There is `no-std` category as well as a `nostd` and `no_std` keywords used + by crates. Not every crate which uses the `nostd`/`no_std` keywords also + uses the `no-std` category, and vice versa, making it harder to list all + `no_std`-supporting crates with a single search. +- **[no_std crates should not permit non-no_std dependencies][cargo#8798]**, [zesterer], Oct 2020 + - Cargo permits `no_std` crates to depend on crates that use the standard + library. When working with a target that does not ship the `std` crate, the + presence of these crates in the dependency graph can result in unexpected + build failures. + +[history]: ./2-history.md + +[JOSH]: https://josh-project.github.io/josh/intro.html +[cargo-xbuild]: https://github.com/rust-osdev/cargo-xbuild +[embedded-wg]: https://github.com/rust-embedded/wg +[portability-wg]: https://github.com/rust-lang-nursery/portability-wg +[rust-lang/cargo]: https://github.com/rust-lang/cargo +[rust-lang/rust]: https://github.com/rust-lang/rust +[wg-cargo-std-aware]: https://github.com/rust-lang/wg-cargo-std-aware +[xargo]: https://github.com/japaric/xargo + +[A vision for portability in Rust]: http://aturon.github.io/tech/2018/02/06/portability-vision/ + +[cargo#10129]: https://github.com/rust-lang/cargo/pull/10129 +[cargo#10308]: https://github.com/rust-lang/cargo/pull/10308 +[cargo#10330]: https://github.com/rust-lang/cargo/pull/10330 +[cargo#10881]: https://github.com/rust-lang/cargo/issues/10881 +[cargo#12088]: https://github.com/rust-lang/cargo/pull/12088 +[cargo#12375]: https://github.com/rust-lang/cargo/pull/12375 +[cargo#13065]: https://github.com/rust-lang/cargo/pull/13065 +[cargo#13404]: https://github.com/rust-lang/cargo/pull/13404 +[cargo#13916]: https://github.com/rust-lang/cargo/pull/13916 +[cargo#14183]: https://github.com/rust-lang/cargo/pull/14183 +[cargo#14317]: https://github.com/rust-lang/cargo/pull/14317 +[cargo#14358]: https://github.com/rust-lang/cargo/pull/14358 +[cargo#14370]: https://github.com/rust-lang/cargo/pull/14370 +[cargo#14589]: https://github.com/rust-lang/cargo/pull/14589 +[cargo#14850]: https://github.com/rust-lang/cargo/pull/14850 +[cargo#14899]: https://github.com/rust-lang/cargo/pull/14899 +[cargo#14938]: https://github.com/rust-lang/cargo/pull/14938 +[cargo#14951]: https://github.com/rust-lang/cargo/pull/14951 +[cargo#15065]: https://github.com/rust-lang/cargo/pull/15065 +[cargo#2768]: https://github.com/rust-lang/cargo/pull/2768 +[cargo#4959]: https://github.com/rust-lang/cargo/issues/4959 +[cargo#5002]: https://github.com/rust-lang/cargo/issues/5002 +[cargo#5003]: https://github.com/rust-lang/cargo/issues/5003 +[cargo#7216]: https://github.com/rust-lang/cargo/pull/7216 +[cargo#7336]: https://github.com/rust-lang/cargo/pull/7336 +[cargo#7337]: https://github.com/rust-lang/cargo/pull/7337 +[cargo#7350]: https://github.com/rust-lang/cargo/pull/7350 +[cargo#7353]: https://github.com/rust-lang/cargo/pull/7353 +[cargo#7421]: https://github.com/rust-lang/cargo/pull/7421 +[cargo#7614]: https://github.com/rust-lang/cargo/issues/7614 +[cargo#7699]: https://github.com/rust-lang/cargo/pull/7699 +[cargo#7931]: https://github.com/rust-lang/cargo/pull/7931 +[cargo#8177]: https://github.com/rust-lang/cargo/pull/8177 +[cargo#8490]: https://github.com/rust-lang/cargo/pull/8490 +[cargo#8733]: https://github.com/rust-lang/cargo/issues/8733 +[cargo#8798]: https://github.com/rust-lang/cargo/issues/8798 +[cargo#8834]: https://github.com/rust-lang/cargo/pull/8834 +[cargo#8945]: https://github.com/rust-lang/cargo/issues/8945 +[cargo#8962]: https://github.com/rust-lang/cargo/issues/8962 +[cargo#8963]: https://github.com/rust-lang/cargo/issues/8963 +[cargo#8968]: https://github.com/rust-lang/cargo/pull/8968 +[cargo#9451]: https://github.com/rust-lang/cargo/issues/9451 +[cargo#9976]: https://github.com/rust-lang/cargo/issues/9976 +[compiler-builtins#411]: https://github.com/rust-lang/compiler-builtins/pull/411 +[compiler-builtins#532]: https://github.com/rust-lang/compiler-builtins/pull/532 +[crates.io#7306]: https://github.com/rust-lang/crates.io/pull/7306 +[internals.r-l.o: Fleshing out libstd scenarios]: https://internals.rust-lang.org/t/fleshing-out-libstd-scenarios/4206 +[internals.r-l.o: Refactoring libstd for ultimate portability]: https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301 +[jamesmunns/rfcs#1]: https://github.com/jamesmunns/rfcs/pull/1 +[rfcs#1133]: https://github.com/rust-lang/rfcs/pull/1133 +[rfcs#1502]: https://github.com/rust-lang/rfcs/pull/1502 +[rfcs#1868]: https://github.com/rust-lang/rfcs/pull/1868 +[rfcs#2663-t-lang]: https://github.com/rust-lang/lang-team/blob/master/minutes/2019-06-06.md?rgh-link-date=2019-06-06T23%3A20%3A17Z- +[rfcs#2663]: https://github.com/rust-lang/rfcs/pull/2663 +[rust#107016]: https://github.com/rust-lang/rust/issues/107016 +[rust#108924]: https://github.com/rust-lang/rust/pull/108924 +[rust#119899]: https://github.com/rust-lang/rust/pull/119899 +[rust#120232]: https://github.com/rust-lang/rust/pull/120232 +[rust#123360]: https://github.com/rust-lang/rust/pull/123360 +[rust#128534]: https://github.com/rust-lang/rust/pull/128534 +[rust#135395]: https://github.com/rust-lang/rust/pull/135395 +[rust#31605]: https://github.com/rust-lang/rust/pull/31605 +[rust#46439]: https://github.com/rust-lang/rust/pull/46439 +[rust#57288]: https://github.com/rust-lang/rust/issues/57288 +[rust#64158]: https://github.com/rust-lang/rust/pull/64158 +[rust#64316]: https://github.com/rust-lang/rust/pull/64316 +[rust#64319]: https://github.com/rust-lang/rust/issues/64319 +[rust#67074]: https://github.com/rust-lang/rust/issues/67074 +[rust#67650]: https://github.com/rust-lang/rust/issues/67650 +[rust#68887]: https://github.com/rust-lang/rust/issues/68887 +[rust#69608]: https://github.com/rust-lang/rust/pull/69608 +[rust#71009]: https://github.com/rust-lang/rust/pull/71009 +[rust#77086]: https://github.com/rust-lang/rust/pull/77086 +[rust#78790]: https://github.com/rust-lang/rust/pull/78790 +[rust#79218]: https://github.com/rust-lang/rust/pull/79218 +[rust#79838]: https://github.com/rust-lang/rust/pull/79838 +[rust#79958]: https://github.com/rust-lang/rust/pull/79958 +[rust#80082]: https://github.com/rust-lang/rust/pull/83655 +[rust#83655]: https://github.com/rust-lang/rust/pull/83655 +[wg-cargo-std-aware#11]: https://github.com/rust-lang/wg-cargo-std-aware/issues/11 +[wg-cargo-std-aware#12]: https://github.com/rust-lang/wg-cargo-std-aware/issues/12 +[wg-cargo-std-aware#13]: https://github.com/rust-lang/wg-cargo-std-aware/issues/13 +[wg-cargo-std-aware#14-review]: https://github.com/rust-lang/wg-cargo-std-aware/issues/14#issuecomment-2315878717 +[wg-cargo-std-aware#14]: https://github.com/rust-lang/wg-cargo-std-aware/issues/14 +[wg-cargo-std-aware#15]: https://github.com/rust-lang/wg-cargo-std-aware/issues/15 +[wg-cargo-std-aware#16]: https://github.com/rust-lang/wg-cargo-std-aware/issues/16 +[wg-cargo-std-aware#17]: https://github.com/rust-lang/wg-cargo-std-aware/issues/17 +[wg-cargo-std-aware#18]: https://github.com/rust-lang/wg-cargo-std-aware/issues/18 +[wg-cargo-std-aware#19-internals]: https://internals.rust-lang.org/t/dogfooding-z-build-std-in-rustbuild/13775/22 +[wg-cargo-std-aware#19]: https://github.com/rust-lang/wg-cargo-std-aware/issues/19 +[wg-cargo-std-aware#20]: https://github.com/rust-lang/wg-cargo-std-aware/issues/20 +[wg-cargo-std-aware#21]: https://github.com/rust-lang/wg-cargo-std-aware/issues/21 +[wg-cargo-std-aware#22]: https://github.com/rust-lang/wg-cargo-std-aware/issues/22 +[wg-cargo-std-aware#23]: https://github.com/rust-lang/wg-cargo-std-aware/issues/23 +[wg-cargo-std-aware#24]: https://github.com/rust-lang/wg-cargo-std-aware/issues/24 +[wg-cargo-std-aware#25]: https://github.com/rust-lang/wg-cargo-std-aware/issues/25 +[wg-cargo-std-aware#26]: https://github.com/rust-lang/wg-cargo-std-aware/issues/26 +[wg-cargo-std-aware#27]: https://github.com/rust-lang/wg-cargo-std-aware/issues/27 +[wg-cargo-std-aware#28]: https://github.com/rust-lang/wg-cargo-std-aware/issues/28 +[wg-cargo-std-aware#29]: https://github.com/rust-lang/wg-cargo-std-aware/issues/29 +[wg-cargo-std-aware#2]: https://github.com/rust-lang/wg-cargo-std-aware/issues/2 +[wg-cargo-std-aware#30]: https://github.com/rust-lang/wg-cargo-std-aware/issues/30 +[wg-cargo-std-aware#31]: https://github.com/rust-lang/wg-cargo-std-aware/issues/31 +[wg-cargo-std-aware#32]: https://github.com/rust-lang/wg-cargo-std-aware/issues/32 +[wg-cargo-std-aware#33]: https://github.com/rust-lang/wg-cargo-std-aware/issues/33 +[wg-cargo-std-aware#34]: https://github.com/rust-lang/wg-cargo-std-aware/issues/34 +[wg-cargo-std-aware#35]: https://github.com/rust-lang/wg-cargo-std-aware/issues/35 +[wg-cargo-std-aware#36]: https://github.com/rust-lang/wg-cargo-std-aware/issues/36 +[wg-cargo-std-aware#37]: https://github.com/rust-lang/wg-cargo-std-aware/issues/37 +[wg-cargo-std-aware#38]: https://github.com/rust-lang/wg-cargo-std-aware/issues/38 +[wg-cargo-std-aware#39]: https://github.com/rust-lang/wg-cargo-std-aware/issues/39 +[wg-cargo-std-aware#3]: https://github.com/rust-lang/wg-cargo-std-aware/issues/3 +[wg-cargo-std-aware#40]: https://github.com/rust-lang/wg-cargo-std-aware/issues/40 +[wg-cargo-std-aware#41]: https://github.com/rust-lang/wg-cargo-std-aware/issues/41 +[wg-cargo-std-aware#42]: https://github.com/rust-lang/wg-cargo-std-aware/issues/42 +[wg-cargo-std-aware#43-internals]: https://internals.rust-lang.org/t/build-std-and-the-standard-library/11459 +[wg-cargo-std-aware#43]: https://github.com/rust-lang/wg-cargo-std-aware/issues/43 +[wg-cargo-std-aware#44]: https://github.com/rust-lang/wg-cargo-std-aware/issues/44 +[wg-cargo-std-aware#45]: https://github.com/rust-lang/wg-cargo-std-aware/issues/45 +[wg-cargo-std-aware#46]: https://github.com/rust-lang/wg-cargo-std-aware/issues/46 +[wg-cargo-std-aware#47]: https://github.com/rust-lang/wg-cargo-std-aware/issues/47 +[wg-cargo-std-aware#48]: https://github.com/rust-lang/wg-cargo-std-aware/issues/48 +[wg-cargo-std-aware#49]: https://github.com/rust-lang/wg-cargo-std-aware/issues/49 +[wg-cargo-std-aware#4]: https://github.com/rust-lang/wg-cargo-std-aware/issues/4 +[wg-cargo-std-aware#5-internals]: https://internals.rust-lang.org/t/pre-pre-rfc-making-std-dependent-cargo-features-a-first-class-concept/10828 +[wg-cargo-std-aware#50]: https://github.com/rust-lang/wg-cargo-std-aware/issues/50 +[wg-cargo-std-aware#51]: https://github.com/rust-lang/wg-cargo-std-aware/issues/51 +[wg-cargo-std-aware#52]: https://github.com/rust-lang/wg-cargo-std-aware/issues/52 +[wg-cargo-std-aware#53]: https://github.com/rust-lang/wg-cargo-std-aware/issues/53 +[wg-cargo-std-aware#54]: https://github.com/rust-lang/wg-cargo-std-aware/issues/54 +[wg-cargo-std-aware#55]: https://github.com/rust-lang/wg-cargo-std-aware/issues/55 +[wg-cargo-std-aware#56]: https://github.com/rust-lang/wg-cargo-std-aware/issues/56 +[wg-cargo-std-aware#57]: https://github.com/rust-lang/wg-cargo-std-aware/issues/57 +[wg-cargo-std-aware#58]: https://github.com/rust-lang/wg-cargo-std-aware/issues/58 +[wg-cargo-std-aware#59]: https://github.com/rust-lang/wg-cargo-std-aware/issues/59 +[wg-cargo-std-aware#5]: https://github.com/rust-lang/wg-cargo-std-aware/issues/5 +[wg-cargo-std-aware#60]: https://github.com/rust-lang/wg-cargo-std-aware/issues/60 +[wg-cargo-std-aware#61]: https://github.com/rust-lang/wg-cargo-std-aware/issues/61 +[wg-cargo-std-aware#62]: https://github.com/rust-lang/wg-cargo-std-aware/issues/62 +[wg-cargo-std-aware#63]: https://github.com/rust-lang/wg-cargo-std-aware/issues/63 +[wg-cargo-std-aware#64]: https://github.com/rust-lang/wg-cargo-std-aware/issues/64 +[wg-cargo-std-aware#65]: https://github.com/rust-lang/wg-cargo-std-aware/issues/65 +[wg-cargo-std-aware#66]: https://github.com/rust-lang/wg-cargo-std-aware/issues/66 +[wg-cargo-std-aware#67]: https://github.com/rust-lang/wg-cargo-std-aware/issues/67 +[wg-cargo-std-aware#68]: https://github.com/rust-lang/wg-cargo-std-aware/issues/68 +[wg-cargo-std-aware#69]: https://github.com/rust-lang/wg-cargo-std-aware/issues/69 +[wg-cargo-std-aware#6]: https://github.com/rust-lang/wg-cargo-std-aware/issues/6 +[wg-cargo-std-aware#70]: https://github.com/rust-lang/wg-cargo-std-aware/issues/70 +[wg-cargo-std-aware#71]: https://github.com/rust-lang/wg-cargo-std-aware/issues/71 +[wg-cargo-std-aware#72]: https://github.com/rust-lang/wg-cargo-std-aware/issues/72 +[wg-cargo-std-aware#73]: https://github.com/rust-lang/wg-cargo-std-aware/issues/73 +[wg-cargo-std-aware#74]: https://github.com/rust-lang/wg-cargo-std-aware/issues/74 +[wg-cargo-std-aware#75]: https://github.com/rust-lang/wg-cargo-std-aware/issues/75 +[wg-cargo-std-aware#76]: https://github.com/rust-lang/wg-cargo-std-aware/issues/76 +[wg-cargo-std-aware#77]: https://github.com/rust-lang/wg-cargo-std-aware/issues/77 +[wg-cargo-std-aware#78]: https://github.com/rust-lang/wg-cargo-std-aware/issues/78 +[wg-cargo-std-aware#79]: https://github.com/rust-lang/wg-cargo-std-aware/issues/79 +[wg-cargo-std-aware#7]: https://github.com/rust-lang/wg-cargo-std-aware/issues/7 +[wg-cargo-std-aware#80]: https://github.com/rust-lang/wg-cargo-std-aware/issues/80 +[wg-cargo-std-aware#81]: https://github.com/rust-lang/wg-cargo-std-aware/issues/81 +[wg-cargo-std-aware#82]: https://github.com/rust-lang/wg-cargo-std-aware/issues/82 +[wg-cargo-std-aware#83]: https://github.com/rust-lang/wg-cargo-std-aware/issues/83 +[wg-cargo-std-aware#84]: https://github.com/rust-lang/wg-cargo-std-aware/issues/84 +[wg-cargo-std-aware#85]: https://github.com/rust-lang/wg-cargo-std-aware/issues/85 +[wg-cargo-std-aware#86]: https://github.com/rust-lang/wg-cargo-std-aware/issues/86 +[wg-cargo-std-aware#87]: https://github.com/rust-lang/wg-cargo-std-aware/issues/87 +[wg-cargo-std-aware#88]: https://github.com/rust-lang/wg-cargo-std-aware/issues/88 +[wg-cargo-std-aware#89]: https://github.com/rust-lang/wg-cargo-std-aware/issues/89 +[wg-cargo-std-aware#8]: https://github.com/rust-lang/wg-cargo-std-aware/issues/8 +[wg-cargo-std-aware#90]: https://github.com/rust-lang/wg-cargo-std-aware/issues/90 +[wg-cargo-std-aware#91]: https://github.com/rust-lang/wg-cargo-std-aware/issues/91 +[wg-cargo-std-aware#92]: https://github.com/rust-lang/wg-cargo-std-aware/issues/92 +[wg-cargo-std-aware#93]: https://github.com/rust-lang/wg-cargo-std-aware/issues/93 +[wg-cargo-std-aware#94]: https://github.com/rust-lang/wg-cargo-std-aware/issues/94 +[wg-cargo-std-aware#95]: https://github.com/rust-lang/wg-cargo-std-aware/issues/95 + +[embed-rs-cargo-toml]: https://github.com/embed-rs/stm32f7-discovery/blob/e2bf713263791c028c2a897f2eb1830d7f09eceb/Cargo.toml#L21 +[embed-rs-source]: https://github.com/embed-rs/stm32f7-discovery/blob/e2bf713263791c028c2a897f2eb1830d7f09eceb/core/src/lib.rs#L7 +[std-build.rs]: https://github.com/rust-lang/rust/blob/f315e6145802e091ff9fceab6db627a4b4ec2b86/library/std/build.rs#L17 + +[12101111]: https://github.com/12101111 +[AZMCode]: https://github.com/AZMCode +[Dirbaio]: https://github.com/Dirbaio +[Ericson2314]: https://github.com/Ericson2314 +[Gankra]: https://github.com/Gankra +[HenryJk]: https://github.com/HenryJk +[Manishearth]: https://github.com/Manishearth +[Mark-Simulacrum]: https://github.com/Mark-Simulacrum +[MauriceKayser]: https://github.com/MauriceKayser +[SparrowLii]: https://github.com/SparrowLii +[TheBlueMatt]: https://github.com/TheBlueMatt +[adamgemmell]: https://github.com/adamgemmell +[alex]: https://github.com/alex +[alexcrichton]: https://github.com/alexcrichton +[bascule]: https://github.com/bascule +[bjorn3]: https://github.com/bjorn3 +[c272]: https://github.com/c272 +[catenacyber]: https://github.com/catenacyber +[chaozju]: https://github.com/chaozju +[cr1901]: https://github.com/cr1901 +[dullbananas]: https://github.com/dullbananas +[ehuss]: https://github.com/ehuss +[errantmind]: https://github.com/errantmind +[fee1-dead]: https://github.com/fee1-dead +[george-hopkins]: https://github.com/george-hopkins +[ghost]: https://github.com/ghost +[gnzlbg]: https://github.com/gnzlbg +[harmou01]: https://github.com/harmou01 +[hnj2]: https://github.com/hnj2 +[illuzen]: https://github.com/illuzen +[jamesmunns]: https://github.com/jamesmunns +[japaric]: https://github.com/japaric +[jdm]: https://github.com/jdm +[jschwe]: https://github.com/jschwe +[jyn514]: https://github.com/jyn514 +[ketsuban]: https://github.com/ketsuban +[madsmtm]: https://github.com/madsmtm +[mkb2091]: https://github.com/mkb2091 +[nagisa]: https://github.com/nagisa +[nazar-pc]: https://github.com/nazar-pc +[parraman]: https://github.com/parraman +[phip1611]: https://github.com/phip1611 +[raoulstrackx]: https://github.com/raoulstrackx +[raphaelcohn]: https://github.com/raphaelcohn +[rust-osdev]: https://github.com/rust-osdev +[saethlin]: https://github.com/saethlin +[skyzh]: https://github.com/skyzh +[tamird]: https://github.com/tamird +[tmiasko]: https://github.com/tmiasko +[tomaak]: https://github.com/tomaak +[vi]: https://github.com/vi +[wcampbell0x2a]: https://github.com/wcampbell0x2a +[weihanglo]: https://github.com/weihanglo +[yogh333]: https://github.com/yogh333 +[zesterer]: https://github.com/zesterer \ No newline at end of file diff --git a/text/0000a-build-std-compatible.md b/text/0000a-build-std-compatible.md new file mode 100644 index 00000000000..d08154d78cd --- /dev/null +++ b/text/0000a-build-std-compatible.md @@ -0,0 +1,188 @@ +- Feature Name: `build-std-compatible` +- Start Date: 2025-06-05 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + + + +# Summary +[summary]: #summary + +This RFC proposes extending the `build-std` option with a new `compatible` +value, which will become the default and automatically rebuild the standard +library when it is necessary to maintain compatibility with the compiler flags +used by the rest of the crate graph. + +# Motivation +[motivation]: #motivation + +This RFC aimed at unblocking the stabilisation of ABI-modifying compiler flags +as per the +[motivations described in the build-std RFC](./0000-build-std/3-motivation.md). + +# Explanation +[explanation]: #explanation + +The `build-std` option in the Cargo configuration will be extended with a new +value: + +```toml +[build] +build-std = "compatible" # or `always`/`never` +``` + +"compatible" will become the default value for `build-std` ([?][rationale-default]). + +When `build-std` is set to "compatible", then the standard library crates will +be rebuilt automatically ([?][rationale-why-automatic]) when a pre-built +standard library is not present or the pre-built standard library is +incompatible with the rest of the crate (due to use of target modifiers). + +Documentation for target modifiers will be updated to reflect that changing a +target modifier will trigger a rebuild of the standard library. + +> [!NOTE] +> +> rustc will be extended with the ability to determine whether an existing +> `rlib` artefact is compatible with the flags passed to rustc +> ([?][rationale-rustc-support]). +> +> The exact mechanism is not specified in this RFC. It may be a flag such as +> `--compatibility-with=core` or an `--emit` or `--print` flag. +> +> Cargo can use this mechanism to determine whether it needs to rebuild the +> standard library to ensure compatibility. Cargo will only need to run rustc +> once to determine this, comparing the flags from the user's profile against +> one of the rlibs in the pre-built standard library (assuming that they are all +> compatible with each other). + +The standard library will be rebuilt in its release profile and will only vary +in the target modifier flags necessarily for it to be compatible +([?][rationale-release-profile]). + +Pre-built available? | User's profile | Target modifiers changed? | Standard library re-built? +-------------------- | -------------- | ------------------------- | -------------------------- +No | N/A | N/A | Yes, std's `release` (by necessity) +Yes | `dev` | Unchanged | No, use pre-built +Yes | `release` | Unchanged | No, use pre-built +Yes | `release` | Changed | Yes, std's `release`, with the addition of the target modifier flag + +> [!IMPORTANT] +> +> It is assumed that enabling a target modifier for a Cargo project will happen +> by setting an options in a Cargo profiles. This RFC does not propose adding +> any specific target modifier flags to Cargo's profiles, that can happen later +> on a flag-by-flag basis, as is typical, it only assumes that this is the +> mechanism that will eventually be used. As such, there is no row in the above +> table for "target modifiers changed, profile unchanged". + +As with the "always" option, the exact crates from the standard library to be +built are determined by the `build-std-crates` option or explicit dependencies on +the standard library if [*Standard library dependencies*][deps] are implemented. + +Multi-target projects (resulting from multiple `--target` flags, the "target" +field in artifact dependencies or the use of `per-pkg-target` fields) results in +the decision to rebuild the standard library being made multiple times - once +for each target in the project. + +As Cargo does not have per-target profiles nor a way to change the standard +library's profile on a per-target basis, the only way to configure the standard +library differently for different targets is with the use of the `[target]` +sections in the Cargo config. + +The default value of all target modifier flags must be the same as their default +values as exposed in Cargo and as used with the pre-built standard library, to +avoid spurious rebuilds. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This section aims to justify all of the decisions made in the proposed design +from [*Explanation*][explanation] and discuss why alternatives were not chosen. + +## Why default to "compatible"? +[rationale-default]: #why-default-to-compatible + +"compatible" will only trigger when it is necessary to ensure that a user's +current configuration will build successfully. It shouldn't trigger rebuilds for +the user or change the standard library they are using if it isn't strictly +required. + +If the default were not "compatible" then when the user enabled a target +modifier through Cargo (when such options are exposed), the user would +immediately face a compilation error and need to go learn about +`build-std = "compatible"` anyway. + +↩ [*Explanation*][explanation] + +## Why rebuild the standard library automatically? +[rationale-why-automatic]: #why-rebuild-the-standard-library-automatically + +Rebuilds of the standard library happening transparently reduce the requirement +that users learn about build-std as something to enable and configure. Combined +with explicit dependencies on the standard library crates from +[*Standard library dependencies*][deps] or `build-std-crates` from +[`build-std="always"`][always], build-std can avoid any cost on users that do +not require it (by triggering automatically when a target modifier is changed, +and having no unnecessary rebuilds otherwise). + +↩ [*Explanation*][explanation] + +## Why add compatibility checks in rustc to support build-std? +[rationale-rustc-support]: #why-add-compatibility-checks-in-rustc-to-support-build-std + +Without support from rustc, Cargo would need to assume that the pre-built +standard library's configuration is entirely configured in its Cargo profile and +would need to compare the profile of the standard library with the profile of +the user's crate and know which rustc flags are target modifiers. + +Furthermore, Cargo does not need to know which flags are target modifiers with +this mechanism, it can just pass all the flags it would normally pass to rustc +(incl. from `RUSTFLAGS`) and be notified of whether a rebuild is required. + +↩ [*Explanation*][explanation] + +## Why build in release profile? +[rationale-release-profile]: #why-build-in-release-profile + +As in [*"Why does "always" rebuild in release profile?"*][always-why-release], +building in the release profile minimises the differences between a newly-built +std and pre-built std, keeping the implications of this RFC small. + +↩ [*Explanation*][explanation] + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +The following small details are likely to be bikeshed prior to RFC acceptance or +stabilisation and aren't pertinent to the overall design: + +## What should the "compatible" value of `build-std` be named? +[unresolved-compatible]: #what-should-the-compatible-value-of-build-std-be-named + +It could be named "target-modifiers" or "automatic". + +↩ [*Explanation*][explanation] + +## What should the interface to rustc compatibility checking be? +[unresolved-rustc-compat-interface]: #what-should-the-interface-to-rustc-compatibility-checking-be + +Should it be an `--emit` flag? A `--print` flag? + +↩ [*Explanation*][explanation] + +# Prior art +[prior-art]: #prior-art + +See the [*Background*][background] and [*History*][history] of the build-std RFC. + +# Future possibilities +[future-possibilities]: #future-possibilities + +There are not currently any documented follow-ups to this RFC. + +[background]: ./0000-build-std/1-background.md +[history]: ./0000-build-std/2-history.md +[always]: ./0000-build-std/4-build-std-always.md#proposal +[always-why-release]: ./0000-build-std/4-build-std-always.md#why-does-always-rebuild-in-release-profile +[deps]: ./0000-build-std/5-standard-library-dependencies.md#proposal \ No newline at end of file diff --git a/text/0000b-build-std-match-profile.md b/text/0000b-build-std-match-profile.md new file mode 100644 index 00000000000..344bcbf327a --- /dev/null +++ b/text/0000b-build-std-match-profile.md @@ -0,0 +1,214 @@ +- Feature Name: `build-std-match-profile` +- Start Date: 2025-06-05 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + + + +# Summary +[summary]: #summary + +This RFC proposes extending the `build-std` option with new values which +automatically rebuild the standard library to match the user's current profile. + +# Motivation +[motivation]: #motivation + +This RFC aimed at allowing users to rebuild the standard library with different +codegen flags or profile as per the +[motivations described in the build-std RFC](./0000-build-std/3-motivation.md). + +# Explanation +[explanation]: #explanation + +Cargo will also permit `build-std` to be set as part of the `[profile]` section +([?][rationale-profile]). `build-std` configuration locations precedence is +updated as follows ([?][rationale-profile-precedence]): + +1. `[target.]` +2. `[target.]` +3. `[profile]` +4. `[build]` + +The `build-std` option in the Cargo configuration will be extended with two new +values - "compatible-profile" or "match-profile": + +```toml +[build] +build-std = "match-profile" # or `compatible-profile`/`compatible`/`always`/`never` +``` + +"match-profile" will become the default value for the release profile +([?][rationale-default], [?][rationale-why-not-always-rebuild]). The `bench` +profile inherits this default from `release`. + +Like `build-std = "compatible"`, when set to either "compatible-profile" or +"match-profile", then the standard library crates will be rebuilt +automatically when a pre-built standard library is not present. + +- "compatible-profile" rebuilds the standard library if the user is using a + different profile than the default "release" profile of the pre-built standard + library or a rebuild is necessary to maintain compatibility with the user's + crate ([?][rationale-compatible-profile]). + + Cargo will build the standard library using the same profile as the user, as + defined in the standard library workspace + ([?][rationale-compatible-profile-std]). It will vary only in the target modifiers + necessary to maintain compatibility with the user's crates. + + Pre-built available? | User's profile | Target modifiers changed? | Standard library re-built? + -------------------- | -------------- | ------------------------- | -------------------------- + No | `dev` | N/A | Yes, std's `dev` + No | `release` | N/A | Yes, std's `release` + Yes | `dev` | N/A | Yes, std's `dev` + Yes | `release` | Unchanged | No + Yes | `release` | Changed | Yes, std's `release` + +- "match-profile" rebuilds the standard library using the configuration of the + user's current profile ([?][rationale-match-profile]). Cargo will check + whether the compilation flags it would intend to use for the standard library + match those used with the pre-built standard library by asking rustc. + + Pre-built available? | User's profile | Target modifiers changed? | Standard library re-built? + -------------------- | -------------- | ------------------------- | -------------------------- + No | `dev` | N/A | Yes, user's `dev` + No | `release` | N/A | Yes, user's `release` + Yes | `dev` | N/A | Yes, user's `dev` + Yes | `release` | Unchanged | Yes, user's `release` + Yes | `release` | Changed | Yes, user's `release` + + > [!NOTE] + > + > rustc's compatibility checking from + > [`build-std = "compatible"`][compatible] will be extended to allow checking + > for any mismatch in relevant compilation flags (e.g. excluding things like + > dependency rlib search paths which will necessarily differ). rustc will not + > be able to serialise the value of each flag into rlib metadata due to + > performance overhead but will be able to serialise and compare a hash of + > these values. + +The above examples apply to any other profile too, such as `bench` or `test`. +When custom profiles are used, the standard library will be built in the profile +that the custom profile ultimately inherited from (via `inherited-from`). These +options are primarily useful for users wanting to use the same codegen flags +with the standard library or have a more debuggable standard library. + +As with the "always" option, the exact crates from the standard library to be +built are determined by the `build-std-crates` option or explicit dependencies on +the standard library if [*Standard library dependencies*][deps] were implemented. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This section aims to justify all of the decisions made in the proposed design +from [*Explanation*][explanation] and discuss why alternatives were not chosen. + +## Why permit `build-std` in `[profile]`? +[rationale-profile]: #why-permit-build-std-in-profile + +Configurations like "match-profile" for `build-std` make most sense when +combined with Cargo profiles that aim to maximise the optimisation of the final +binary. It is more likely that users would want to use "match-profile" with the +release profile than by default (as in `[build]`) or for a specific target (as +in `[target]`). + +However, permitting `build-std` in `[profile]` when in Cargo configurations, but +not in Cargo manifests, is inconsistent with other options that exist in +profiles. + +↩ [*Explanation*][explanation] + +## Why does `[profile]` have higher precedence than `[build]` and lower than `[target]`? +[rationale-profile-precedence]: #why-does-profile-have-higher-precedence-than-build-and-lower-than-target + +`[target]` configuration is more narrowly scoped than `[profile]` which is in +turn more narrowly scoped than the global default in `[build]`. There is no +existing precedent in Cargo for these sections having the precedence currently +proposed. + +↩ [*Explanation*][explanation] + +## Why have "match-profile" as the default for the release profile? +[rationale-default]: #why-have-match-profile-as-the-default-for-the-release-profile + +`build-std = "match-profile"` is intended to be used when additional time +spent building the standard library is not a problem and the quality of the +final artifact is paramount. This corresponds closely with the release profile, +where additional time spent on optimisations (e.g. with `-Ctarget-cpu`) is +acceptable. Always re-building the standard library with the user's profile +configuration in release mode is likely to result in a more optimised build than +with the pre-built standard library and is thus a reasonable default for the +`release` and `bench` profiles. + +↩ [*Explanation*][explanation] + +### Why not always default to "match-profile"? +[rationale-why-not-always-rebuild]: #why-not-always-default-to-match-profile + +Cargo's users don't currently expect that changing any part of their profile +configuration, such as trying a different optimisation level, would trigger a +rebuild of the standard library. For small projects, rebuilding the standard +library could be a significant increase in the overall build time for a project. + +If `build-std = "match-profile"` were the default, the standard library +could be rebuilt quite frequently without much benefit. Especially as the +pre-built standard library is built using the release profile, all debug profile +builds would immediately trigger a rebuild of the standard library. + +↩ [*Explanation*][explanation] + +### Why add "compatible-profile"? +[rationale-compatible-profile]: #why-add-compatible-profile + +"compatible-profile" is useful for when users want a more debuggable standard +library while keeping rebuilds of the standard library to a minimum. + +↩ [*Explanation*][explanation] + +### Why does "compatible-profile" use the standard library's profiles? +[rationale-compatible-profile-std]: #why-does-compatible-profile-use-the-standard-librarys-profiles + +By using the standard library's profile definitions, the library team will be +able to define a "dev" profile that is most useful for the standard library. + +↩ [*Explanation*][explanation] + +### Why add "match-profile"? +[rationale-match-profile]: #why-add-match-profile + +"match-profile" is useful for rebuilding the standard library with the same +codegen flags as the rest of the user's project, such as using `-Ctarget-cpu` to +gain additional optimisations. + +↩ [*Explanation*][explanation] + +# Prior art +[prior-art]: #prior-art + +See the [*Background*][background] and [*History*][history] of the build-std RFC. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +The following small details are likely to be bikeshed prior to RFC acceptance or +stabilisation and aren't pertinent to the overall design: + +## What should the "match-profile" and "compatible-profile" values of `build-std` be named? +[unresolved-naming]: #what-should-the-match-profile-and-compatible-profile-values-of-build-std-be-named + +It could be named something else. + +## Should `build-std` be in `[profile]` if it only makes in the Cargo configuration `[profile]`? +[unresolved-profile]: #should-build-std-be-in-profile-if-it-only-makes-in-the-cargo-configuration-profile + +This could be unintuitive for users. + +# Future possibilities +[future-possibilities]: #future-possibilities + +There are not currently any documented follow-ups to this RFC. + +[background]: ./0000-build-std/1-background.md +[history]: ./0000-build-std/2-history.md +[compatible]: ./0000a-build-std-compatible.md +[deps]: ./0000-build-std/5-standard-library-dependencies.md \ No newline at end of file