Skip to content

Commit 7d71a55

Browse files
1c3t3a1c3t3a
committed
Propose a 2025H2 goal for comprehensive niche checks
1 parent 0d0f95d commit 7d71a55

File tree

1 file changed

+116
-0
lines changed

1 file changed

+116
-0
lines changed

‎src/2025h2/comprehensive-niche-checks.md

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Comprehensive niche checks for Rust
2+
3+
| Metadata | |
4+
| :------------------| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
5+
| Point of contact | [@1c3t3a](https://github.com/1c3t3a) |
6+
| Teams | opsem, compiler |
7+
| Status | Proposed |
8+
| Task owners | [@1c3t3a](https://github.com/1c3t3a), [@jakos-sec](https://github.com/jakos-sec) |
9+
| Tracking issue | [rust-lang/rust-project-goals#262] |
10+
| Zulip channel | https://rust-lang.zulipchat.com/#narrow/channel/435869-project-goals/topic/Null.20and.20enum-discriminant.20runtime.20checks.20in.20.28goals.23262.29/with/508256920 |
11+
12+
## Summary
13+
14+
Add runtime checks to rustc that check for valid niche values. This is an
15+
extension of a [previous project goal](https://rust-lang.github.io/rust-project-goals/2025h1/null-enum-discriminant-debug-checks.html)
16+
that added null pointer and enum checks and generally works towards checking for
17+
Undefined behavior at runtime (in debug builds / behind compiler flags).
18+
19+
The check should fire for invalid values that are already created by inserting
20+
a check at load-time / on function entry.
21+
22+
## Motivation
23+
24+
Rust has a few types that specify "invalid values" for themselves, which means
25+
values that can never be occupied by an instance of this type. An example of
26+
that would be `0` for `std::num::NonZeroUsize`, or `null` for
27+
`std::ptr::NonNull`. These so called *niches* are used for optimizations, for
28+
example when being used together with `Option`, where `Option<NonNull<T>>` has
29+
the same size as `*mut T`.
30+
31+
In safe Rust it is never possible to construct such values, but that is
32+
different for unsafe Rust or FFI. E.g. with Rusts growing interop with C (e.g.
33+
in the Linux Kernel) and C++ it makes sense to expose such types over the
34+
language boundary. That can make for situations where a valid C or C++ value
35+
becomes an invalid Rust value.
36+
37+
### The status quo
38+
39+
While [Miri](https://github.com/rust-lang/miri) exists and does a great job at
40+
catching various types of UB in unsafe Rust code, it has the downside of only
41+
working on pure Rust code. Extern functions can not be called and a mixed
42+
language binary is unable to be executed in Miri.
43+
44+
We already added support for checking enum discriminants which is somewhat related,
45+
but this does not cover general niches.
46+
47+
Selecting where to insert these runtime checks for UB can be tricky. For example,
48+
if we want to detect a call in C++ which passes a null pointer to a Rust function
49+
that takes a reference (which thus gets the LLVM `nonnull` attribute), we cannot
50+
just insert a check inside the Rust function, because our check would be trivially
51+
optimized out (the parameter is guaranteed to not be null!). At a minimum, codegen
52+
needs to also not use such parameter attributes when niche checks are enabled
53+
then also insert the checks in a new block at the start of the function before the
54+
parameter is used.
55+
56+
In addition, technically any typed copy asserts the validity of the copied value
57+
as the type in question. But naively inserting a runtime check before every typed
58+
copy of a value with a niche can more than double compile time:
59+
https://github.com/rust-lang/rust/pull/104862#issuecomment-1773363404.
60+
61+
Thus validating all niches will require something more strategic, such as
62+
only inserting checks when SSA values are loaded from memory.
63+
64+
### The next 6 months
65+
66+
Within the next six months we would like to do the following things:
67+
68+
1. Develop a comprehensive niche value check for all niches.
69+
2. Insert it ideally when values are loaded from memory (e.g. a "checked-load") or on function entry.
70+
3. Migrate the enum check to the same system mentioned in 2.
71+
72+
### The "shiny future" we are working towards
73+
74+
Similar to how [UBSan](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html)
75+
exists in Clang, we would like to see an option in Rust to detect undefined
76+
behavior at runtime. As mentioned above, this is critical for cross-language
77+
interoperability and can help catch UB before it reaches production.
78+
79+
The extension of these checks can be done step-by-step, keeping in mind the
80+
overhead. Different checks can be enabled by separate flags (like in UBSan).
81+
Eventually we would like to check (sanitize) most items listed as
82+
[UB in Rust](https://doc.rust-lang.org/reference/behavior-considered-undefined.html).
83+
84+
## Ownership and team asks
85+
86+
**Owner:** [@1c3t3a](https://github.com/1c3t3a), [@jakos-sec](https://github.com/jakos-sec)
87+
88+
| Task | Owner(s) or team(s) | Notes |
89+
|------------------------------|------------------------------------------------------------------------------------|------------------------------------------|
90+
| Discussion and moral support | ![Team][], [compiler], [opsem] | |
91+
| Implementation | [@1c3t3a](https://github.com/1c3t3a), [@jakos-sec](https://github.com/jakos-sec) | |
92+
| MCP decision | ![Team][], [compiler] | Where to insert the check / checked load |
93+
| Dedicated reviewer | ![Team][] [compiler], [opsem] | @saethlin |
94+
95+
### Definitions
96+
97+
For definitions for terms used above, see the [About > Team Asks](https://rust-lang.github.io/rust-project-goals/about/team_asks.html) page.
98+
99+
* *Discussion and moral support* is the lowest level offering, basically committing the team to nothing but good vibes and general support for this endeavor.
100+
* *Author RFC* and *Implementation* means actually writing the code, document, whatever.
101+
* *Design meeting* means holding a synchronous meeting to review a proposal and provide feedback (no decision expected).
102+
* *RFC decisions* means reviewing an RFC and deciding whether to accept.
103+
* *Org decisions* means reaching a decision on an organizational or policy matter.
104+
* *Secondary review* of an RFC means that the team is "tangentially" involved in the RFC and should be expected to briefly review.
105+
* *Stabilizations* means reviewing a stabilization and report and deciding whether to stabilize.
106+
* *Standard reviews* refers to reviews for PRs against the repository; these PRs are not expected to be unduly large or complicated.
107+
* *Prioritized nominations* refers to prioritized lang-team response to nominated issues, with the expectation that there will be *some* response from the next weekly triage meeting.
108+
* *Dedicated review* means identifying an individual (or group of individuals) who will review the changes, as they're expected to require significant context.
109+
* Other kinds of decisions:
110+
* [Lang team experiments](https://lang-team.rust-lang.org/how_to/experiment.html) are used to add nightly features that do not yet have an RFC. They are limited to trusted contributors and are used to resolve design details such that an RFC can be written.
111+
* Compiler [Major Change Proposal (MCP)](https://forge.rust-lang.org/compiler/mcp.html) is used to propose a 'larger than average' change and get feedback from the compiler team.
112+
* Library [API Change Proposal (ACP)](https://std-dev-guide.rust-lang.org/development/feature-lifecycle.html) describes a change to the standard library.
113+
114+
## Frequently asked questions
115+
116+
None yet.

0 commit comments

Comments
 (0)