initial implementation of wildcard provenence for tree borrows #4630

royAmmerschuber · 2025-10-13T20:51:17Z

initial support for wildcard writes in tree borrows.

basic tests for wildcard provenance wildcard tracking data structure & expose_tag implementation basic wildcard accesses fix compilation errors & first working testcase comments & protector test update wildcard tracking when neccessary & fix ui tests remove either state move exposed_as to node & use own AccessLevel enum deallocation through wildcards wildcard reborrowing Location struct & comments test for correctly activating through wildcards & fix updating idempotent foreign access compelete verify function use check_nondet helper in a few more places

rustbot · 2025-10-13T20:51:20Z

Thank you for contributing to Miri!
Please remember to not force-push to the PR branch except when you need to rebase due to a conflict or when the reviewer asks you for it.

RalfJung

Here's a first round of comments. I didn't get to the wildcard.rs file yet. Also you did some really deep surgery in this code, much deeper than I expected, so we may have to discuss this in person because this is code I didn't write and therefore don't remember all that much about how it works.

Some general points that I didn't comment on everywhere, but that should be fixed everywhere:

Please write complete, correctly capitalized and punctuated sentences.
Please leave empty lines between functions and between types (except when you define a group of types that should be seen as a single unit).

View changes since this review

RalfJung · 2025-10-21T19:33:21Z

src/bin/miri.rs

            rustc_args.push(arg);
        }
    }
    // Tree Borrows implies strict provenance, and is not compatible with native calls.


This comment is now outdated.

native functions are only incompatible with strict provenance? So this check can be removed completely?

What do you mean?

"Tree Borrows implies strict provenance" is not true any more.

I'm not familiar with the nativelib interface. So I'm unsure if there is any other reason (besides strict provenance) why tree borrows would be incompatible with nativelib.
I assume there isn't, and I will remove the comment along with the check directly after it.

Indeed, we can now allow TB + native_lib. (I didn't even realize the check can be removed entirely, I just noted that the comment states an incorrect implication.)

RalfJung · 2025-10-21T19:33:40Z

src/borrow_tracker/mod.rs

        // The body of this loop needs `borrow_tracker` immutably
        // so we can't move this code inside the following `end_call`.
+
+        // TODO support protected wildcard pointers


It's unclear what "support" means.

RalfJung · 2025-10-21T19:34:08Z

src/borrow_tracker/tree_borrows/diagnostics.rs

    /// Which tag the access that caused this error was made through, i.e.
    /// which tag was used to read/write/deallocate.
-    pub accessed_info: &'node NodeDebugInfo,
+    pub accessed_info: Option<&'node NodeDebugInfo>,


This needs an explanation of what None means.

RalfJung · 2025-10-21T19:35:13Z

src/borrow_tracker/tree_borrows/mod.rs

        machine: &MiriMachine<'tcx>,
    ) -> InterpResult<'tcx> {
        // TODO: for now we bail out on wildcard pointers. Eventually we should
        // handle them as much as we can.


This comment seems outdated now?

RalfJung · 2025-10-21T19:38:37Z

src/borrow_tracker/tree_borrows/mod.rs

-        let orig_tag = match parent_prov {
-            ProvenanceExtra::Wildcard => return interp_ok(place.ptr().provenance), // TODO: handle wildcard pointers
-            ProvenanceExtra::Concrete(tag) => tag,
+        let (orig_tag, provenance) = match parent_prov {


So orig_tag is exactly the same as parent_prov except you converted it from ProvenanceExtra to Option...? What's the point of that?

Also, provenance is way to imprecise as a name. There's so many provenances floating around here, this isn't useful. I'd suggest new_prov.

RalfJung · 2025-10-21T19:55:54Z

src/borrow_tracker/tree_borrows/tree.rs

    pub fn dealloc(
        &mut self,
-        tag: BorTag,
+        tag: Option<BorTag>,


It's odd that dealloc changes its signature like this but perform_access does not.

RalfJung · 2025-10-21T19:56:22Z

src/borrow_tracker/tree_borrows/tree.rs

+        for (perms_range, Location { perms, .. }) in
+            self.rperms.iter_mut(access_range.start, access_range.size)
+        {
+            let tag = if let Some(tag) = tag {
+                tag
+            } else {
+                // the order in which we check if any nodes are invalidated doesnt matter,
+                // so we use the root as a default tag
+                self.nodes.get(self.root).unwrap().tag
+            };
+            TreeVisitor {
+                nodes: &mut self.nodes,
+                tag_mapping: &self.tag_mapping,
+                perms,
+                wildcard_accesses: None,
+            }
+            .traverse_this_parents_children_other(
+                tag,
+                // visit all children, skipping none
+                |_| ContinueTraversal::Recurse,
+                |args: NodeAppArgs<'_>| -> Result<(), TransitionError> {
+                    let NodeAppArgs { node, perm, .. } = args;
+                    let perm = perm.get().copied().unwrap_or_else(|| node.default_location_state());
+                    if global.borrow().protected_tags.get(&node.tag)


This looks like you entirely changed the old logic here... what is happening?

These changes are mostly whitespace.
Only the let tag = ... on line 787 and wildcard_accesses: None, on line 798 are actual changes.

RalfJung · 2025-10-21T19:57:11Z

src/borrow_tracker/tree_borrows/tree.rs

+        if node.is_exposed {
+            return None;
+        }


Why does this make sense? This is definitely non-trivial, therefore needs a comment.

RalfJung · 2025-10-21T19:57:31Z

src/borrow_tracker/tree_borrows/tree.rs

+/// methods for wildcard borrows
+impl<'tcx> Tree {
+    /// analogous to `perform_access`, but we do not know from which exposed reference the access happens.
+    pub fn perform_wildcard_access(


Shouldn't this be in wildcard.rs?

This is in tree.rs mostly because it accesses private functions of LocationState (perform_access, skip_if_known_noop, record_new_access)

RalfJung · 2025-10-21T19:58:28Z

src/borrow_tracker/tree_borrows/tree.rs

+    /// We do not know the accessed pointer, but we know that it is a child of this pointer
+    WildcardChildAccess,
+    /// We do not know the accessed pointer, but we know that it is foreign to this pointer
+    WildcardForeignAccess,


Why does it make sense to have these as new variants in this type, rather than having a separate type for the wildcard traversal?

This way I can reuse the code of perms.rs and LocationState::perform_access directly, which both take a AccessRelatedness as an argument. Also, no code seems to rely on the concrete value of AccessRelatedness. Only if its foreign or child access.

So could this enum be simplified to just Local vs Foreign?

This could also be part of the traversal adjustment preparation PR.

RalfJung · 2025-10-23T06:44:59Z

src/borrow_tracker/tree_borrows/mod.rs

                // Keep original provenance.
                return interp_ok(place.ptr().provenance);
            }
        };


There is a ptr_try_get_alloc_id above (which github doesn't let me add a comment to -- thanks github) which is subtle and I think not entirely correct. For a wildcard pointer, this will resolve the pointer to an AllocId if it can. If ptr_size is 0, however, that might not be the only legal AllocId!

If the size is 0 on a wildcard pointer I think we have to bail early, there's not much we can do.

ptr_size=0 support remove unneccessary nativelib check use provenance extra instead of option unify perform_access

RalfJung · 2025-10-27T09:32:10Z

src/borrow_tracker/tree_borrows/tree.rs

+    /// with possible lazy initialization.
+    ///
+    /// NOTE: same guarantees on entry initialisation as for `perms`
+    pub wildcard_accesses: UniValMap<WildcardAccessTracking>,


This needs a more extensive comment for why it is a separate map.

RalfJung · 2025-10-27T09:49:43Z

src/borrow_tracker/tree_borrows/wildcard.rs

+/// childrens max_child_access/max_foreign_access
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+pub struct WildcardAccessTracking {
+    /// how many of this nodes direct children have `max_child_access==Write`


Suggested change

/// how many of this nodes direct children have `max_child_access==Write`

/// how many of this node's direct children have `max_child_access==Write`

RalfJung · 2025-10-27T09:50:21Z

src/borrow_tracker/tree_borrows/wildcard.rs

+/// were relative to the pointer the access happened from
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+pub enum WildcardAccessRelatedness {
+    /// the access definitively happened through a child pointer


"pointer" is confusing here, do you mean "node"?

RalfJung · 2025-10-27T09:50:38Z

src/borrow_tracker/tree_borrows/wildcard.rs

+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+pub enum WildcardAccessRelatedness {
+    /// the access definitively happened through a child pointer
+    ChildAccess,


When you use "child" to mean the reflexive transitive closure, please say "local".

RalfJung · 2025-10-27T09:51:47Z

src/borrow_tracker/tree_borrows/tree.rs

Please factor the changes to the visitor that give the callbacks access to the entire tree into a separate commit/PR. We also don't need two layers of visitors, just adjust the existing NodeAppArgs to have the extra state.

RalfJung · 2025-10-27T09:53:29Z

src/borrow_tracker/tree_borrows/wildcard.rs

+            AccessKind::Write => self.write_access_relatedness(exposed_as),
+        }
+    }
+    /// from where relative to this pointer a read access could happen


"this pointer"/"this node" is confusing here since you don't actually identify a node here.
What you mean is "a node with this wildcard info" or so?

RalfJung · 2025-10-27T09:59:15Z

src/borrow_tracker/tree_borrows/wildcard.rs

+        /// this function calculates the siblings `max_child_access`, both of the other fields need to be passed as arguments
+        ///
+        /// * `other_factors`:  we only ever change one of these values. The max value of the other fields we dont change should be passed through the `other_factors` parameter
+        /// * `old_access_type`,`access_type`: we change the parameter not covered by `other_factors` from `old_access_type`


"we change" seems to mean "the caller changed/changes" or so? It sounds like this function is doing the changing.

RalfJung · 2025-10-27T10:00:43Z

src/borrow_tracker/tree_borrows/wildcard.rs

+        /// pushes children onto the stack, if their `max_foreign_access` field needs to be updated
+        ///
+        /// the `max_foreign_access` fields is set based on the max of the parents `max_foreign_access`,
+        /// `exposed_as` and its siblings `max_child_access`.


2 of these aren't fields though...

RalfJung · 2025-10-27T10:06:00Z

src/borrow_tracker/tree_borrows/wildcard.rs

+        perms: &UniValMap<LocationState>,
+        wildcard_accesses: &mut UniValMap<WildcardAccessTracking>,
+    ) {
+        /// pushes children onto the stack, if their `max_foreign_access` field needs to be updated


This needs a much more extensive comment:

on some node (say which!), one of these "fields" (properties? they aren't all fields) changed, from old_access_type to new_access type I think?

the other two stay the same, and are stored in other_factors

but then there's some more complicated stuff for siblings?

Also, if there's three modes to call this function (depending on what changed), every caller needs to document which mode it is using and why the special requirements for that mode are upheld.

But somehow the function doesn't even know what the mode is so... is there some more uniform way to do this? Can the function be called with just the one info on how the foreign access mode that can come in via the parent edge changed?

RalfJung · 2025-10-27T10:09:23Z

src/borrow_tracker/tree_borrows/wildcard.rs

+            access_type: WildcardAccessLevel,
+            old_access_type: WildcardAccessLevel,
+            access: WildcardAccessTracking,
+            mut children: impl Iterator<Item = UniIndex>,


I guess all these have a common parent? Is it all children of that parent?
Is that parent the node that the main comment refers to, whose properties changed?

RalfJung · 2025-10-27T10:29:20Z

src/borrow_tracker/tree_borrows/wildcard.rs

+                /* other factors */ src_access.max_foreign_access,
+                access_type,
+                old_access_type,
+                src_access.clone(),


Maybe just pass a reference instead of cloning.

EDIT: Or maybe not, turns out that is not always the parent node info...

RalfJung · 2025-10-27T10:29:32Z

src/borrow_tracker/tree_borrows/wildcard.rs

+            other_factors: WildcardAccessLevel,
+            access_type: WildcardAccessLevel,
+            old_access_type: WildcardAccessLevel,
+            access: WildcardAccessTracking,


The comment should say what this is. Also maybe call it something involving parent in its name?

In fact, just replace the entire thing by the read and write counts.

RalfJung · 2025-10-27T10:33:48Z

src/borrow_tracker/tree_borrows/wildcard.rs

+                // Read -> Write
+                // Write -> Read
+                // Write -> None
+                access.child_writes


Why is it okay to ignore child_reads here, given that the read permissions can also change?

RalfJung · 2025-10-27T10:41:01Z

src/borrow_tracker/tree_borrows/wildcard.rs

+            other_factors: WildcardAccessLevel,
+            access_type: WildcardAccessLevel,
+            old_access_type: WildcardAccessLevel,
+            access: WildcardAccessTracking,


In fact, just replace the entire thing by the read and write counts.

RalfJung · 2025-10-27T10:48:06Z

src/borrow_tracker/tree_borrows/wildcard.rs

+        old_access_type: WildcardAccessLevel,
+        access_type: WildcardAccessLevel,


Maybe call these old_exposed_as and new_exposed_as?

RalfJung · 2025-10-27T10:49:23Z

src/borrow_tracker/tree_borrows/wildcard.rs

+        // dont change (for parents child_permissions and for the other children foreign permissions)
+        {
+            // we need to keep track of how the previous permissions changed
+            let mut prev_old_access = old_access_type;


child_old_access? Also what exactly is the invariant on this field? It's instantiated based on an exposed_as but gets updated with max_child?

RalfJung · 2025-10-27T10:54:14Z

src/borrow_tracker/tree_borrows/wildcard.rs

+        {
+            // we need to keep track of how the previous permissions changed
+            let mut prev_old_access = old_access_type;
+            let mut prev = id;


child (the one we come from in our upwards traversal)?

RalfJung · 2025-10-27T10:57:01Z

src/borrow_tracker/tree_borrows/tree.rs

+    /// We do not know the accessed pointer, but we know that it is a child of this pointer
+    WildcardChildAccess,
+    /// We do not know the accessed pointer, but we know that it is foreign to this pointer
+    WildcardForeignAccess,


This could also be part of the traversal adjustment preparation PR.

RalfJung · 2025-10-27T10:58:37Z

src/borrow_tracker/tree_borrows/wildcard.rs

+                // is defined by this child. So we only need to update this one child
+                stack.push((
+                    children
+                        .find(|id| {


This should give a unique node, right?

I'd say make this filter, call next once to get the node, then call it again to test that it is unique.

RalfJung · 2025-10-27T11:00:00Z

src/borrow_tracker/tree_borrows/wildcard.rs

+    ) {
+        // find root node
+        let mut root = id;
+        while let Some(parent) = nodes.get(root).and_then(|n| n.parent) {


Suggested change

while let Some(parent) = nodes.get(root).and_then(|n| n.parent) {

while let Some(parent) = nodes.get(root).unwrap().parent {

RalfJung · 2025-10-27T11:06:26Z

src/borrow_tracker/tree_borrows/wildcard.rs

+                    .map(|child| {
+                        let node = nodes.get(child).unwrap();
+                        let perm = perms.get(child).map(LocationState::permission);
+                        let access = wildcard_accesses.get(child).unwrap();


This uses the data we are checking (wildcard_accesses) to check said data... doesn't that risk that we miss some inconsistencies?

For instance, what if we have a tree where no node is exposed, but there's a node somewhere that says "one of my children is exposed" and there's exactly one child that says "my parent is exposed"? Would we catch that?

I should probably rename max_other_children to max_access_siblings to make this code clearer.

| A | \ B C

So in the example A has child_reads=1 and B has max_foreign_access=Read (everything else 0)?

This will fail, because B and C both have max_child_access()==None, therefore A should be child_reads=0.

It would also fail because for B the parents max_foreign_access, exposed_as and its siblings (C) max_child_access() are all None, meaning B's max_foreign_access should also be None

The invariants verify_consistency checks should be described in the WildcardAccessTracking struct definition.

This will fail, because B and C both have max_child_access()==None, therefore A should be child_reads=0.

It would also fail because for B the parents max_foreign_access, exposed_as and its siblings (C) max_child_access() are all None, meaning B's max_foreign_access should also be None

Sounds good then, thanks!

RalfJung · 2025-10-27T14:11:22Z

@rustbot author

rustbot · 2025-10-27T14:11:27Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

RalfJung · 2025-10-27T14:13:22Z

src/borrow_tracker/tree_borrows/diagnostics.rs

    // What kind of access caused this error (read, write, reborrow, deallocation)
    pub access_cause: AccessCause,
-    /// Which tag the access that caused this error was made through, i.e.
+    /// Which tag if any the access that caused this error was made through, i.e.


Should be something like (if any) or , if any,.

royAmmerschuber · 2025-10-27T19:20:44Z

#4654 contains the updates to the TreeVisitor and AccessRelatedness

This makes strict consistency requirement between wildcard tracking datastructure and the rest of the tree looser, giving us more flexibility in how we update it.

…rence to node

…ppen

royAmmerschuber · 2025-10-29T15:04:54Z

@rustbot ready

royAmmerschuber · 2025-10-29T15:12:47Z

src/borrow_tracker/tree_borrows/wildcard.rs

+/// Represensts the maximum access level that is possible.
+///
+/// Note that we derive Ord and PartialOrd, so the order in which variants are listed below matters:
+/// None < Read < Write. Do not change that order.
+#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Default)]
+pub enum WildcardAccessLevel {
+    #[default]
+    None,
+    Read,
+    Write,
+}


There exists an identical enum in foreign_access_skipping.rs called IdempotentForeignAccess.
I'm unsure if they should be combined into one single AccessLevel type.
Or maybe just rename WildcardAccessLevel to MaxAccessLevel as it isn't really specific to wildcard accesses.

rustbot added the S-waiting-on-review Status: Waiting for a review to complete label Oct 13, 2025

royAmmerschuber added 2 commits October 19, 2025 14:10

respect idempotent foreign access optimization

165a18a

fix formatting

0d40e7b

RalfJung reviewed Oct 21, 2025

View reviewed changes

RalfJung reviewed Oct 23, 2025

View reviewed changes

royAmmerschuber added 3 commits October 26, 2025 14:23

properly handle conflicted protected tags

e518de0

smaller changes

66aa932

ptr_size=0 support remove unneccessary nativelib check use provenance extra instead of option unify perform_access

improve comments

fd2438b

RalfJung reviewed Oct 27, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: Waiting for the PR author to address review comments and removed S-waiting-on-review Status: Waiting for a review to complete labels Oct 27, 2025

RalfJung reviewed Oct 27, 2025

View reviewed changes

fix verify_consistency

fd1f4f7

fix formatting & improve error messages

adab680

royAmmerschuber added 6 commits October 28, 2025 20:12

make update_exposure clearer & fix some bugs

3c3f264

add exposed_as field to WildcardAccessTracking

a52fde6

This makes strict consistency requirement between wildcard tracking datastructure and the rest of the tree looser, giving us more flexibility in how we update it.

run fmt/clippy & rename child_access to local_access and pointer/refe…

121b535

…rence to node

stop garbage collecting exposed tags through which an access could ha…

90e30fd

…ppen

formatting...

c572e06

rename Location, WildcardAccessTracking an rperms

d52c0fc

rustbot added S-waiting-on-review Status: Waiting for a review to complete and removed S-waiting-on-author Status: Waiting for the PR author to address review comments labels Oct 29, 2025

royAmmerschuber commented Oct 29, 2025

View reviewed changes

	/// how many of this nodes direct children have `max_child_access==Write`
	/// how many of this node's direct children have `max_child_access==Write`

		old_access_type: WildcardAccessLevel,
		access_type: WildcardAccessLevel,

	while let Some(parent) = nodes.get(root).and_then(\|n\| n.parent) {
	while let Some(parent) = nodes.get(root).unwrap().parent {

Uh oh!

initial implementation of wildcard provenence for tree borrows #4630

Are you sure you want to change the base?

initial implementation of wildcard provenence for tree borrows #4630

Conversation

royAmmerschuber commented Oct 13, 2025

Uh oh!

rustbot commented Oct 13, 2025

Uh oh!

RalfJung left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

RalfJung left a comment •

edited by rustbot

Loading

RalfJung Oct 27, 2025 •

edited

Loading

RalfJung Oct 27, 2025 •

edited

Loading