Unify AST Visitors with a macro like MIR Visitors

[maxcabrajac]

This is a PR to work on #127615

[rustbot]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @lcnr (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

• @rustbot author: the review is finished, PR author should check the comments and take action accordingly
• @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

[lcnr]

r? compiler

[lcnr]

r? @cjgillot woops

[cjgillot]

Wow, that's some great work you did @maxcabrajac!
Please take this as an opportunity to simplify stuff. If some distinction has you make weird corner cases, don't hesitate to remove it. My comments are suggestions in this direction.

[cjgillot]

Should we remove the P! from all these?

[maxcabrajac]

I put these P! to maintain all function signatures the same, thus avoiding changes on implementers. I don't know why the original MutVisitor received &mut P<T> instead of &mut T for these types, but I just kept what was being done.

[maxcabrajac]

I think it's better to do two passes. Do what can be done without changing the trait API, after that, change what is weird about their original implementation and change all implementers accordingly. What do you think about it?
There is a pretty big roadblock now... Everything around walk_item seems to be pretty different from Visitor to MutVisitor and I really don't know where to start =/. Do you have any advice on that?

[petrochenkov]

Now I'm doing what can be done before introducing macros, ok?

👍

Is make_visit the one that bothers?

I didn't look into much detail yet, just a general feeling (e.g. there are multiple layers of macros, a dozen of helper macros, etc).

[bors]

☔ The latest upstream changes (presumably #132116) made this pull request unmergeable. Please resolve the merge conflicts.

[Dylan-DPC]

@maxcabrajac any updates on this? thanks

[maxcabrajac]

Hi @Dylan-DPC!
I haven't worked on it this past week =/
Last week, I was trying to remove P<> wrappers from some functions that didn't seem to need it on rustc_expand.
But I came to the conclusion that removing them might be either harmful or too much for me.

The motivation for doing it was functions like this:

(found in: https://github.com/rust-lang/rust/blob/3d1dba830a564d1118361345d7ada47a05241f45/compiler/rustc_expand/src/placeholders.rs#L303C1-L308C6)

 fn visit_expr(&mut self, expr: &mut P<ast::Expr>) {
     match expr.kind {
         ast::ExprKind::MacCall(_) => *expr = self.remove(expr.id).make_expr(),
         _ => walk_expr(self, expr),
     }
 }

In the code above the P<Expr> that is in expr is dropped and substituted by the one received from make_expr(). This throws away the the first allocation and uses the new one.

After removing the wrapper from visit_expr the same code turns into this:

 fn visit_expr(&mut self, expr: &mut ast::Expr) {
     match expr.kind {
         ast::ExprKind::MacCall(_) => *expr = self.remove(expr.id).make_expr().into_inner(),
         _ => walk_expr(self, expr),
     }
 }

This reuses the previous allocation, but throws away the one make_expr made and introduces one additional Expr move (as the Expr is now copied from new to old memory in the assignment) . So I thought: "what about changing make_expr() to return an unwrapped Expr and make the callee wrap it if they need to."

But changing this requires a huge amount of other changes and thinking about whether each callee requires the wrapper or should also return an unwrapped Expr.
On the other hand, this might introduce a bunch of places where Exprs are moved in the stack itself, which might outweigh the benefits of not doing some temporary allocations.

All of this multiplied by the amount of structs that are wrapped in P<>s (6 or 7 if I'm not mistaken)

I still think removing the wrappers from the visit_* is a good idea for standardization between versions of Visitor.
What do you think about it?

There are still other unrelated changes to do. I'll focus on them for now. =)

[Dylan-DPC]

@maxcabrajac any updates on this? thanks

[petrochenkov]

IIRC, the last time I looked at this PR it was mostly ready for merging.
There was a small question regarding P<>, but it could be delayed, and it's probably better to keep those P<>s around anyway, given the explanation in #128974 (comment).
@maxcabrajac Do you still plan to return to this?

Upd: cc "Remove P<> from visit_s in ast MutVisitor", it was not linked from this PR, so I forgot about it when writing the comment.

[petrochenkov]

Closing due to inactivity.