Implement AST visitors using a derive macro.

[cjgillot]

AST visitors are large and error-prone beasts. This PR attempts to write them using a derive macro.

The design uses three traits: Visitor, Visitable, Walkable.

• Visitor is the trait implemented by downstream crates, it lists visit_stuff methods, which call Walkable::walk_ref by default;
• Walkable is derived using the macro, the generated walk_ref method calls Visitable::visit on each component;
• Visitable is implemented by common_visitor_and_walkers macro, to call the proper Visitor::visit_stuff method if it exists, to call Walkable::walk_ref if there is none.

I agree this is quite a lot of spaghetti macros. I'm open to suggestions on how to reduce the amount of boilerplate code.

If this PR is accepted, I believe the same design can be used for the HIR visitor.

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[cjgillot]

@bors try @rust-timer queue

[bors]

⌛ Trying commit 8dea854 with merge 1140f76...

[bors]

☀️ Try build successful - checks-actions
Build commit: 1140f76 (1140f76b9982bc0f7d192946fc7f08237146878b)

[rust-timer]

Finished benchmarking commit (1140f76): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.7%	[0.6%, 0.9%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.2%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (secondary 2.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.4%	[2.4%, 2.4%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Cycles

Results (primary 4.5%, secondary -1.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.5%	[4.5%, 4.5%]	1
Regressions ❌ (secondary)	3.2%	[3.2%, 3.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-5.7%	[-5.7%, -5.7%]	1
All ❌✅ (primary)	4.5%	[4.5%, 4.5%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 465.776s -> 467.271s (0.32%)
Artifact size: 374.73 MiB -> 374.77 MiB (0.01%)

[jieyouxu]

r? compiler (review capacity)

[SparrowLii]

I think this need a reviewer who is familiar with AST and macros.
r? @petrochenkov I guess you have the ability to review it?

[petrochenkov]

The code in compiler/rustc_ast/src/(mut_)visit.rs is clearly write-only now, but it was already true since the visitor/mut-visitor deduplication that introduced heavy use of macros, so this PR doesn't make things significantly worse.

[petrochenkov]

I'm open to suggestions on how to reduce the amount of boilerplate code.

I think in this PR the remaining code in mut_visit.rs can be merged into visit.rs, it will allow to get rid of macro_exporting and importing the helper macros at least.

[petrochenkov]

At high level, there are two calls to common_visitor_and_walkers!.

common_visitor_and_walkers!(Visitor<'a>);
common_visitor_and_walkers!((mut) MutVisitor);

At the same time the logic in helper macros is duplicated for mutable OR immutable visitor, e.g. like this

#[macro_export]
macro_rules! impl_visitable_noop {
    (<$lt:lifetime> $($ty:ty,)*) => {
        $(
            impl_visitable!(|&$lt self: $ty, _vis: &mut V, _extra: ()| {
                V::Result::output()
            });
        )*
    };
    (<mut> $($ty:ty,)*) => {
        $(
            impl_visitable!(|&mut self: $ty, _vis: &mut V, _extra: ()| {});
        )*
    }
}

If the two visitors are placed into the same file, then perhaps the common_visitor_and_walkers macro can be removed, and both its calls inlined, and the logic in helper macros can be changed to produce code for mutable AND immutable visitor?

macro_rules! impl_visitable_noop {
    (<$lt:lifetime> $($ty:ty,)*) => {
        $(
            impl_visitable!(|&$lt self: $ty, _vis: &mut V, _extra: ()| {
                V::Result::output()
            });
        )*

        $(
            impl_visitable!(|&mut self: $ty, _vis: &mut V, _extra: ()| {});
        )*
    };
}

[cjgillot]

@petrochenkov Merging both files seems a bit tricky, as they are used by downstream code to chose which version of walk_* to call. The churn seems a bit too high. Instead I changed the macros to just have a version in visit and another one in mut_visit. Do you want me to duplicate contents of common_visitor_and_walkers too?

[petrochenkov]

@petrochenkov Merging both files seems a bit tricky, as they are used by downstream code to chose which version of walk_* to call. The churn seems a bit too high. Instead I changed the macros to just have a version in visit and another one in mut_visit. Do you want me to duplicate contents of common_visitor_and_walkers too?

Yeah, certainly not a change for this PR, this one is already quite large.

[petrochenkov]

r=me after squashing commits.
@rustbot author

[rustbot]

Reminder, once the PR becomes ready for a review, use @rustbot ready.