Skip to content

Commit 07ff999

Browse files
borsbors
committed
Auto merge of #144722 - ywxt:parallel-reproducibile, r=oli-obk
Fix parallel rustc not being reproducible due to unstable sorts of items Currently, A tuple `(DefId, SymbolName)` is used to determine the order of items in the final binary. However `DefId` is expected as non-deterministic, which leads to some not reproducible issues under parallel compilation. (See #140425 (comment)) Theoretically, we don't need the sorting because the order of these items is already deterministic. However, codegen tests reply on the same order of items between in binary and source. So here we added a new option `codegen-source-order` to indicate whether sorting based on the order in source. For codegen tests, items are sorted according to the order in the source code, whereas in the normal path, no sorting is performed. Specially, for codegen tests, in preparation for parallel compilation potentially being enabled by default in the future, we use `Span` replacing `DefId` to make the order deterministic. This PR is purposed to fix #140425, but seemly works on #140413 too. This behavior hasn't added into any test until we have a test suit for the parallel frontend. (See #143953) Related discussion: [Zulip](https://rust-lang.zulipchat.com/#narrow/channel/187679-t-compiler.2Fparallel-rustc/topic/Async.20closures.20not.20reproducible.28.23140425.29) #144576 Update #113349 r? `@oli-obk` cc `@lqd` `@cramertj` `@matthiaskrgr` `@Zoxc` `@SparrowLii` `@bjorn3` `@cjgillot` `@joshtriplett`
2 parents 2886b36 + 9b3e280 commit 07ff999

File tree

7 files changed

+107
-83
lines changed

7 files changed

+107
-83
lines changed

‎compiler/rustc_interface/src/tests.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -689,6 +689,7 @@ fn test_unstable_options_tracking_hash() {
689689
// Make sure that changing an [UNTRACKED] option leaves the hash unchanged.
690690
// tidy-alphabetical-start
691691
untracked!(assert_incr_state, Some(String::from("loaded")));
692+
untracked!(codegen_source_order, true);
692693
untracked!(deduplicate_diagnostics, false);
693694
untracked!(dump_dep_graph, true);
694695
untracked!(dump_mir, Some(String::from("abc")));

‎compiler/rustc_middle/src/mir/mono.rs

Lines changed: 36 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@ use rustc_hashes::Hash128;
1212
use rustc_hir::ItemId;
1313
use rustc_hir::attrs::InlineAttr;
1414
use rustc_hir::def_id::{CrateNum, DefId, DefIdSet, LOCAL_CRATE};
15-
use rustc_index::Idx;
1615
use rustc_macros::{HashStable, TyDecodable, TyEncodable};
1716
use rustc_query_system::ich::StableHashingContext;
1817
use rustc_session::config::OptLevel;
@@ -526,44 +525,50 @@ impl<'tcx> CodegenUnit<'tcx> {
526525
tcx: TyCtxt<'tcx>,
527526
) -> Vec<(MonoItem<'tcx>, MonoItemData)> {
528527
// The codegen tests rely on items being process in the same order as
529-
// they appear in the file, so for local items, we sort by node_id first
528+
// they appear in the file, so for local items, we sort by span first
530529
#[derive(PartialEq, Eq, PartialOrd, Ord)]
531-
struct ItemSortKey<'tcx>(Option<usize>, SymbolName<'tcx>);
532-
530+
struct ItemSortKey<'tcx>(Option<Span>, SymbolName<'tcx>);
531+
532+
// We only want to take HirIds of user-defines instances into account.
533+
// The others don't matter for the codegen tests and can even make item
534+
// order unstable.
535+
fn local_item_id<'tcx>(item: MonoItem<'tcx>) -> Option<DefId> {
536+
match item {
537+
MonoItem::Fn(ref instance) => match instance.def {
538+
InstanceKind::Item(def) => def.as_local().map(|_| def),
539+
InstanceKind::VTableShim(..)
540+
| InstanceKind::ReifyShim(..)
541+
| InstanceKind::Intrinsic(..)
542+
| InstanceKind::FnPtrShim(..)
543+
| InstanceKind::Virtual(..)
544+
| InstanceKind::ClosureOnceShim { .. }
545+
| InstanceKind::ConstructCoroutineInClosureShim { .. }
546+
| InstanceKind::DropGlue(..)
547+
| InstanceKind::CloneShim(..)
548+
| InstanceKind::ThreadLocalShim(..)
549+
| InstanceKind::FnPtrAddrShim(..)
550+
| InstanceKind::AsyncDropGlue(..)
551+
| InstanceKind::FutureDropPollShim(..)
552+
| InstanceKind::AsyncDropGlueCtorShim(..) => None,
553+
},
554+
MonoItem::Static(def_id) => def_id.as_local().map(|_| def_id),
555+
MonoItem::GlobalAsm(item_id) => Some(item_id.owner_id.def_id.to_def_id()),
556+
}
557+
}
533558
fn item_sort_key<'tcx>(tcx: TyCtxt<'tcx>, item: MonoItem<'tcx>) -> ItemSortKey<'tcx> {
534559
ItemSortKey(
535-
match item {
536-
MonoItem::Fn(ref instance) => {
537-
match instance.def {
538-
// We only want to take HirIds of user-defined
539-
// instances into account. The others don't matter for
540-
// the codegen tests and can even make item order
541-
// unstable.
542-
InstanceKind::Item(def) => def.as_local().map(Idx::index),
543-
InstanceKind::VTableShim(..)
544-
| InstanceKind::ReifyShim(..)
545-
| InstanceKind::Intrinsic(..)
546-
| InstanceKind::FnPtrShim(..)
547-
| InstanceKind::Virtual(..)
548-
| InstanceKind::ClosureOnceShim { .. }
549-
| InstanceKind::ConstructCoroutineInClosureShim { .. }
550-
| InstanceKind::DropGlue(..)
551-
| InstanceKind::CloneShim(..)
552-
| InstanceKind::ThreadLocalShim(..)
553-
| InstanceKind::FnPtrAddrShim(..)
554-
| InstanceKind::AsyncDropGlue(..)
555-
| InstanceKind::FutureDropPollShim(..)
556-
| InstanceKind::AsyncDropGlueCtorShim(..) => None,
557-
}
558-
}
559-
MonoItem::Static(def_id) => def_id.as_local().map(Idx::index),
560-
MonoItem::GlobalAsm(item_id) => Some(item_id.owner_id.def_id.index()),
561-
},
560+
local_item_id(item)
561+
.map(|def_id| tcx.def_span(def_id).find_ancestor_not_from_macro())
562+
.flatten(),
562563
item.symbol_name(tcx),
563564
)
564565
}
565566

566567
let mut items: Vec<_> = self.items().iter().map(|(&i, &data)| (i, data)).collect();
568+
if !tcx.sess.opts.unstable_opts.codegen_source_order {
569+
// It's already deterministic, so we can just use it.
570+
return items;
571+
}
567572
items.sort_by_cached_key(|&(i, _)| item_sort_key(tcx, i));
568573
items
569574
}

‎compiler/rustc_session/src/options.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2165,6 +2165,8 @@ options! {
21652165
"hash algorithm of source files used to check freshness in cargo (`blake3` or `sha256`)"),
21662166
codegen_backend: Option<String> = (None, parse_opt_string, [TRACKED],
21672167
"the backend to use"),
2168+
codegen_source_order: bool = (false, parse_bool, [UNTRACKED],
2169+
"emit mono items in the order of spans in source files (default: no)"),
21682170
contract_checks: Option<bool> = (None, parse_opt_bool, [TRACKED],
21692171
"emit runtime checks for contract pre- and post-conditions (default: no)"),
21702172
coverage_options: CoverageOptions = (CoverageOptions::default(), parse_coverage_options, [TRACKED],

‎src/doc/unstable-book/src/compiler-flags/codegen-source-order.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# `codegen-source-order`
2+
3+
---
4+
5+
This feature allows you to have a predictive and
6+
deterministic order for items after codegen, which
7+
is the same as in source code.
8+
9+
For every `CodegenUnit`, local `MonoItem`s would
10+
be sorted by `(Span, SymbolName)`, which
11+
makes codegen tests rely on the order of items in
12+
source files work.

‎src/tools/compiletest/src/runtest.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1695,6 +1695,10 @@ impl<'test> TestCx<'test> {
16951695
}
16961696
TestMode::Assembly | TestMode::Codegen => {
16971697
rustc.arg("-Cdebug-assertions=no");
1698+
// For assembly and codegen tests, we want to use the same order
1699+
// of the items of a codegen unit as the source order, so that
1700+
// we can compare the output with the source code through filecheck.
1701+
rustc.arg("-Zcodegen-source-order");
16981702
}
16991703
TestMode::Crashes => {
17001704
set_mir_dump_dir(&mut rustc);

‎tests/ui/intrinsics/bad-intrinsic-monomorphization.stderr

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,20 @@
1-
error[E0511]: invalid monomorphization of `cttz` intrinsic: expected basic integer type, found `Foo`
2-
--> $DIR/bad-intrinsic-monomorphization.rs:16:5
1+
error[E0511]: invalid monomorphization of `simd_add` intrinsic: expected SIMD input type, found non-SIMD `Foo`
2+
--> $DIR/bad-intrinsic-monomorphization.rs:26:5
33
|
4-
LL | intrinsics::cttz(v)
5-
| ^^^^^^^^^^^^^^^^^^^
4+
LL | intrinsics::simd::simd_add(a, b)
5+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
66

77
error[E0511]: invalid monomorphization of `fadd_fast` intrinsic: expected basic float type, found `Foo`
88
--> $DIR/bad-intrinsic-monomorphization.rs:21:5
99
|
1010
LL | intrinsics::fadd_fast(a, b)
1111
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
1212

13-
error[E0511]: invalid monomorphization of `simd_add` intrinsic: expected SIMD input type, found non-SIMD `Foo`
14-
--> $DIR/bad-intrinsic-monomorphization.rs:26:5
13+
error[E0511]: invalid monomorphization of `cttz` intrinsic: expected basic integer type, found `Foo`
14+
--> $DIR/bad-intrinsic-monomorphization.rs:16:5
1515
|
16-
LL | intrinsics::simd::simd_add(a, b)
17-
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
16+
LL | intrinsics::cttz(v)
17+
| ^^^^^^^^^^^^^^^^^^^
1818

1919
error: aborting due to 3 previous errors
2020

‎tests/ui/intrinsics/non-integer-atomic.stderr

Lines changed: 44 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,59 +1,53 @@
1-
error[E0511]: invalid monomorphization of `atomic_load` intrinsic: expected basic integer type, found `bool`
2-
--> $DIR/non-integer-atomic.rs:15:5
1+
error[E0511]: invalid monomorphization of `atomic_load` intrinsic: expected basic integer type, found `&dyn Fn()`
2+
--> $DIR/non-integer-atomic.rs:55:5
33
|
44
LL | intrinsics::atomic_load::<_, { SeqCst }>(p);
55
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
66

7-
error[E0511]: invalid monomorphization of `atomic_store` intrinsic: expected basic integer type, found `bool`
8-
--> $DIR/non-integer-atomic.rs:20:5
7+
error[E0511]: invalid monomorphization of `atomic_load` intrinsic: expected basic integer type, found `Foo`
8+
--> $DIR/non-integer-atomic.rs:35:5
9+
|
10+
LL | intrinsics::atomic_load::<_, { SeqCst }>(p);
11+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
12+
13+
error[E0511]: invalid monomorphization of `atomic_store` intrinsic: expected basic integer type, found `&dyn Fn()`
14+
--> $DIR/non-integer-atomic.rs:60:5
915
|
1016
LL | intrinsics::atomic_store::<_, { SeqCst }>(p, v);
1117
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1218

13-
error[E0511]: invalid monomorphization of `atomic_xchg` intrinsic: expected basic integer type, found `bool`
14-
--> $DIR/non-integer-atomic.rs:25:5
19+
error[E0511]: invalid monomorphization of `atomic_xchg` intrinsic: expected basic integer type, found `[u8; 100]`
20+
--> $DIR/non-integer-atomic.rs:85:5
1521
|
1622
LL | intrinsics::atomic_xchg::<_, { SeqCst }>(p, v);
1723
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1824

19-
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `bool`
20-
--> $DIR/non-integer-atomic.rs:30:5
25+
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `&dyn Fn()`
26+
--> $DIR/non-integer-atomic.rs:70:5
2127
|
2228
LL | intrinsics::atomic_cxchg::<_, { SeqCst }, { SeqCst }>(p, v, v);
2329
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2430

25-
error[E0511]: invalid monomorphization of `atomic_load` intrinsic: expected basic integer type, found `Foo`
26-
--> $DIR/non-integer-atomic.rs:35:5
27-
|
28-
LL | intrinsics::atomic_load::<_, { SeqCst }>(p);
29-
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
30-
3131
error[E0511]: invalid monomorphization of `atomic_store` intrinsic: expected basic integer type, found `Foo`
3232
--> $DIR/non-integer-atomic.rs:40:5
3333
|
3434
LL | intrinsics::atomic_store::<_, { SeqCst }>(p, v);
3535
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3636

37-
error[E0511]: invalid monomorphization of `atomic_xchg` intrinsic: expected basic integer type, found `Foo`
38-
--> $DIR/non-integer-atomic.rs:45:5
39-
|
40-
LL | intrinsics::atomic_xchg::<_, { SeqCst }>(p, v);
41-
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
42-
43-
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `Foo`
44-
--> $DIR/non-integer-atomic.rs:50:5
37+
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `[u8; 100]`
38+
--> $DIR/non-integer-atomic.rs:90:5
4539
|
4640
LL | intrinsics::atomic_cxchg::<_, { SeqCst }, { SeqCst }>(p, v, v);
4741
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
4842

49-
error[E0511]: invalid monomorphization of `atomic_load` intrinsic: expected basic integer type, found `&dyn Fn()`
50-
--> $DIR/non-integer-atomic.rs:55:5
43+
error[E0511]: invalid monomorphization of `atomic_store` intrinsic: expected basic integer type, found `[u8; 100]`
44+
--> $DIR/non-integer-atomic.rs:80:5
5145
|
52-
LL | intrinsics::atomic_load::<_, { SeqCst }>(p);
53-
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
46+
LL | intrinsics::atomic_store::<_, { SeqCst }>(p, v);
47+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5448

55-
error[E0511]: invalid monomorphization of `atomic_store` intrinsic: expected basic integer type, found `&dyn Fn()`
56-
--> $DIR/non-integer-atomic.rs:60:5
49+
error[E0511]: invalid monomorphization of `atomic_store` intrinsic: expected basic integer type, found `bool`
50+
--> $DIR/non-integer-atomic.rs:20:5
5751
|
5852
LL | intrinsics::atomic_store::<_, { SeqCst }>(p, v);
5953
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -64,36 +58,42 @@ error[E0511]: invalid monomorphization of `atomic_xchg` intrinsic: expected basi
6458
LL | intrinsics::atomic_xchg::<_, { SeqCst }>(p, v);
6559
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
6660

67-
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `&dyn Fn()`
68-
--> $DIR/non-integer-atomic.rs:70:5
69-
|
70-
LL | intrinsics::atomic_cxchg::<_, { SeqCst }, { SeqCst }>(p, v, v);
71-
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
72-
7361
error[E0511]: invalid monomorphization of `atomic_load` intrinsic: expected basic integer type, found `[u8; 100]`
7462
--> $DIR/non-integer-atomic.rs:75:5
7563
|
7664
LL | intrinsics::atomic_load::<_, { SeqCst }>(p);
7765
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
7866

79-
error[E0511]: invalid monomorphization of `atomic_store` intrinsic: expected basic integer type, found `[u8; 100]`
80-
--> $DIR/non-integer-atomic.rs:80:5
67+
error[E0511]: invalid monomorphization of `atomic_load` intrinsic: expected basic integer type, found `bool`
68+
--> $DIR/non-integer-atomic.rs:15:5
8169
|
82-
LL | intrinsics::atomic_store::<_, { SeqCst }>(p, v);
83-
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
70+
LL | intrinsics::atomic_load::<_, { SeqCst }>(p);
71+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
8472

85-
error[E0511]: invalid monomorphization of `atomic_xchg` intrinsic: expected basic integer type, found `[u8; 100]`
86-
--> $DIR/non-integer-atomic.rs:85:5
73+
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `bool`
74+
--> $DIR/non-integer-atomic.rs:30:5
8775
|
88-
LL | intrinsics::atomic_xchg::<_, { SeqCst }>(p, v);
89-
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
76+
LL | intrinsics::atomic_cxchg::<_, { SeqCst }, { SeqCst }>(p, v, v);
77+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
9078

91-
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `[u8; 100]`
92-
--> $DIR/non-integer-atomic.rs:90:5
79+
error[E0511]: invalid monomorphization of `atomic_cxchg` intrinsic: expected basic integer type, found `Foo`
80+
--> $DIR/non-integer-atomic.rs:50:5
9381
|
9482
LL | intrinsics::atomic_cxchg::<_, { SeqCst }, { SeqCst }>(p, v, v);
9583
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
9684

85+
error[E0511]: invalid monomorphization of `atomic_xchg` intrinsic: expected basic integer type, found `Foo`
86+
--> $DIR/non-integer-atomic.rs:45:5
87+
|
88+
LL | intrinsics::atomic_xchg::<_, { SeqCst }>(p, v);
89+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
90+
91+
error[E0511]: invalid monomorphization of `atomic_xchg` intrinsic: expected basic integer type, found `bool`
92+
--> $DIR/non-integer-atomic.rs:25:5
93+
|
94+
LL | intrinsics::atomic_xchg::<_, { SeqCst }>(p, v);
95+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
96+
9797
error: aborting due to 16 previous errors
9898

9999
For more information about this error, try `rustc --explain E0511`.

0 commit comments

Comments
 (0)