Establish Infrastructure #105

makai410 · 2025-08-18T21:20:33Z

devtool: A helper for maintaining rustc_public. The main logic for commands like build is implemented here. This PR implements ./x build, ./x test [--bless], ./x fmt [--check], ./x githook install/uninstall and ./x clean. Windows is not supported yet.
This PR turns the compiletest crate into a single file, making it possilbe to run test suites using cargo test.

celinval · 2025-08-19T01:18:52Z

Amazing! On the file structure, since this repo is also meant to store files related to the project, can I suggest that we create a rustc_public folder with the code relevant for the crate. Or would that mess up with the josh-sync?

celinval · 2025-08-19T01:20:22Z

I was also wondering if you could break down the PR a bit more. I think there are changes here that are ready to be merged independently

makai410 · 2025-08-19T15:35:47Z

Amazing! On the file structure, since this repo is also meant to store files related to the project, can I suggest that we create a rustc_public folder with the code relevant for the crate. Or would that mess up with the josh-sync?

It seems feasible by writing a filter to the josh, though I haven't tried it yet. IMO it's OK to keep our rustc_public code in the src/ folder, and miri does the same.

I was also wondering if you could break down the PR a bit more. I think there are changes here that are ready to be merged independently

Yea makes sense. I'll split it up.

oli-obk · 2025-08-26T08:42:20Z

CI wants some rustfmting

oli-obk · 2025-08-26T10:40:31Z

.github/workflows/nightly.yml

-        env:
-          TOOLS_BIN: "/tmp/smir/bin"
+          # Note that the downloaded version is dated 2025-08-09.
+          toolchain: nightly-2025-08-10


I would prefer a toolchain toml file, as we'll update it frequently and that's easier to automate than editing a github workflow

Fine for now tho if you want to land this first and tune later

Ah makes sense. I'd like to update it later together with the automation pr

Will that be a follow up PR?

oli-obk · 2025-08-26T12:22:26Z

This changes the rustc_public crate. Can you check that it still works inside the rust repo if synced back?

makai410 · 2025-08-27T01:32:08Z

The rustc compilation fails directly since we have a really strict msrv in our build.rs. Can we use the :exclude filter to avoid syncing those changes back?

makai410 · 2025-08-27T12:25:49Z

wait, is it possible to keep the files for the crates.io version in a separate directory? We can specify which Cargo.toml to use when building, and Cargo.toml itself can define the lib and build files.

oli-obk · 2025-08-29T18:13:43Z

wait, is it possible to keep the files for the crates.io version in a separate directory? We can specify which Cargo.toml to use when building, and Cargo.toml itself can define the lib and build files.

Yea. I think the plan was to overwrite the sources of the crates.io copy every time we make a breaking change?

makai410 · 2025-08-29T18:41:36Z

Yes the rustc version would be the base of a new major release

celinval · 2025-09-03T23:49:09Z

.github/workflows/format.yml

+        uses: actions/checkout@v4
+
+      - name: Rust Toolchain
+        uses: dtolnay/rust-toolchain@nightly


Do we need nightly for format check?

celinval · 2025-09-03T23:50:23Z

.github/workflows/nightly.yml

-        env:
-          TOOLS_BIN: "/tmp/smir/bin"
+          # Note that the downloaded version is dated 2025-08-09.
+          toolchain: nightly-2025-08-10


Will that be a follow up PR?

celinval · 2025-09-03T23:52:33Z

.github/workflows/nightly.yml

-          TOOLS_BIN: "/tmp/smir/bin"
-        # Don't fail CI for now. See: https://github.com/rust-lang/project-stable-mir/issues/39
-        continue-on-error: true
+  latest:


Btw, you should be able to create an action and reuse it for these two jobs. Not a big deal at this point, but if you have a follow up PR, that might be a nice change.

I also want to add a test for latest stable

Btw, you should be able to create an action and reuse it for these two jobs. Not a big deal at this point, but if you have a follow up PR, that might be a nice change.

Yes, that would be done as follow up

I also want to add a test for latest stable

ummm iirc the stable version can’t enable rustc_private

People would still need to set bootstrap env variable

celinval · 2025-09-04T00:07:43Z

rustc_public/tests/compiletest.rs

+    Ok(())
+}
+
+macro_rules! regexes {


What are these regexes for? I thought ui_test handled this for us. @oli-obk?

Not really, e.g., it won't normalize 0x0000000000000000 to $HEX, and the line and column info to :LL:CC.

Gotcha! Can you add a comment saying what patterns we are matching and why? Regex are always so painful to maintain. Thanks

celinval · 2025-09-04T00:15:27Z

rustc_public/Cargo.toml

+harness = false

 [features]
-# tidy-alphabetical-start


Why did you remove these directives?

because we don't have the tidy tool in our repo

I have mixed feelings about this one. On the one hand, this crate will get synchronized with the rust repo, where they do enforce this. It will also be consistent with other rustc crates. On the other hand, leaving it on without guardrails can make sync a bit more painful.

I would suggest we leave it on and add a comment to folks changing Cargo.toml to ensure imports are in alphabetical order. If this becomes a hassle, we can remove it.

celinval · 2025-09-04T00:15:52Z

rustc_public/Cargo.toml

 name = "rustc_public"
 version = "0.1.0-preview"
+authors = ["rustc_public team"]
+description = "Define compiler intermediate representation usable by external tools"


celinval · 2025-09-10T17:26:58Z

BTW, sorry for the delay. Next time, do you mind creating separate commits for moving files and content changes. It makes it much easier to visualize the content changes. Otherwise, everything looks like new code.

makai410 · 2025-09-11T07:25:16Z

Oh sorry about that, thanks for the reminder though. I'll make sure to do that next time.

makai410 · 2025-09-12T11:19:36Z

wait, is it possible to keep the files for the crates.io version in a separate directory? We can specify which Cargo.toml to use when building, and Cargo.toml itself can define the lib and build files.

I feel like maintaining two versions of lib.rs and Cargo.toml is quite ad-hoc. The problem here is that the crates.io version wants to enable rustc_private but the rustc version shouldn't. I think the conditional compilation should be good to solve this but how can we detect if it's living in the rustc or not?
@oli-obk @celinval Any ideas?

celinval · 2025-09-13T16:26:34Z

Can we use a cargo feature to indicate that we are being built as part of the compiler?

makai410 · 2025-09-17T07:57:01Z

Yea but I think we'd still have to maintain two Cargo.toml.

celinval · 2025-09-19T17:43:45Z

I actually think a config is probably the right way to go. We can set the config as part of the compiler bootstrap. Wouldn't we be able to keep the same code in that case?

makai410 · 2025-09-24T13:56:32Z

Sorry for the delay. I think the problem is that they don't share the same dependencies in the Cargo.toml, so if we sync things like rustc_middle = { path = "../rustc_middle" } into our repo, the build would fail.

But TBH now I feel OK with just maintaining two versions of Cargo.toml files and using a cargo feature to indicate where we're being built. And we could add a filter for the josh sync to exclude the Cargo.toml :) What do you think?

oli-obk · 2025-09-24T14:43:43Z

If you feel comfortable implementing that, I think it would be maintainable and understandable, so I'm good with it

celinval · 2025-09-24T23:27:49Z

Sorry for the delay. I think the problem is that they don't share the same dependencies in the Cargo.toml, so if we sync things like rustc_middle = { path = "../rustc_middle" } into our repo, the build would fail.

But TBH now I feel OK with just maintaining two versions of Cargo.toml files and using a cargo feature to indicate where we're being built. And we could add a filter for the josh sync to exclude the Cargo.toml :) What do you think?

Ok... let's start there. We can adjust as we go. Thanks!

makai410 force-pushed the infra-cl branch from ff05f46 to f1c8dfa Compare August 18, 2025 21:32

makai410 force-pushed the infra-cl branch from f1c8dfa to f8af664 Compare August 25, 2025 12:48

makai410 marked this pull request as ready for review August 25, 2025 12:51

makai410 added 2 commits August 26, 2025 17:32

make rustc_public a crates.io crate

1f6d11c

establish infrastructrure

00ce048

makai410 force-pushed the infra-cl branch from f8af664 to 00ce048 Compare August 26, 2025 09:34

oli-obk reviewed Aug 26, 2025

View reviewed changes

celinval reviewed Sep 10, 2025

View reviewed changes

Establish Infrastructure #105

Are you sure you want to change the base?

Establish Infrastructure #105

Uh oh!

Conversation

makai410 commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

celinval commented Aug 19, 2025

Uh oh!

celinval commented Aug 19, 2025

Uh oh!

makai410 commented Aug 19, 2025

Uh oh!

oli-obk commented Aug 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oli-obk commented Aug 26, 2025

Uh oh!

makai410 commented Aug 27, 2025

Uh oh!

makai410 commented Aug 27, 2025

Uh oh!

oli-obk commented Aug 29, 2025

Uh oh!

makai410 commented Aug 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

celinval commented Sep 10, 2025

Uh oh!

makai410 commented Sep 11, 2025

Uh oh!

makai410 commented Sep 12, 2025

Uh oh!

celinval commented Sep 13, 2025

Uh oh!

makai410 commented Sep 17, 2025

Uh oh!

celinval commented Sep 19, 2025

Uh oh!

makai410 commented Sep 24, 2025

Uh oh!

oli-obk commented Sep 24, 2025

Uh oh!

celinval commented Sep 24, 2025

Uh oh!

Uh oh!

makai410 commented Aug 18, 2025 •

edited

Loading