Skip to content

Conversation

@kkysen
Copy link
Contributor

@kkysen kkysen commented Nov 3, 2025

So I previously thought that Hayroll can handle cross-OS transpilation, just not cross-arch transpilation, but that turned out to be more of a fluke. Hayroll can only substantively handle cross-target conditional compilation if c2rust transpile can cross-transpile. Cross-transpiling is fairly easy for cross-arch transpilation since we just need to pass the right sysroot. But when trying to cross-OS transpile, getting that sysroot is much more difficult, namely for macOS. zig cc is an excellent C cross-compiler and it bundles headers for basically every possible target, so we can use zig cc to find out what args we need to pass to libclangTooling to find the right headers to cross-transpile. c2rust transpile uses libclangTooling, and it can't go through zig cc directly, so we can call zig cc on an empty file with the -target we want and -### to print out the -cc1 commands, which contains the -isystems, -Ds, etc. that we want. Then we can append those cross-compiling args to c2rust transpile's extra clang args. This lets us successfully cross-transpile on most targets.

To start, I've added support for these Rust targets:

  • x86_64-unknown-linux-gnu
  • x86_64-apple-darwin
  • aarch64-unknown-linux-gnu
  • aarch64-apple-darwin
  • i686-unknown-linux-gnu
  • armv7-unknown-linux-gnueabihf
  • riscv64gc-unknown-linux-gnu

These are the targets we cared about in rav1d, for example. riscv64gc-unknown-linux-gnu isn't supported, though, since we don't handle a bunch of its builtin types yet, so while the tests are set up to handle it, we don't actually test on it.

While testing on the others, I discovered some tests had to be further disambiguated, as they were different on the other targets we're now testing as well. The UINTPTR_MAX from macros.c had to be moved to ptr-width-specific/macros.c since it depends on the ptr width. os-specific/rotate.c had to be moved to os-ptr-width-specific/rotate.c since it was different depending on the ptr width in addition to the OS. And vm_x86.c's transpiled code on x86 didn't compile, so that part is commented out for now (I'll open an issue).

For now, this is only done in snapshots.rs for the snapshot tests, as this was the easiest to start with. But we should be able to move a bunch of this into c2rust transpile itself with a --target option so that cross-transpilation is very easy and Hayroll can use it to handle cross-target conditional compilation. And our snapshot tests should work much better now, since cargo test -p c2rust-transpile runs all of them, and you don't have to wait for CI to tell you that a different platform is slightly different and update the snapshots manually.

Also, since we're now cross-transpiling on 6 different targets, cargo test -p c2rust-transpile takes significantly longer (about 6x longer). But c2rust_transpile::transpile is not parallelizable within the same process, so we can't easily parallelize this. This was an old issue I remember running into, and it's due to some weird way libclangTooling works and stores stuff internally.

kkysen added 20 commits November 2, 2025 18:54
…ED_RUST_TOOLCHAIN_TOML` and use it to deduplicate
Contrary to its naming, `-fsyntax-only` checks semantics, too.
It just doesn't emit anything, which is what we want.
`zig` isn't available through `apt`, but it is in PyPI,
so we can install it with `uv`.
For cross-arch compilation, getting the right headers, sysroot, etc. is not, too, difficult,
but for cross-os compilation, this is more difficult, as some OSes like macOS restrict these.
`zig cc` contains headers for all possible C cross-compilation targets, so we use it to look them up.
For platform-specific snapshots, test in groups of all targets that match that platform (arch/os).
For Rust targets, they seem to match the clang target,
at least for all of the targets we're testing now.
Some of these tests (`macros.c` and `rotate.c`) had to be further disambiguated
in `ptr-width-specific` and `os-ptr-width-specific`.

`vm_x86.c` also didn't compile after transpilation on `x86` due to an assembly error,
so that's now commented out for now with a TODO.
…ve all of the `dummy.c`s

The `dummy.c`s were needed because if an `insta::glob!` only found one path,
then the common prefix was the whole thing and removed too much.
@kkysen kkysen requested a review from fw-immunant November 3, 2025 08:10
Also, print the `python` and `zig` versions.
@thedataking thedataking removed the request for review from fw-immunant November 6, 2025 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants