Cargo cross compilation support #14952
Open
+939
−255
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bonzini
added
cross
dependencies:cargo
bonzini
force-pushed
the
cargo-cross
branch
3 times, most recently
from
6104cd5 to
9cc66a2
Compare
bonzini
force-pushed
the
cargo-cross
branch
2 times, most recently
from
e7ea0cb to
17b9495
Compare
These could also be target dependencies, which would fail the test in _add_dependency Extracted from a patch by Xavier Claessens <[email protected]>
This avoids cloning the same repo multiple times, instead a single wrap can provide multiple cargo dependencies.
Do not create a single cargo.Interpreter for the whole execution, instead create one as soon as a Cargo.lock file is read and only make it last until that Cargo.lock has been processed. Signed-off-by: Paolo Bonzini <[email protected]>
Allow creating the cargo.Interpreter (and thus parsing the Cargo.lock file) even before 'rust' is added as a language. Signed-off-by: Paolo Bonzini <[email protected]>
When processing cargo subprojects, each Resolver was independently loading Cargo.lock files, leading to duplicate wraps and errors like 'Multiple wrap files provide dependency'. Instead, build the cargo.Interpreter as soon as a Cargo.lock file is found, and merge that Cargo.lock into the wrap resolver. There is no need anymore to build a separate resolver for the dependent packages, because their indirect dependencies are all listed in the parent lockfile.
This allows modifying the version number before the package is fetched. Extracted from a patch by Xavier Claessens <[email protected]>. Signed-off-by: Paolo Bonzini <[email protected]>
bonzini
force-pushed
the
cargo-cross
branch
3 times, most recently
from
b90d5b9 to
a0807a3
Compare
dnicolodi
reviewed
Sep 24, 2025
bonzini
force-pushed
the
cargo-cross
branch
2 times, most recently
from
0ec1437 to
49a5ac6
Compare
While developing I often got a confusing 'Neither a subproject directory nor a None.wrap file was found'. Replace it with a clear message that explains: 1. Which dependency couldn't be found 2. What versions are available in the root Cargo.lock 3. That this is likely a version conflict 4. How to fix it (update dependencies to match root Cargo.lock) Example: ERROR: Dependency 'socket2-0.4-rs' not found. Root Cargo.lock provides: socket2-0.5-rs, socket2-0.6-rs. This could be a Meson bug, please report it. I am actually not sure if this is really always a Meson bug. But to be safe, suggest reporting it until more experience is gained. Signed-off-by: Paolo Bonzini <[email protected]>
A dependency might specify a version that is compatible with the one in Cargo.lock, but not the exact same version. Whenever Cargo.lock contains the package, only use the pinned versions that can be used; update the version in the manifest so that the correct API level is used as well. Originally by Xavier Claessens <[email protected]>, but almost completely rewritten.
Extracted from a patch by Xavier Claessens <[email protected]>
With workspace support, backslashes can appear in subdir() invocations.
If the arguments form valid escapes (e.g. subdir(srcfoo), the
interpreter can be confused and fail in weird ways such as
os.makedirs(os.path.join(self.environment.build_dir, subdir), exist_ok=True)
File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\os.py", line 223, in makedirs
mkdir(name, mode)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'D:\\a\\1\\s\\b c09188aed5\\subprojects\\foo\\src\x0coo'
Signed-off-by: Paolo Bonzini <[email protected]>
Extracted from a patch by Xavier Claessens <[email protected]>
Extracted from a patch by Xavier Claessens <[email protected]>
Extracted from a patch by Xavier Claessens <[email protected]>
Cargo workspaces will use this to have a single subproject defining multiple crates.
Extracted from a patch by Xavier Claessens <[email protected]>
Extracted from a patch by Xavier Claessens <[email protected]>
Extracted from a patch by Xavier Claessens <[email protected]>, adjusted to make it compile with cargo.
This makes it possible to pass rust_args as a list. Signed-off-by: Paolo Bonzini <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
The lints table in Cargo.toml has a very human-targeted syntax. When building manifest.from_raw, flatten everything into a single list, prefixing the tool name to every warning option and sorting by priority. Signed-off-by: Paolo Bonzini <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>
Leave the cfg symbol free. Only one function is used. Signed-off-by: Paolo Bonzini <[email protected]>
Do not look anmore at proc_macro after init, keeping crate_type as the sole source of truth about the desired crate types.
Part of PackageState refers to the package itself, and part to the feature and dependency resolution process. Pull the latter outside, in preparation for having different features and dependencies for the build and the host machine. Signed-off-by: Paolo Bonzini <[email protected]>
extra_deps and extra_args may depend on the set of enabled features, so store it in a variable for use in meson/meson.build. Signed-off-by: Paolo Bonzini <[email protected]>
In the long term, dependencies that do not have a [lib] table will not create an invocation of override_dependency; do not expect there to be a handwritten meson.build that does it. In particular, this is the case for extra-dep-1-rs in the "rust/22 cargo subproject" test case, so change that to use the extra_deps mechanism instead to invoke the subproject. Signed-off-by: Paolo Bonzini <[email protected]>
self._add_dependency is idempotent so do it freely, and process DEP/FEATURE as a self._add_dependency followed by regular DEP?/FEATURE processing. Signed-off-by: Paolo Bonzini <[email protected]>
Storing dependency information in a separate object makes it easy to see if the package had already been found earlier. Make the work done by cargo.Interpreter less obscured, by removing many unnecessary calls to _prepare_package and doing the work in the caller after _dep_package or _fetch_package. Signed-off-by: Paolo Bonzini <[email protected]>
Make the PackageConfiguration a PerMachine object. This makes it possible to do dependency resolution separately for build- and/or host-side libraries. For now, all users only fill in or consume the host side, but that will change.
While this has no effect, it clarifies the logic with which cargo creates --extern options. This is good to have prior to the introduction of per-machine mangling of crate target names, which is needed to be able to build a target for both the build and the host machine. Signed-off-by: Paolo Bonzini <[email protected]>
Change the Meson code generator to take a MachineChoice and use it to generate "native" keyword arguments.
Signed-off-by: Paolo Bonzini <[email protected]>
_get_rust_dependency_name was less lenient in accepting targets containing invalid characters for a crate name. This made it impossible to link to those targets without using a dependency map. Use the same algorithm to do the conversion. Signed-off-by: Paolo Bonzini <[email protected]>
Allow multiple targets for the same crate name, which is useful when the same crate is used for both the host and the build machine. Signed-off-by: Paolo Bonzini <[email protected]>
When a crate name is compiled for both the build and the host machine, give it two different names to avoid conflicts. Signed-off-by: Paolo Bonzini <[email protected]>
override_dependency does not support overriding the same dependency twice, for both the build and the host machine. To work around this, use a separate dependency name. Signed-off-by: Paolo Bonzini <[email protected]>
Change the dependency resolution code to take a MachineChoice as an argument. Create build- and/or host-side configurations through the machines_from() method in Manifest.
bonzini
force-pushed
the
cargo-cross
branch
from
49a5ac6 to
0f7e572
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A fourth part of my Cargo interpreter fixes, adding cross compilation support for procedural macro crates and everything that they depend on.
A very early draft, I'll make sure it works on a few more projects before making it non-draft.Note that machine selection has the same issue as feature selection: if a subproject has a dependency on crate A for the host and another subproject has a dependency on crate B for the build machine, picking them one after another with
subprojectwill miss theoverride_dependencyfor the build machine and cross compilation will remain broken.Fixing this requires either a two-phase approach where all subprojects are added before feature resolution (and
meson.buildgeneration) proceeds; or triggering subproject compilation from aCargo.tomlin the toplevel projects instead of having Cargo interpretation just for subprojects (#14639).