Fix Rust static linking with +verbatim= on macos

[elmarco]

rustc -lstatic+verbatim=filename is not portable, given that linkers may not support passing unmodified filenames.

If we use -bundle, then also use -verbatim (default) and let the system linker find the library by adding prefix/suffix itself.

@bonzini @xclaesse

[xclaesse]

The current behaviour is very much intentional, I don't remember the exact reason but IIRC rustc does not name their static lib correctly on Windows so linker won't find them, or something...

Relevant links:

• rust: Linking with C static libraries in subdirectories fails #12484
• rust: Fix linking with C libraries in subdir #12485
• Windows: Inconsistent import lib naming rust-lang/rust#114013

I'm not saying current code is perfect, but I would be extremely cautious in changing it. I would start by challenging the claim "linkers may not support passing unmodified filenames". I find that hard to believe, please back that with evidence.

[xclaesse]

Trying to remember what was the story... I think this comment is the relevant part:

            # Pass native libraries directly to the linker with "-C link-arg"
            # because rustc's "-l:+verbatim=" is not portable and we cannot rely
            # on linker to find the right library without using verbatim filename.
            # For example "-lfoo" won't find "foo.so" in the case name_prefix set
            # to "", or would always pick the shared library when both "libfoo.so"
            # and "libfoo.a" are available.
            # See https://doc.rust-lang.org/rustc/command-line-arguments.html#linking-modifiers-verbatim.
            #
            # However, rustc static linker (rlib and staticlib) requires using
            # "-l" argument and does not rely on platform specific dynamic linker.

I think we want to bypass rustc as much as possible by passing our args directly to the linker with -Clink-arg= otherwise we get all kind of lib naming issues. However, when building rlib or staticlib we have to pass their deps as -l and in that case it's not the linker but rustc that need to resolve them. Maybe it's a rustc issue you're having?

[bonzini]

Yes, it's a rustc issue. More precisely: linkers may not support searching for an unmodified filename on the search path, which is currently the only way for rustc to record a dependency of a rlib or staticlib. In the case of ld64, it supports -lfoo.o but not -l:bar.a.

The breakage is self-evident from the code in rustc:

    fn link_staticlib_by_name(&mut self, name: &str, verbatim: bool, whole_archive: bool) {
        self.hint_static();
        let colon = if verbatim && self.is_gnu { ":" } else { "" };
        if !whole_archive {
            self.link_or_cc_arg(format!("-l{colon}{name}"));

where verbatim is completely ignored for non-GNU linkers. This is being worked on in rust-lang/rust#138753.

[xclaesse]

From private conversation with @elmarco, I think the context is this build error: https://gitlab.com/marcandre.lureau/qemu/-/jobs/11301499123.

I'm not sure what's the issue because as you can see Meson is not using +verbatim but instead -Clink-arg=libqemuutil.a to bypass rustc completely. But then there are a bunch of "-llibqemuutil.a" that magically appear on the linker command, those can only come from rustc I believe.

[bonzini]

I'm not sure what's the issue because as you can see Meson is not using +verbatim but instead -Clink-arg=libqemuutil.a to bypass rustc completely

This is an executable target, and therefore can use -Clink-arg=libqemuutil.a, but the same C static library is also passed to some of the crates that the executable depends on.

I suspect the -l was passed when creating the rlibs for these crates. If I understand correctly how rustc works -l+verbatim:libqemuutil.a directs rustc to record the command line argument into the rlib, and the breakage happens later when rustc translates it (incorrectly) into -llibqemuutil.a.

[bonzini]

This is not maintainable, as it relies on synchronizing the conditions between _link_library and here. Let me put together a quick refactoring, and then we can look at fixing on top of that.

[xclaesse]

I'm not sure we need to fix it in Meson at all, feels like completely a rustc issue. But I could be missing something of course.

[xclaesse]

I suspect the -l was passed when creating the rlibs for these crates. If I understand correctly how rustc works -l+verbatim:libqemuutil.a directs rustc to record the command line argument into the rlib, and the breakage happens later when rustc translates it (incorrectly) into -llibqemuutil.a.

That makes sense, yes. But then it seems to be a different issue than rust-lang/rust#138753, -l prefix should simply be removed in the +verbatim case.

[bonzini]

I agree it's a rustc issue in that verbatim is completely broken. :( But then we shouldn't use it.

[xclaesse]

@elmarco I would be curious to see build error of the unit test you wrote but without the fix. Then compare with rustc patched with rust-lang/rust#138753.

[bonzini]

@elmarco Can you test branch verbatim-avoidance of my fork on macOS, and/or rebase your fix on top of it?

[elmarco]

@elmarco Can you test branch verbatim-avoidance of my fork on macOS, and/or rebase your fix on top of it?

it seems to work

[bonzini]

@elmarco if you want, can you just push it here so we don't open a new PR for the same issue. Otherwise close this one and I will get the message!