Unable to crosscompile project with GCC

[harraiken]

Hello,

I would like to build llvm-mingw for Windows under Linux using mingw-w64-gcc. Unfortunately, every time I run the build-llvm.sh script I get a bunch of 'undefined reference' errors, e.g. when linking libLTO.dll, e.g.:

undefined reference to llvm::StringMapImpl::hash(llvm::StringRef)'

This happens every time I execute the command: ./build-llvm.sh --host=x86_64-w64-mingw32 /LLVM_MINGW. It looks like the following command is used by the script to build the toolchain:

cmake -DCMAKE_INSTALL_PREFIX=/LLVM-MINGW -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_ENABLE_PROJECTS=clang;lld;lldb;clang-tools-extra -DLLVM_TARGETS_TO_BUILD=ARM;AArch64;X86;NVPTX -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON -DLLVM_LINK_LLVM_DYLIB=ON -DLLVM_TOOLCHAIN_TOOLS=llvm-ar;llvm-ranlib;llvm-objdump;llvm-rc;llvm-cvtres;llvm-nm;llvm-strings;llvm-readobj;llvm-dlltool;llvm-pdbutil;llvm-objcopy;llvm-strip;llvm-cov;llvm-profdata;llvm-addr2line;llvm-symbolizer;llvm-windres;llvm-ml;llvm-readelf;llvm-size;llvm-cxxfilt;llvm-lib -DLLVM_HOST_TRIPLE=x86_64-w64-mingw32 -DLLVM_BUILD_INSTRUMENTED=OFF -DCMAKE_C_COMPILER=x86_64-w64-mingw32-gcc -DCMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++ -DCMAKE_SYSTEM_PROCESSOR=x86_64 -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_RC_COMPILER=x86_64-w64-mingw32-windres -DCMAKE_FIND_ROOT_PATH=/usr/x86_64-w64-mingw32 -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY ..

CMAKE_FIND_ROOT_PATH correctly points to /usr/x86_64-w64-mingw32, where the GCC-based toolchain is installed. This directory contains the bin/, include/ and lib/ directories containing the mingw headers and libraries, among other things.

From what I have seen, llvm-mingw itself is used to build the Windows version. However, I already have GCC-based binaries of mingw-w64 installed and I wanted to use them so as not to compile LLVM twice. Finally, more error output:

/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$_ZTVN4llvm2cl11OptionValueIcEE[_ZTVN4llvm2cl11OptionValueIcEE]+0x18): undefined reference to llvm::cl::GenericOptionValue::anchor()' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$_ZTVN4llvm2cl11OptionValueIbEE[_ZTVN4llvm2cl11OptionValueIbEE]+0x18): undefined reference to llvm::cl::GenericOptionValue::anchor()'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$_ZTV24LTOToolDiagnosticHandler[_ZTV24LTOToolDiagnosticHandler]+0x28): undefined reference to llvm::DiagnosticHandler::isAnalysisRemarkEnabled(llvm::StringRef) const' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$_ZTV24LTOToolDiagnosticHandler[_ZTV24LTOToolDiagnosticHandler]+0x30): undefined reference to llvm::DiagnosticHandler::isMissedOptRemarkEnabled(llvm::StringRef) const'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$_ZTV24LTOToolDiagnosticHandler[_ZTV24LTOToolDiagnosticHandler]+0x38): undefined reference to llvm::DiagnosticHandler::isPassedOptRemarkEnabled(llvm::StringRef) const' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$_ZTV24LTOToolDiagnosticHandler[_ZTV24LTOToolDiagnosticHandler]+0x40): undefined reference to llvm::DiagnosticHandler::isAnyRemarkEnabled() const'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm2cl6parserIbEE[.refptr._ZTVN4llvm2cl6parserIbEE]+0x0): undefined reference to vtable for llvm::cl::parser<bool>' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm2cl6parserIcEE[.refptr._ZTVN4llvm2cl6parserIcEE]+0x0): undefined reference to vtable for llvm::cl::parser'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm18raw_string_ostreamE[.refptr._ZTVN4llvm18raw_string_ostreamE]+0x0): undefined reference to vtable for llvm::raw_string_ostream' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm27DiagnosticPrinterRawOStreamE[.refptr._ZTVN4llvm27DiagnosticPrinterRawOStreamE]+0x0): undefined reference to vtable for llvm::DiagnosticPrinterRawOStream'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm2cl3optIcLb0ENS0_6parserIcEEEE[.refptr._ZTVN4llvm2cl3optIcLb0ENS0_6parserIcEEEE]+0x0): undefined reference to vtable for llvm::cl::opt<char, false, llvm::cl::parser<char> >' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm2cl6OptionE[.refptr._ZTVN4llvm2cl6OptionE]+0x0): undefined reference to vtable for llvm::cl::Option'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm2cl17basic_parser_implE[.refptr._ZTVN4llvm2cl17basic_parser_implE]+0x0): undefined reference to vtable for llvm::cl::basic_parser_impl' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm2cl3optIbLb0ENS0_6parserIbEEEE[.refptr._ZTVN4llvm2cl3optIbLb0ENS0_6parserIbEEEE]+0x0): undefined reference to vtable for llvm::cl::opt<bool, false, llvm::cl::parser >'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(lto.cpp.obj):lto.cpp:(.rdata$.refptr._ZTVN4llvm17DiagnosticHandlerE[.refptr._ZTVN4llvm17DiagnosticHandlerE]+0x0): undefined reference to vtable for llvm::DiagnosticHandler' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x5): undefined reference to LLVMInitializeARMTargetInfo'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0xa): undefined reference to LLVMInitializeAArch64TargetInfo' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0xf): undefined reference to LLVMInitializeX86TargetInfo'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x14): undefined reference to LLVMInitializeNVPTXTargetInfo' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x19): undefined reference to LLVMInitializeARMTargetMC'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x1e): undefined reference to LLVMInitializeAArch64TargetMC' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x23): undefined reference to LLVMInitializeX86TargetMC'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x28): undefined reference to LLVMInitializeNVPTXTargetMC' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x2d): undefined reference to LLVMInitializeARMAsmParser'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x32): undefined reference to LLVMInitializeAArch64AsmParser' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x37): undefined reference to LLVMInitializeX86AsmParser'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x3c): undefined reference to LLVMInitializeARMDisassembler' /usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x41): undefined reference to LLVMInitializeAArch64Disassembler'
/usr/bin/x86_64-w64-mingw32-ld: CMakeFiles/LTO.dir/objects.a(LTODisassembler.cpp.obj):LTODisassembler.cpp:(.text$lto_initialize_disassembler+0x4b): undefined reference to `LLVMInitializeX86Disassembler'

Any suggestions would be appreciated.

[mstorsjo]

It would seem like building LLVM with a dylib for a mingw target, with GCC, currently is broken. (When building with dylib, all the executables are linked against a shared libLLVM-<version>.dll and libclang-cpp.dll, vastly reducing the binary size of the distribution.) This seems to have broken since llvm/llvm-project@c87d198; since this commit, JITLoaderGDB.cpp.obj, when compiled with GCC, contains dllexport directives for a couple symbols. This means that libLLVM-<version>.dll only ends up exporting those symbols, instead of exporting all symbols. This doesn't happen when compiled with Clang, for separate reasons. If libLLVM has object files with dllexport directives, we'd might need to have something like https://github.com/llvm/llvm-project/blob/llvmorg-20.1.7/clang/tools/clang-shlib/CMakeLists.txt#L63-L70, which adds -Wl,--export-all-symbols, to the build of libLLVM-<version>.dll as well.

Even before this breakage, building a dylib for a mingw target with GCC wasn't very well supported. Each DLL has a max limit of 64k exported symbols, and in recent years, libLLVM would end up exceeding this. To avoid this issue, Clang learnt to translate __attribute__((visibility("hidden"))), -fvisibility=hidden and similar ones, to a new directive, that excludes symbols from autoexporting - this makes the limit on exported symbols much more bearable. As GCC doesn't support this, building a dylib with GCC hasn't been viable for some time (I had to disable it in my nightly build in da480f0 in Feb 2024) - so that's why this configuration has bitrotten. But even if we wouldn't have this issue with the stray dllexport attributes, the DLL would probably exceed the 64k symbol limit anyway.

So TL;DR, try passing --disable-dylib to build-llvm.sh, and this may work better.

---

Then secondly, as to your overall objective. If you just want a build of the plain LLVM tools on Windows, the above should be enough.

But if you're aiming to building a regular setup of llvm-mingw for Windows, on Linux this way, there are a number of other uncommon details you'd need to sort out.

Once you have built LLVM for Windows, you'd need the mingw runtimes as well. Normally when building llvm-mingw for Windows with a llvm-mingw build on Linux, we just copy over the mingw runtimes from the Linux llvm-mingw toolchain. In your case you don't have that; you could try copying some of the mingw runtime files from the GCC mingw setup on Linux. The import libraries will be a different kind of import library than what llvm-mingw normally uses, but it may work. But then you'd also need to transplant the libgcc and libstdc++ files from the GCC install, instead of the compiler-rt and libc++ libraries that llvm-mingw usually would use. (And you'd need to edit the clang config files to no longer pass -rtlib=compiler-rt -unwindlib=libunwind -stdlib=libc++, and make it find libgcc/libtsdc++ instead.) And you can't easily build these runtimes here anyway, as you don't have a Clang setup that you actually can execute on Linux.

So, if you actually want to build a regular llvm-mingw for Windows, on Linux, then do build llvm-mingw natively on Linux first - otherwise there are lots of things you'd need to reassemble yourself. (Or just download the preexisting binaries - there's also nightly builds at https://github.com/mstorsjo/llvm-mingw/releases/nightly.)

[harraiken]

Thank you for your reply and help so far.
I have managed to finally build LLVM. As you suggested, the --disable-dylib parameter solved the problem. Unfortunately you were also right that the binaries are much larger - LLVM itself takes up 2.2GB.

Thanks to the information you provided, I started to delve deeper into this topic.
Following the Arch Linux wiki, I decided to give it one more try. According to the information found there: "one can simply use clang specifying an *-w64-mingw32 platform triple as target. Since the regular clang package does not contain the compiler runtime libraries for Windows targets one needs to compile them separately or use the runtime provided by mingw-w64-gcc."
I have installed all necessary tools and launched build-llvm.sh script with --host=x86_64-w64-mingw32 --with-clang /LLVM_MINGW what led to execute cmake with the following set of parameters:

cmake '' -DCMAKE_INSTALL_PREFIX=/LLVM_MINGW -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=OFF '-DLLVM_ENABLE_PROJECTS=clang;lld;lldb;clang-tools-extra' '-DLLVM_TARGETS_TO_BUILD=ARM;AArch64;X86;NVPTX' -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON -DLLVM_LINK_LLVM_DYLIB=ON '-DLLVM_TOOLCHAIN_TOOLS=llvm-ar;llvm-ranlib;llvm-objdump;llvm-rc;llvm-cvtres;llvm-nm;llvm-strings;llvm-readobj;llvm-dlltool;llvm-pdbutil;llvm-objcopy;llvm-strip;llvm-cov;llvm-profdata;llvm-addr2line;llvm-symbolizer;llvm-windres;llvm-ml;llvm-readelf;llvm-size;llvm-cxxfilt;llvm-lib' -DLLVM_HOST_TRIPLE=x86_64-w64-mingw32 -DLLVM_BUILD_INSTRUMENTED=OFF -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_USE_LINKER=lld -DCMAKE_C_COMPILER_TARGET=x86_64-w64-mingw32 -DCMAKE_CXX_COMPILER_TARGET=x86_64-w64-mingw32 -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_RC_COMPILER=x86_64-w64-mingw32-windres -DLLVM_NATIVE_TOOL_DIR=/usr/sbin -DCMAKE_FIND_ROOT_PATH=/usr/x86_64-w64-mingw32 -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY ..

It correctly discovers path to gcc provided libs and headers under /usr/x86_64-w64-mingw32. However, this time compilation fails with:

ld.lld: error: undefined symbol: pthread_mutex_destroy

referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(__dtor__ZL17ErrorHandlerMutex)
referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(__dtor__ZL25BadAllocErrorHandlerMutex)
referenced by libLLVMSupport.a(CrashRecoveryContext.cpp.obj):(__dtor__ZZN12_GLOBAL__N_128getCrashRecoveryContextMutexEvE25CrashRecoveryContextMutex)

ld.lld: error: undefined symbol: pthread_mutex_lock

referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(llvm::install_fatal_error_handler(void ()(void, char const*, bool), void*))
referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(llvm::remove_fatal_error_handler())
referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(llvm::report_fatal_error(llvm::Twine const&, bool))
referenced 26 more times

ld.lld: error: undefined symbol: pthread_mutex_unlock

referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(llvm::install_fatal_error_handler(void ()(void, char const*, bool), void*))
referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(llvm::remove_fatal_error_handler())
referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(llvm::report_fatal_error(llvm::Twine const&, bool))
referenced 26 more times

ld.lld: error: undefined symbol: pthread_mutex_init

referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(_GLOBAL__sub_I_ErrorHandling.cpp)
referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(_GLOBAL__sub_I_ErrorHandling.cpp)
referenced by libLLVMSupport.a(CrashRecoveryContext.cpp.obj):(llvm::CrashRecoveryContext::Enable())
referenced 1 more times

ld.lld: error: undefined symbol: std::__once_callable

referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::NamedRegionTimer::NamedRegionTimer(llvm::StringRef, llvm::StringRef, llvm::StringRef, llvm::StringRef, bool))
referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::NamedRegionTimer::NamedRegionTimer(llvm::StringRef, llvm::StringRef, llvm::StringRef, llvm::StringRef, bool))
referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::TimerGroup::constructForStatistics())
referenced 2 more times

ld.lld: error: undefined symbol: std::__once_call

referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::NamedRegionTimer::NamedRegionTimer(llvm::StringRef, llvm::StringRef, llvm::StringRef, llvm::StringRef, bool))
referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::NamedRegionTimer::NamedRegionTimer(llvm::StringRef, llvm::StringRef, llvm::StringRef, llvm::StringRef, bool))
referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::TimerGroup::constructForStatistics())
referenced 1 more times

ld.lld: error: undefined symbol: pthread_once

referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::NamedRegionTimer::NamedRegionTimer(llvm::StringRef, llvm::StringRef, llvm::StringRef, llvm::StringRef, bool))
referenced by libLLVMSupport.a(Timer.cpp.obj):(llvm::TimerGroup::constructForStatistics())

Do you think it is possible to use libraries provided by mingw-w64-gcc and clang compiler to build Windows version of LLVM under Linux?

[mstorsjo]

Thanks to the information you provided, I started to delve deeper into this topic. Following the Arch Linux wiki, I decided to give it one more try. According to the information found there: "one can simply use clang specifying an *-w64-mingw32 platform triple as target. Since the regular clang package does not contain the compiler runtime libraries for Windows targets one needs to compile them separately or use the runtime provided by mingw-w64-gcc." I have installed all necessary tools and launched build-llvm.sh script with --host=x86_64-w64-mingw32 --with-clang /LLVM_MINGW what led to execute cmake with the following set of parameters:

[...]

It correctly discovers path to gcc provided libs and headers under /usr/x86_64-w64-mingw32. However, this time compilation fails with:

ld.lld: error: undefined symbol: pthread_mutex_destroy

referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(__dtor__ZL17ErrorHandlerMutex)
referenced by libLLVMSupport.a(ErrorHandling.cpp.obj):(__dtor__ZL25BadAllocErrorHandlerMutex)
referenced by libLLVMSupport.a(CrashRecoveryContext.cpp.obj):(__dtor__ZZN12_GLOBAL__N_128getCrashRecoveryContextMutexEvE25CrashRecoveryContextMutex)

I would think this is because your GCC uses pthreads for its "thread model". (Background: There are primarily two different settings, win32 and posix. Previously, you would only get full support for C++11 threads if you used posix - that's why this is commonly the default. Since the last couple of GCC releases, you also get full C++11 thread support with the win32 thread setting, without the extra omnipresent dependency on pthreads.) When GCC is built with pthreads for its thread model, GCC itself adds pthreads as one of the always automatically linked libs. Clang, interoperating with a GCC installation that it just finds on disk, doesn't know which variant of GCC it encounters, and doesn't link pthreads automatically.

So if you want to rip things up from the ground, you may consider rebuilding GCC and its runtimes with the win32 thread model. If not, you may try adding something like -DCMAKE_EXE_LINKER_FLAGS=-lpthread -DCMAKE_SHARED_LINKER_FLAGS=-lpthread.

Also, if using Clang for a mingw target, with GCC's libstdc++, you may need to build with -femulated-tls if the code is using std::once anywhere, as GCC also defaults to a different TLS form than Clang, and this is exposed in the DLL interface of libstdc++.

[longnguyen2004]

Sorry for hijacking this issue, but

Since the last couple of GCC releases, you also get full C++11 thread support with the win32 thread setting, without the extra omnipresent dependency on pthreads.

That's interesting 👀 is it using mcfgthread or an entirely new implementation?

[mati865]

@longnguyen2004 it's the win32 variant. Until GCC 13 it provided no C++11 threading support but was finally implemented by https://gcc.gnu.org/pipermail/libstdc++-cvs/2022q4/039044.html

[mstorsjo]

@longnguyen2004 it's the win32 variant. Until GCC 13 it provided no C++11 threading support but was finally implemented by https://gcc.gnu.org/pipermail/libstdc++-cvs/2022q4/039044.html

Btw, have you considered switching the GCC setups in msys2 (mingw32/mingw64/ucrt64) over to the win32 thread model? I guess it could cause some amount of breakage for cases that implicitly depend on pthreas, but the clang64/clangarm64 environments probably already should have sorted out that to some extent?

[mati865]

I do not participate in GCC/Binutils related discussions or development for quite some time, so I have no idea, unfortunately.

[gerekon]

So, if you actually want to build a regular llvm-mingw for Windows, on Linux, then do build llvm-mingw natively on Linux first - otherwise there are lots of things you'd need to reassemble yourself. (Or just download the preexisting binaries - there's also nightly builds at https://github.com/mstorsjo/llvm-mingw/releases/nightly.)

@mstorsjo I am trying to make use of llvm-mingw for our builds (https://github.com/espressif/esp-llvm-embedded-toolchain). Some of our users needs distro with dynamic libs to be used in their upper level frameworks. So we need to build a set of DLLs for Windows. For LLVM -19 based release it build was ok with MINGW+GCC. But start failing on LLVM 20. Currently I am trying to pass pipelines for our dev branch rebased on LLVM 20.1.1.

Just to be sure that I am going in right direction...
If I use one of your latest releases do I still need to mark LLVM APIs with LLVM_ABI to make them exposed by Windows DLL? Or it should work as is?

[mstorsjo]

If I use one of your latest releases do I still need to mark LLVM APIs with LLVM_ABI to make them exposed by Windows DLL? Or it should work as is?

It should work as is; llvm-mingw itself is built for Windows with dylibs enabled - you shouldn't need to touch the LLVM_ABI markings.

[gerekon]

Hmm... I still do see undef refs when linking bin/libclang-cpp.dll

ld.lld: error: undefined symbol: llvm::SmallVectorBase<unsigned long long>::grow_pod(void*, unsigned long long, unsigned long long)
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Attributes.cpp.obj:(normalizeName(clang::IdentifierInfo const*, clang::IdentifierInfo const*, clang::AttributeCommonInfo::Syntax))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Attributes.cpp.obj:(normalizeName(clang::IdentifierInfo const*, clang::IdentifierInfo const*, clang::AttributeCommonInfo::Syntax))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Attributes.cpp.obj:(normalizeName(clang::IdentifierInfo const*, clang::IdentifierInfo const*, clang::AttributeCommonInfo::Syntax))
>>> referenced 1502 more times

ld.lld: error: undefined symbol: llvm::StringMapImpl::hash(llvm::StringRef)
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Builtins.cpp.obj:(clang::Builtin::Context::initializeBuiltins(clang::IdentifierTable&, clang::LangOptions const&))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Builtins.cpp.obj:(clang::IdentifierTable::get(llvm::StringRef))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Builtins.cpp.obj:(clang::Builtin::TargetFeatures::getAndFeatures(llvm::StringRef))
>>> referenced 1321 more times

ld.lld: error: undefined symbol: llvm::StringMapImpl::FindKey(llvm::StringRef, unsigned int) const
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Builtins.cpp.obj:(clang::Builtin::Context::initializeBuiltins(clang::IdentifierTable&, clang::LangOptions const&))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Builtins.cpp.obj:(clang::Builtin::TargetFeatures::getAndFeatures(llvm::StringRef))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Builtins.cpp.obj:(clang::Builtin::TargetFeatures::getAndFeatures(llvm::StringRef))
>>> referenced 239 more times

Ok. Thanks. I will dig into this deeper.

[mstorsjo]

Hmm... I still do see undef refs when linking bin/libclang-cpp.dll

ld.lld: error: undefined symbol: llvm::SmallVectorBase<unsigned long long>::grow_pod(void*, unsigned long long, unsigned long long)
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Attributes.cpp.obj:(normalizeName(clang::IdentifierInfo const*, clang::IdentifierInfo const*, clang::AttributeCommonInfo::Syntax))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Attributes.cpp.obj:(normalizeName(clang::IdentifierInfo const*, clang::IdentifierInfo const*, clang::AttributeCommonInfo::Syntax))
>>> referenced by tools/clang/lib/Basic/CMakeFiles/obj.clangBasic.dir/Attributes.cpp.obj:(normalizeName(clang::IdentifierInfo const*, clang::IdentifierInfo const*, clang::AttributeCommonInfo::Syntax))
>>> referenced 1502 more times

Ok. Thanks. I will dig into this deeper.

That's odd - is this in a fully clean build?

To be clear; if you take a release of llvm-mingw and add it to your PATH, you should be able to do ./build-llvm.sh --full-llvm /path/to/install/into --host=x86_64-w64-mingw32 using the scripts from llvm-mingw, and get a build of LLVM for that target, using dylibs. This doesn't do much more than a specific cmake configuration and builds that, but it should be a cmake configuration that does build successfully - a setup that should be reasonably easy for you to reproduce.

[gerekon]

That's odd - is this in a fully clean build?

Maybe there is problem in our build system. Checking it.

It should work as is;

One more question. libclang-cpp.dll uses a lot of functions from libllvm.dll, so all of those funcs should be exported. Right? The most of the LLVM symbols (used by libclang-cpp.dll) are not marked with LLVM_ABI (only few classes are marked). So I wonder how it is supposed to work w/o LLVM_ABI?
clang-shlib does this by adding --export-all-symbols, but there is no such option for llvm-shlib.

[mstorsjo]

One more question. libclang-cpp.dll uses a lot of functions from libllvm.dll, so all of those funcs should be exported. Right? The most of the LLVM symbols (used by libclang-cpp.dll) are not marked with LLVM_ABI (only few classes are marked). So I wonder how it is supposed to work w/o LLVM_ABI? clang-shlib does this by adding --export-all-symbols, but there is no such option for llvm-shlib.

Indeed, llvm-shlib perhaps also should use --export-all-symbols for consistency. So far, it works when built with mingw-clang, as the LLVM_ABI markings don't expand to dllexport, but into __attribute__((visibility("default"))) for mingw-clang (or something along those lines). So as long as there's no symbols marked explicitly with dllexport, all symbols get exported, but if there are explicit dllexported ones, it would need --export-all-symbols.

(But all of this is somewhat transitionary - once the LLVM_ABI and similar markings are complete, the intent is that we should be able to use explicit dllexport/dllimport for it all. That's a work in progress though, so for mingw-clang, it doesn't use dllexport yet. And once that's in place, we no longer should need --export-all-symbols at all.)

[gerekon]

Thank you very much for the details. You confirmed my thoughts on this. So looks like in my build env some symbols get exported explicitily with dllexport therefore __attribute__((visibility("default"))) does not work for others anymore.

[mstorsjo]

Thank you very much for the details. You confirmed my thoughts on this. So looks like in my build env some symbols get exported explicitily with dllexport therefore __attribute__((visibility("default"))) does not work for others anymore.

Yes, exactly. Mingw builds with clang currently use the same approach as on Unix platforms - using __attribute__((visibility("hidden"))), __attribute__((visibility("default"))) together with -fvisibility=hidden, and -fvisibility-inlines-hidden, to reduce the number of exported symbols - the approach where everything is implicitly exported by default, unless explicitly hidden. This is supported only by Clang, not by GCC. (But both LLD and binutils ld.bfd supports the directives it produces.) But once there's an explicit dllexport involved, that switches to the other mode with only explicit exports.