[Flang][OpenMP] Move builtin .mod generation into runtimes

[Meinersbur]

Move building the .mod files from openmp/flang to openmp/flang-rt using a shared mechanism. Motivations to do so are:

1. Most modules are target-dependent and need to be re-compiled for each target separate, which is something the LLVM_ENABLE_RUNTIMES system already does. Prime example is iso_c_binding.mod which encodes the target's ABI. Most other modules have #ifdef-enclosed code as well.

2. CMake has support for Fortran that we should use. Among other things, it automatically determines module dependencies so there is no need to hardcode them manually.

3. It allows using Fortran itself to implement Flang-RT. Currently, only iso_fortran_env_impl.f90 emits object files that are needed by Fortran applications ([flang] Linker for non-constant accesses to kind arrays (integer_kind, logical_kind, real_kind) #89403). The workaround of [flang][runtime] Build ISO_FORTRAN_ENV to export kind arrays as linkable symbols #95388 could be reverted.

Some new dependencies come into play:

• openmp depends on flang-rt for building lib_omp.mod and lib_omp_kinds.mod. Currently, if flang-rt is not found then the modules are not built
• check-flang depends on flang-rt: If not found, the majority of tests are disabled. If not building in a bootstrpping build, the location of the module files can be pointed to using -DFLANG_INTRINSIC_MODULES_DIR=<path>, e.g. in a flang-standalone build. Alternatively, the test needing any of the instrinsic modules could be marked with REQUIRES: flangrt-modules which would affect ~217 files.
• check-flang depends on openmp: Not a change; tests requiring lib_omp.mod and lib_omp_kinds.mod those are already marked with openmp_runtime.

As intrinsic are now specific to the target, their location os moved from include/flang to <resource-dir>/finclude/<triple>. The mechnism to compute the location have been moved from flang-rt (previously used to compute the location of libflang_rt.*.a) to common locations in cmake/GetToolchainDirs.cmake and runtimes/CMakeLists.txt so they can be used by both, openmp and flang-rt. Potentially the mechnism could also be shared by other libraries such as compiler-rt.

finclude was chosen because gfortran uses it as well and avoids misuse such as #include <flang/iso_c_binding.mod>. The search location is now determined by ToolChain in the driver, instead of by the frontend. The now the driver adds -fintrinsic-module-path for that location to the frontend call (Just like gcc does). -fintrinsic-module-path had to be fixed for this because ironically it was only added to searchDirectories, but not intrinsicModuleDirectories_. Since the driver determines the location, tests invoking flang -fc1 and bbc must also be passed the location by llvm-lit. This works like llvm-lit does for finding the include dirs for Clang using -print-file-name=....

Related PRs:

• [Flang] Solved issue with inline compiler directive #143699 (allows removing the workaround in flang-rt/lib/runtime/ieee_arithmetic.f90)
• [Flang] Add -fintrinsic-modules-path= alias #149417 (allows removing the SHELL: workaround)
• [OpenMP] Fix runtimes default build #149871 (fixes a build configuration issue)

[jhuber6]

What's the main limitation here? If this is just a file dependency it should be identical to how all the OpenMP tests depend on omp.h being in the resource directory. IMHO this is trivial if we do a runtimes build, since we can just require that openmp;flang-rt are in the same toolchain, which then gives us well defined access to openmp's CMake targets so long as it's listed before flang-rt.

[Meinersbur]

While I appreciate the review, it is not yet in the state that warants one. It is still in an experimentation stage, so I did not yet care about formatting. There are also a lot of changes in here that will eventually not be needed.

Goals are:

1. Currently modules files are expected at $prefix/include/flang/*.mod where prefix is the parent of bin where flang is located. It should be in $prefix/lib/clang/finclude/<triple>/*.h, i.e. the resource directory since mod-files are specific to the version of flang, and distinct for each target triple since the mod files can be different for each target. Necessary for cross-compilation. In addition to the CMake code, the flang driver code needs to change as well because it hardcodes $path/../include/flang.

2. Use cmake to build the module files within the flang-rt/ runtime. The LLVM_ENABLE_RUNTIMES system handles which target triples to build and Flang being available. CMake should care about the build dependencies. Have to change the driver again to tell it where to emit the module files.

3. Use the same mechanism as above to build omp_lib.mod and omp_lib_kinds.mod, but in the openmp/ runtime. Since in the same CMake builddir, CMake will ensure dependencies already.

4. This means flang-rt (and openmp) is necessary to be compiled before running flang's tests which require those module files. Flang's OpenMP tests already require openmp's modules to be compiled, it will be no different to flang-rt's builtin modules.

Sounds relatively simple, but there have been many small issues, starting with CMake's misspelling of CMAKE_Fortran_BUILDING_INSTRINSIC_MODULES.

[DanielCChen]

1. It should be in $prefix/lib/clang/finclude/<triple>/*.h

Just want to make sure: Should it be $prefix/lib/clang/${LLVM_VERSION_MAJOR}/finclude/<triple>/*.mod?

[Meinersbur]

Just want to make sure: Should it be $prefix/lib/clang/${LLVM_VERSION_MAJOR}/finclude/<triple>/*.mod?

That is correct, I forgot the version number that is part of the resource directory.

[github-actions]

✅ With the latest revision this PR passed the Python code formatter.

[Meinersbur]

What's the main limitation here? If this is just a file dependency it should be identical to how all the OpenMP tests depend on omp.h being in the resource directory.

omp.h is created by configure_file at configure time. No dependency other than runtimes-configure needed.

IMHO this is trivial if we do a runtimes build, since we can just require that openmp;flang-rt are in the same toolchain,

With toolchain you mean a bootstrapping build with LLVM_ENABLE_RUNTIMES=openmp;flang-rt ? Don't forget the users of a Flang-standalone build (cmake -S <llvm-project>/flang).

which then gives us well defined access to openmp's CMake targets so long as it's listed before flang-rt.

check-flang (LLVM_ENABLE_PROJECTS=flang) needs access to libomp.mod (LLVM_ENABLE_RUNTIMES=openmp) and the flang modules as well to work.

[vzakhari]

Is this RUN command supposed to be invoked without the intrinsic module flag?

[Meinersbur]

With this PR, the default intrinsic path is now determined by the driver, not by the frontend. To emulate driver behavior, the test needs to add the default intrinsics path manually.

Alternatively, this test could invoke the driver instead. It's in the test/Driver directory after all.

[vzakhari]

Oh, I see. Thanks!

[Meinersbur]

Some additional notes:
%flang_bare actually is the driver (i.e. not flang -fc1, have to change the comments). The problem is that the intrinsic modules may not be in the resource directory, but in the directory defined with cmake -DFLANG_INTRINSIC_MODULES_DIR=<path>, as otherwise it would not be possible run check-flang tests in a Flang-standalone build (that does not build flang-rt, hence no intrinsic modules in the default location).
FLANG_INTRINSIC_MODULES_DIR is implemented by substituting flang with flang -fintrinsic-modules-path=${FLANG_INTRINSIC_MODULES_DIR}, i.e. always comes first in the command line. This means it will always have priority over -fintrinsic-modules-path=%S/Inputs/. This is incompatible with the second RUN line that depends on the files in %S/Inputs/ being picked up.
Hence, this test adds -fintrinsic-modules-path=${FLANG_INTRINSIC_MODULES_DIR} (if necessary) after -fintrinsic-modules-path=%S/Inputs/, since with a %flang substitution it is not possible to put it behind all other flags. %flang_bare avoids that -fintrinsic-modules-path=${FLANG_INTRINSIC_MODULES_DIR} is (also) added before %S/Inputs/.

One could redesign this test, i.e. that it expects the default intrinsic modules to be matched instead of the custom ones, not relying on priority.

[klausler]

I don't understand how flang tests that depend on fortran_type_info.mod are going to work.

[Meinersbur]

@klausler
If building using a bootstrapping configuration (-DLLVM_ENABLE_RUNTIMES=flang-rt, implicitly added when -DLLVM_ENABLE_PROJECTS=flang), the mod files are automatically put into the Clang/Flang resource directory as a dependency of check-flang. If explictly disabled using -DFLANG_ENABLE_FLANG_RT=OFF or in a Flang-standalone build (that currently does not support LLVM_ENABLE_RUNTIMES), Flang-RT can be compiled separately and FLANG_INTRINSIC_MODULES_DIR pointed to it. lit.cfg.py will add the path via -fintrinsic-module-path to each flang/%flang_fc1 call. If FLANG_INTRINSIC_MODULES_DIR is not set then tests in test/ will be skipped.

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[Meinersbur]

ping

[Meinersbur]

Should we choose a different module path than finclude to avoid accidental confusion between flang and gfortran modules files, e.g. fmod?