[Mac] MacOS build documentation + Wheels#14340
[Mac] MacOS build documentation + Wheels#14340takeshimg92 wants to merge 8 commits intoKratosMultiphysics:masterfrom
Conversation
- Updated build_wheel.py to detect Homebrew Python installations - Expanded Darwin platform config to support Python 3.9-3.14 (was 3.9 only) - Fixed getPythonInterpreter() to properly parse version strings (314 -> 3.14) - Updated darwin/configure.sh with Homebrew and CI environment detection - Added ARM64 support: CPU flags now conditional on architecture - Added mac_build_wheel script for convenient wheel building - Updated INSTALL.md with wheel building documentation - Configure script now auto-detects Homebrew LLVM, Boost paths - Paths configurable via environment variables for CI/local flexibility Fixes compatibility issues with existing PyPI wheels that use GCC and hardcoded paths.
- Updated mac_build_wheel to actually build wheels using hatchling - Sets KRATOS_VERSION environment variable for hatch build hook - Creates proper wheel staging directory structure - Auto-detects Python installations and sets up virtual environments - Verified Clang++/Homebrew build approach works in isolated environments - Successfully tested Python 3.14 module in clean venv Verification: ✅ Built binaries work perfectly in isolated Python environments ✅ OpenMP support detected (8 threads) ✅ All modules import correctly ✅ Model instantiation works without errors ✅ No GCC library dependency issues (unlike PyPI wheels)
Replace hardcoded user path with os.path.expanduser to make it portable across different users while maintaining environment variable override capability.
7a5c71f to
6d5f322
Compare
Extract Python version from executable instead of hardcoding to 314. This ensures the install path matches the Python version being built for.
roigcarlo
left a comment
There was a problem hiding this comment.
Hi @takeshimg92
I don't fully understand the changes in the wheel directory. This is something that we use internally to generate redistributables.
In general for mac our policy is to give support only for the most recent version of mac for M4 processors, as we lack hardware to target intel and linux releases are compatible.
The aim of the build_wheel.py is to provide the configure.sh with the correct variables, so fetching them directly from env variables makes little sense there.
Likewise, I would like to avoid changes in build_wheel.py regarding os variables, as paths are fixed to our environment, building python version as its targeting the only version available in our hardware and targets python versions for pretty much the same reason.
In part all this is caused because while in Windows and Linux our pipeline is docker based, Apple does not provide support to use MacOS containers anywhere (we can run linux containers on mac, linux on win, win on win, linunx on linux, but not MacOS on anything, not even in mac itself), so the scripts may seem counter-intuitive as I try to maintain symmetry with the win and linux pipelines.
Rest of the changes are ok to me as long as @matekelemen or any other Mac users can verify, but please I would kindly ask you to revert the ones in the wheel section
@roigcarlo Thanks for explaining the context. I'm happy to revert the changes to Going forward, though, it would be good to have some script that is also Mac friendly. Some of the current Darwin settings on master seem incompatible with M4 targets, for instance:
|
The scripts/wheels/ directory targets the project's specific CI environment for building redistributables. Reverted to master versions and removed scripts/mac_build_wheel. Full changes preserved in branch feature/macos-build-documentation-full.
|
Ty @takeshimg92! Also, If it helps: #!/bin/bash
# Function to add apps
add_app () {
export KRATOS_APPLICATIONS="${KRATOS_APPLICATIONS}$1;"
}
# Set variables
export KRATOS_SOURCE=$2
export KRATOS_BUILD="${KRATOS_SOURCE}/build"
export KRATOS_APP_DIR="${KRATOS_SOURCE}/applications"
export KRATOS_BUILD_TYPE="Release"
export PYTHON_EXECUTABLE=$1
# Set applications to compile
export KRATOS_APPLICATIONS=
add_app ${KRATOS_APP_DIR}/StructuralMechanicsApplication
add_app ${KRATOS_APP_DIR}/FluidDynamicsApplication
add_app ${KRATOS_APP_DIR}/DEMApplication
add_app ${KRATOS_APP_DIR}/ContactStructuralMechanicsApplication
add_app ${KRATOS_APP_DIR}/MPMApplication;
add_app ${KRATOS_APP_DIR}/ConvectionDiffusionApplication;
add_app ${KRATOS_APP_DIR}/DamApplication;
add_app ${KRATOS_APP_DIR}/PoromechanicsApplication;
add_app ${KRATOS_APP_DIR}/FSIApplication;
add_app ${KRATOS_APP_DIR}/SwimmingDEMApplication;
add_app ${KRATOS_APP_DIR}/LinearSolversApplication;
add_app ${KRATOS_APP_DIR}/ConstitutiveLawsApplication;
add_app ${KRATOS_APP_DIR}/MeshingApplication;
add_app ${KRATOS_APP_DIR}/DemStructuresCouplingApplication;
add_app ${KRATOS_APP_DIR}/MeshMovingApplication;
add_app ${KRATOS_APP_DIR}/CSharpWrapperApplication;
add_app ${KRATOS_APP_DIR}/ShapeOptimizationApplication;
add_app ${KRATOS_APP_DIR}/CoSimulationApplication;
add_app ${KRATOS_APP_DIR}/CableNetApplication;
add_app ${KRATOS_APP_DIR}/RANSApplication;
add_app ${KRATOS_APP_DIR}/MappingApplication;
add_app ${KRATOS_APP_DIR}/CompressiblePotentialFlowApplication;
add_app ${KRATOS_APP_DIR}/IgaApplication;
add_app ${KRATOS_APP_DIR}/ChimeraApplication;
add_app ${KRATOS_APP_DIR}/StatisticsApplication;
add_app ${KRATOS_APP_DIR}/RomApplication;
add_app ${KRATOS_APP_DIR}/ShallowWaterApplication;
add_app ${KRATOS_APP_DIR}/MedApplication;
add_app ${KRATOS_APP_DIR}/HDF5Application;
add_app ${KRATOS_APP_DIR}/OptimizationApplication;
add_app ${KRATOS_APP_DIR}/GeoMechanicsApplication;
add_app ${KRATOS_APP_DIR}/SystemIdentificationApplication;
# Clean
rm -rf "${KRATOS_BUILD}/${KRATOS_BUILD_TYPE}/cmake_install.cmake"
rm -rf "${KRATOS_BUILD}/${KRATOS_BUILD_TYPE}/CMakeCache.txt"
rm -rf "${KRATOS_BUILD}/${KRATOS_BUILD_TYPE}/CMakeFiles"
cmake -H"${KRATOS_SOURCE}" -B"${KRATOS_BUILD}/${KRATOS_BUILD_TYPE}" \
-DCMAKE_POLICY_VERSION_MINIMUM=3.5 \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DCMAKE_C_COMPILER=/opt/homebrew/opt/gcc/bin/gcc-15 \
-DCMAKE_CXX_COMPILER=/opt/homebrew/opt/gcc/bin/g++-15 \
-DKRATOS_USE_FUTURE=ON \
-DCMAKE_INSTALL_PREFIX=$3 \
-DCMAKE_C_FLAGS="${CMAKE_C_FLAGS} -I/opt/homebrew/include" \
-DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -I/opt/homebrew/include" \
-DOpenMP_C_FLAGS="-fopenmp" \
-DOpenMP_CXX_FLAGS="-fopenmp" \
-DOpenMP_C_LIB_NAMES="gomp" \
-DOpenMP_CXX_LIB_NAMES="gomp" \
-DOpenMP_gomp_LIBRARY=/opt/homebrew/lib/gcc/15/libgomp.dylib \
-DCMAKE_THREAD_LIBS_INIT="-lpthread" \
-DCMAKE_HAVE_THREADS_LIBRARY=1 \
-DCMAKE_USE_PTHREADS_INIT=1 \
-DCMAKE_OSX_SYSROOT=$(xcrun --show-sdk-path) \
-DBOOST_ROOT="/Users/Kratos/Tools/boost_1_87_0" \
-DINCLUDE_MMG=OFF \
-DMMG_ROOT="/workspace/external_libraries/mmg/mmg_5_5_1" \
-DHDF5_ROOT="/Users/Kratos/Tools/hdf5/bin" \
-DMED_ROOT="/Users/Kratos/Tools/med/bin" \
-DINSTALL_TPL_LIBRARIES=ON \
-DKRATOS_BUILD_TESTING=OFF \
-DKRATOS_GENERATE_PYTHON_STUBS=ON This currently works for M4 macs and is capable of building a complete Release of Kratos, but will need to have gcc15 from homebrew. To the best of my knowledge this is pure g++ with no LLVM involved, as shown by (As you can see this basically shows the lib that gave you an error while loading, as we normally don't include libstd++ in our releases because we build in manylinux and those ensure compatibility with most linux systems) But again I might be wrong, we are still pretty newbies as for what mac compilation reefers and I don't have much experience digging in what mac does under the hood with compilers. Would if be ok If I contact you after eastern so we can tests the new binaries produced to see if they work on your environment? |
Tested locally.
Related to #14336