Maturin can't build in manylinux container

[DanielChaseButterfield]

Bug Description

I'm trying to build a Python 3.9 wheel for my repository. I'll tell the whole story from beginning to end. First, maturin built a wheel that was incompatible for installation with my latest form of pip:

📦 Built wheel for CPython 3.9 to /drone/src/target/wheels/actag-0.1.0-cp39-cp39-manylinux_2_35_x86_64.whl
+ pip install target/wheels/*cp39*.whl
ERROR: actag-0.1.0-cp39-cp39-manylinux_2_35_x86_64.whl is not a supported wheel on this platform.

By running pip debug --verbose, I discovered that the maximum manylinux version supported by pip was 2_31:

Compatible tags: 663
  cp39-cp39-manylinux_2_31_x86_64
  cp39-cp39-manylinux_2_30_x86_64
  cp39-cp39-manylinux_2_29_x86_64
  cp39-cp39-manylinux_2_28_x86_64
  cp39-cp39-manylinux_2_27_x86_64
  cp39-cp39-manylinux_2_26_x86_64
  cp39-cp39-manylinux_2_25_x86_64

Therefore, I tried to have maturin generate a wheel with a 2_31 version. However, it told me:

🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.9 at /usr/local/bin/python3
    Blocking waiting for file lock on build directory
    Finished release [optimized] target(s) in 1m 45s
💥 maturin failed
  Caused by: Error ensuring manylinux_2_31 compliance
  Caused by: Your library is not manylinux_2_31 compliant because of the presence of too-recent versioned symbols: ["libm.so.6 offending symbols: hypot@GLIBC_2.35", "libc.so.6 offending symbols: pthread_join@GLIBC_2.34, pthread_setname_np@GLIBC_2.34, pthread_attr_getguardsize@GLIBC_2.34, pthread_setspecific@GLIBC_2.34, pthread_getattr_np@GLIBC_2.32, pthread_create@GLIBC_2.34, stat64@GLIBC_2.33, pthread_key_create@GLIBC_2.34, fstat64@GLIBC_2.33, pthread_attr_getstack@GLIBC_2.34, pthread_detach@GLIBC_2.34, pthread_attr_setstacksize@GLIBC_2.34, pthread_key_delete@GLIBC_2.34, pthread_getspecific@GLIBC_2.34, dlsym@GLIBC_2.34"]. Consider building in a manylinux docker container

For this, I generally understood that my current platform was using functions from newer versions of GLIBC than supported. Therefore, if I built in a manylinux docker container, it would hopefully use functions that were available in the docker container. I setup a quay.io/pypa/manylinux_2_28_x86_64 docker container, and ran into a new error:

📦 Including license file "/drone/src/LICENSE.md"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.9 at /usr/bin/python3
    Blocking waiting for file lock on build directory
   Compiling pyo3-build-config v0.16.6
error: failed to run custom build command for `pyo3-build-config v0.16.6`

Caused by:
  process didn't exit successfully: `/drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build` (exit status: 1)
  --- stderr
  /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build: /lib64/libc.so.6: version `GLIBC_2.29' not found (required by /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build)
  /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build)
  /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build)
  /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /drone/src/target/release/build/pyo3-build-config-f054e43b211639d4/build-script-build)
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `PYO3_ENVIRONMENT_SIGNATURE="cpython-3.9-64bit" PYO3_PYTHON="/usr/bin/python3" PYTHON_SYS_EXECUTABLE="/usr/bin/python3" "cargo" "rustc" "--message-format" "json-render-diagnostics" "--manifest-path" "/drone/src/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"`

Basically, maturin is failing because it is now missing the required versions of GLIBC to build the wheel for 2_28. I find this strange however, as on my personal machine, maturin had no issue building a actag-0.1.0-cp39-cp39-manylinux_2_31_x86_64.whl wheel, meaning that at the very least, GLIBC_2.32-GLIBC_2.34 shouldn't be required.

It is also weird that it's complaining about this, as wasn't the point of the manylinux container to remove these GLIBC versions so that it could build a wheel with lower GLIBC requirements?

How can I fix this issue? Is it possible that I have a slightly outdated version of Python, which means that I can't install my 2_35 wheel from the beginning? Do I have a newer version of Rust or Pyo3, which stops it from making a 2_31 wheel? Or something else entirely?

NOTE: On my personal computer, where this issue is completely avoided (maturin builds a 2_31 wheel that is supported by pip), I have the following versions:

• maturin 1.0.1
• Python 3.9.16
• pip 23.2.1

Could it be that the newest version of maturin is the issue?

Your maturin version (maturin --version)

maturin 1.2.0

Your Python version (python -V)

Python 3.9.16

Your pip version (pip -V)

pip 23.2.1

What bindings you're using

pyo3

Does cargo build work?

• Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

• Yes

Steps to Reproduce

The full repository where this problem was reproduced can be found here: actag repository.

The current failing build, with all of the output can be found here: Test Python 3.9 - Build #120. For the Drone CI file that runs this test, see line 52 of the .drone.yml file.

[DanielChaseButterfield]

Also, I just set the maturin version of the docker build from 1.2.0 to 1.0.1 to match my personal computer, but unfortunately, it still failed.

[DanielChaseButterfield]

This also looks like it's semi-related to Issue #1490.

[messense]

Have you tried cargo clean first? Looks like you are sharing build cache between host and container.