Skip to content

Add support for building ARM64 iOS wheels on CI #181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

ambv
Copy link

@ambv ambv commented Jul 24, 2025

This will greatly enable the iOS platform in terms of support for other packages that depend on cffi. See PEP 730 for details about the iOS platform support in Python.

This was a little tricky to get right:

  • every iOS build is a cross build so depending on pkg-config or homebrew is invalid;
  • for iOS builds, the setup.py in this repository can't be called directly, but rather through cibuildwheel, which effectively runs python -m build but with a special python that pretends it's running on sys.platform == "ios" with an almost empty $PATH to avoid pulling in ABI-incompatible dependencies;
  • this platform requires dedicated builds of libffi which additionally preferably are statically linked into our resulting wheel's .dylib;
  • the location of the unzipped sdist is different from what cibuildwheel docs suggest you should put in CIBW_TEST_SOURCES.

Now, the current status is such that we're running the src/test_c.py tests with some expected skips (callbacks = executable memory, disallowed on iOS) but also two unexpected failures. I marked one with xfail, the other is a skip because it crashed the test runner with Python 3.13. Worth investigating after the fact once the CI is set up.

The tests from testing/ are not enabled for iOS since they depend on ffi.verify(), which attempts in-place compilation, impossible on iOS.

@ambv
Copy link
Author

ambv commented Jul 24, 2025

Everything built fine on the branch:
https://github.com/ambv/cffi/actions/runs/16500758807

@ambv
Copy link
Author

ambv commented Jul 25, 2025

Newest run with macOS 15 passed as well: https://github.com/ambv/cffi/actions/runs/16519616901.

@ambv
Copy link
Author

ambv commented Jul 28, 2025

Everything's still green after #178: https://github.com/ambv/cffi/actions/runs/16579222190

# Download prebuilt libffi from beeware/cpython-apple-source-deps
platform="${{ matrix.platform }}"
version="${CFFI_IOS_LIBFFI_VERSION}"
url="https://github.com/beeware/cpython-apple-source-deps/releases/download/libFFI-${version}/libffi-${version}-${platform}.arm64.tar.gz"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it looks a little weird to download binaries from beeware, cibuildwheel internally actually does the same for the "iOS support" python interpreter used to run python -m build with.

@ambv
Copy link
Author

ambv commented Jul 28, 2025

The CI failures here are intermittent GitHub.com HTTPS availability issues. They'll pass if you kick them.

@ambv
Copy link
Author

ambv commented Jul 28, 2025

Looks like GitHub's got an outage:
Screenshot 2025-07-29 at 00 54 12

@mattip
Copy link
Contributor

mattip commented Jul 28, 2025

Will try again in a few hours

Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <[email protected]>
@mattip
Copy link
Contributor

mattip commented Jul 30, 2025

the last commit does not seem to work

@ambv
Copy link
Author

ambv commented Jul 30, 2025

Yeah, that added -I broke it. I'll just revert it.

This reverts commit c4e3a87.
@ambv
Copy link
Author

ambv commented Aug 19, 2025

@mattip do you need me to do anything here?

@mattip
Copy link
Contributor

mattip commented Aug 19, 2025

LGTM. @nitzmahone would you like to take a look?

Comment on lines +421 to +446
- name: build wheel prereqs
run: |
set -eux
python3 -m pip install --user --upgrade cibuildwheel>=3.1.0
brew uninstall --ignore-dependencies libffi 2>&1 || true

- name: download libffi for iOS
env:
CFFI_IOS_LIBFFI_VERSION: '3.4.7-2'
run: |
set -eux

# Download prebuilt libffi from beeware/cpython-apple-source-deps
platform="${{ matrix.platform }}"
version="${CFFI_IOS_LIBFFI_VERSION}"
url="https://github.com/beeware/cpython-apple-source-deps/releases/download/libFFI-${version}/libffi-${version}-${platform}.arm64.tar.gz"

echo "Downloading libffi for iOS (${platform})..."
curl -L -o libffi-ios.tar.gz "${url}"

# Extract libffi
mkdir -p libffi-ios
tar zxf libffi-ios.tar.gz -C libffi-ios

# Set up paths for cibuildwheel
echo "LIBFFI_IOS_DIR=$(pwd)/libffi-ios" >> "$GITHUB_ENV"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would make sense to plug this into the cibuildwheel config so that it's usable outside the CI and isn't coupled with it.

@ambv
Copy link
Author

ambv commented Aug 19, 2025

@webknjaz every little change I make here requires somebody to manually approve the CI to run for me. Let's ignore the nits for now.

@ngoldbaum ngoldbaum mentioned this pull request Aug 19, 2025
3 tasks
@nitzmahone
Copy link
Member

@ambv do you happen to know offhand if the libffi build you're embedding (and/or iOS in general) needs the same trampoline dylib stub hacks that MacOS aarch64 does? It's been several years since I dug into that, but IIRC it was at least required to enable marking pages executable for dynamic codegen without using a highly-restricted entitlement. Most vanilla libffi builds were just delegating that part to the Apple-built stub (which has the necessary entitlement), which is of course very fragile for statically-linked stuff should it ever change.

I also see a bunch of the tests around dynamic codegen have been disabled for iOS anyway, so maybe it's all moot, but the details escape me if libffi needed that stub dylib for anything else.

@ambv
Copy link
Author

ambv commented Aug 20, 2025

@nitzmahone asks:

do you happen to know offhand if the libffi build you're embedding (and/or iOS in general) needs the same trampoline dylib stub hacks that MacOS aarch64 does? It's been several years since I dug into that, but IIRC it was at least required to enable marking pages executable for dynamic codegen without using a highly-restricted entitlement. Most vanilla libffi builds were just delegating that part to the Apple-built stub (which has the necessary entitlement), which is of course very fragile for statically-linked stuff should it ever change.

The bad news is that I have no idea about trampoline dylib stub hacks. The "good" news is that on iOS any form of marking pages for execution is disallowed by the kernel (no userland JITs supported due to security concerns), so it seems like at the moment it's not a problem either way.

@mattip
Copy link
Contributor

mattip commented Aug 20, 2025

CI is quickly failing

@ambv
Copy link
Author

ambv commented Aug 20, 2025

@mattip, reverted the breaking change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants