Installation fails on macOS ARM64 due to x86-specific assembly code

[matsen]

Problem

Installation via pip fails on macOS ARM64 (Apple Silicon) during compilation due to x86-specific assembly code.

Steps to Reproduce

pip install git+https://github.com/psathyrella/partis.git

Error

Compilation fails with errors like:

/Library/Developer/CommandLineTools/usr/lib/clang/17/include/emmintrin.h:14:2: error: "This header is only meant to be used on x86 and x64 architecture"

The build process includes x86-specific headers (emmintrin.h, xmmintrin.h, mmintrin.h) that are incompatible with ARM64 architecture.

Environment

• macOS with Apple Silicon (ARM64)
• Python 3.11
• Clang 17

Expected Behavior

Installation should succeed on ARM64 platforms or provide clear documentation about platform limitations.

Possible Solutions

1. Add conditional compilation for ARM64 vs x86 architectures
2. Use platform-agnostic SIMD intrinsics
3. Provide pre-compiled binaries for ARM64
4. Document x86-only limitation if ARM64 support is not planned

[psathyrella]

CC seems to think that porting the ig-sw code to work on ARM64 macs would be very time consuming since it'd require translating a ton of SIMD calls, but I wanted to check what you think before just removing the SIMD optimizations for ARM64 macs, since you were actually involved in the ig-sw code (I've never only poked at a few unrelated lines).

[matsen]

I was? Boy, I can't remember that!

In any case, I think that an ARM install without SIMD is better than no install at all, but that we should warn mac users that this isn't optimized so for best performance they should use linux.

What do you think?

[psathyrella]

Well, I thought so! But at least I wasn't involved ;-). Maybe it was Connor and/or some interns?

Yeah I'll do that and just see what the performance hit is. SW in general isn't a bottleneck, and hopefully it stays that way without this optimization.

[psathyrella]

Huh, it seems that in all my back and forth with CC that it was lying about its stub implementations still performing (non-parallelized) alignment. Instead, it seems that they just write a truncated file with null result, and getting this working on newer macs would require reimplementing the alignment without the emmintrin/parallelization library. To me at least this would seem to require reimplementing basically from scratch, since the code here, especially in ksw.c, is extraordinarily opaque to me. My understanding is that binaries compiled on older macs work fine on newer ones, and this should remain true, so perhaps we should just tell macOS 14 users to use pipx?

[matsen]

I guess my main thought is that this can wait.

It's not obvious to me that there is emulation software would translate intel instructions to new arm macs but I could certainly be wrong there.

[psathyrella]

Ooops, right, but ARM64 macs can't run the intel-compiled binaries from pipx, so my suggestion doesn't make sense. I did just get an email from github saying they'll deprecate all mac intel runners in 2027, but until then it seems like i can use macos-15-intel

Long term, it does seem like our choice might be 1) reimplement ig-sw 2) don't support macs. Both of which really suck.