Update Docker image and investigate ARM64 binary support

[s-canchi]

Summary

Two related infrastructure items from #337

1. Docker image out of date: The published image (quay.io/matsengrp/partis) still reflects the old python/ directory structure. The source package directory has been renamed to partis/. The image should be rebuilt from the current codebase.

2. ARM64 binary support: The compiled binaries (ig-sw and bcrham) are x86-only. ARM64 support is missing. Currently development on ARM64 requires x86 emulation via Docker, which works but adds overhead.

   Questions for investigation:

   • Can ig-sw be compiled on ARM64? It uses SSE2 intrinsics for Smith-Waterman alignment. Options may include porting to NEON, using a translation layer like SIMDe, or finding an alternative.
   • Can bcrham be compiled on ARM64? Need to check for architecture-specific dependencies.

[psathyrella]

I don't remember the status of the docker image -- maybe I've just forgotten I was doing it by hand, and/or maybe their auto builds failed, and/or it's something else. Thanks for looking into it

As to ARM64, life sucks and everything is terrible: #330

except erick is working on converting ig-sw to zig, which hopefully will resolve this issue.

[s-canchi]

Pre-existing test failure: simulate step crashes in Docker

Running partis-test.py --paired (without --no-simu) in the dev Docker container fails at the simulate step during tree generation.

Error: subprocess.CalledProcessError in the generate-trees step (bin/partis line 872).

Failing command:

/partis-dev/bin/partis simulate --generate-trees --outfname .../trees.nwk

Workaround: partis-test.py --paired --no-simu passes all tests.

What dependency is needed for tree generation that may be missing from the Docker image?

[psathyrella]

Simulation has a bunch of extra deps:

https://github.com/psathyrella/partis/blob/dev/docs/install.md#simulation

[s-canchi]

Simulation dependencies in Docker

Investigated what is needed to run partis-test.py --paired (with simulation) in the dev Docker container. Three issues found:

1. R and R packages (TreeSim, TreeSimGM, geiger) are not installed. Adding r-base, libopenblas-dev, gfortran via apt and installing the R packages to /usr/local/lib/R/site-library resolves tree generation. Packages must be installed to a system-wide library path, not the root user library, since the container runs as mambauser.

2. The bpp-newlik binary (bppseqgen) is not compiled. The base image has the older bpp at /partis/packages/bpp/bin/bppseqgen but the code expects bpp-newlik at packages/bpp-newlik/_build/bin/bppseqgen. Compiling bpp-newlik requires additional system dependencies (libblas, liblapack, libboost, libeigen3, libsuitesparse, libfftw3) and ./bin/build.sh with-simulation (approximately 30 minutes).

3. Symlinking the older bpp binary in place of bpp-newlik partially works but produces a UnicodeDecodeError when partis reads the bpp log output (non-UTF-8 bytes from the older binary version).

Simulation is not needed for the current disjoint grouping work, so I will continue using --no-simu. Leaving this here for reference if the Docker image is rebuilt with full simulation support.

[s-canchi]

Slow paired ref results missing subset-partition

Running partis-test.py --slow --paired crashes with:

FileNotFoundError: [Errno 2] No such file or directory: 'test/paired/ref-results-slow/subset-partition-new-simu/partition-igh.yaml'

The non-slow ref results (test/paired/ref-results/) have subset-partition-new-simu/ but test/paired/ref-results-slow/ does not. The slow ref results were last updated before the subset-partition test was added.

[psathyrella]

Oh, yes, sorry, it's fine to use --no-simu. The disjoint grouping stuff shouldn't affect anything in simulation.

For the sake of posterity, re: 1. and 2. I believe those are handled by the simulation install methods in the docs (although it's definitely annoying that they have to be installed), but please let me know if not. Re: 3. yeah symlinking seems dangerous even if it did run, but --no-per-base-mutation uses the bppseqgen version that doesn't require compilation.

The slow ref results were last updated before the subset-partition test was added.

Whoops! Sorry. Let me know if I should update those or if it's clear to you how to do it.