Kernel fails to compile due to assembly issues on aarch64

[RossComputerGuy]

Trying to get krunvm to work on NixOS, I updated libkrunfw to 3.12.0. I also updated the kernel to 6.2.14 and added cpio and perl. I then got these assembly errors. I am on an M1 Pro running NixOS with the Asahi kernel and packages.

libkrunfw> /build/ccaOHYMx.s: Assembler messages:
libkrunfw> /build/ccaOHYMx.s:40: Error: selected processor does not support `aese v2.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:41: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:44: Error: selected processor does not support `aese v0.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:45: Error: selected processor does not support `aesmc v0.16b,v0.16b'
libkrunfw> /build/ccaOHYMx.s:53: Error: selected processor does not support `aese v1.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:54: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:60: Error: selected processor does not support `aese v9.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:61: Error: selected processor does not support `aesmc v9.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:67: Error: selected processor does not support `aese v15.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:68: Error: selected processor does not support `aesmc v15.16b,v15.16b'
libkrunfw> /build/ccaOHYMx.s:75: Error: selected processor does not support `aese v11.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:76: Error: selected processor does not support `aesmc v11.16b,v11.16b'
libkrunfw> /build/ccaOHYMx.s:83: Error: selected processor does not support `aese v4.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:84: Error: selected processor does not support `aesmc v4.16b,v4.16b'
libkrunfw> /build/ccaOHYMx.s:91: Error: selected processor does not support `aese v15.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:92: Error: selected processor does not support `aesmc v15.16b,v15.16b'
libkrunfw> /build/ccaOHYMx.s:99: Error: selected processor does not support `aese v0.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:100: Error: selected processor does not support `aesmc v0.16b,v0.16b'
libkrunfw> /build/ccaOHYMx.s:107: Error: selected processor does not support `aese v1.16b,v8.16b'
libkrunfw> /build/ccaOHYMx.s:108: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:558: Error: selected processor does not support `aese v4.16b,v11.16b'
libkrunfw> /build/ccaOHYMx.s:559: Error: selected processor does not support `aesmc v4.16b,v4.16b'
libkrunfw> /build/ccaOHYMx.s:562: Error: selected processor does not support `aese v0.16b,v11.16b'
libkrunfw> /build/ccaOHYMx.s:563: Error: selected processor does not support `aesmc v0.16b,v0.16b'
libkrunfw> /build/ccaOHYMx.s:570: Error: selected processor does not support `aese v2.16b,v11.16b'
libkrunfw> /build/ccaOHYMx.s:571: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:574: Error: selected processor does not support `aese v3.16b,v11.16b'
libkrunfw> /build/ccaOHYMx.s:575: Error: selected processor does not support `aesmc v3.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:585: Error: selected processor does not support `aese v1.16b,v11.16b'
libkrunfw> /build/ccaOHYMx.s:586: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:862: Error: selected processor does not support `aese v1.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:863: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:866: Error: selected processor does not support `aese v4.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:867: Error: selected processor does not support `aesmc v4.16b,v4.16b'
libkrunfw> /build/ccaOHYMx.s:874: Error: selected processor does not support `aese v3.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:875: Error: selected processor does not support `aesmc v3.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:882: Error: selected processor does not support `aese v3.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:883: Error: selected processor does not support `aesmc v3.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:890: Error: selected processor does not support `aese v3.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:891: Error: selected processor does not support `aesmc v3.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:925: Error: selected processor does not support `aese v3.16b,v5.16b'
libkrunfw> /build/ccaOHYMx.s:926: Error: selected processor does not support `aesmc v3.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:933: Error: selected processor does not support `aese v4.16b,v5.16b'
libkrunfw> /build/ccaOHYMx.s:934: Error: selected processor does not support `aesmc v4.16b,v4.16b'
libkrunfw> /build/ccaOHYMx.s:941: Error: selected processor does not support `aese v4.16b,v5.16b'
libkrunfw> /build/ccaOHYMx.s:942: Error: selected processor does not support `aesmc v4.16b,v4.16b'
libkrunfw> /build/ccaOHYMx.s:949: Error: selected processor does not support `aese v4.16b,v5.16b'
libkrunfw> /build/ccaOHYMx.s:950: Error: selected processor does not support `aesmc v4.16b,v4.16b'
libkrunfw> /build/ccaOHYMx.s:957: Error: selected processor does not support `aese v4.16b,v5.16b'
libkrunfw> /build/ccaOHYMx.s:958: Error: selected processor does not support `aesmc v4.16b,v4.16b'
libkrunfw> /build/ccaOHYMx.s:1526: Error: selected processor does not support `aese v0.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:1527: Error: selected processor does not support `aesmc v0.16b,v0.16b'
libkrunfw> /build/ccaOHYMx.s:1530: Error: selected processor does not support `aese v2.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:1531: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:1538: Error: selected processor does not support `aese v1.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:1539: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:1546: Error: selected processor does not support `aese v1.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:1547: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:1554: Error: selected processor does not support `aese v1.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:1555: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:1590: Error: selected processor does not support `aese v1.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:1591: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:1598: Error: selected processor does not support `aese v2.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:1599: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:1606: Error: selected processor does not support `aese v2.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:1607: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:1614: Error: selected processor does not support `aese v2.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:1615: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:1622: Error: selected processor does not support `aese v2.16b,v3.16b'
libkrunfw> /build/ccaOHYMx.s:1623: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:2172: Error: selected processor does not support `aese v0.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:2173: Error: selected processor does not support `aesmc v0.16b,v0.16b'
libkrunfw> /build/ccaOHYMx.s:2176: Error: selected processor does not support `aese v2.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:2177: Error: selected processor does not support `aesmc v2.16b,v2.16b'
libkrunfw> /build/ccaOHYMx.s:2184: Error: selected processor does not support `aese v1.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:2185: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:2192: Error: selected processor does not support `aese v1.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:2193: Error: selected processor does not support `aesmc v1.16b,v1.16b'
libkrunfw> /build/ccaOHYMx.s:2200: Error: selected processor does not support `aese v1.16b,v9.16b'
libkrunfw> /build/ccaOHYMx.s:2201: Error: selected processor does not support `aesmc v1.16b,v1.16b'

[Lemm1]

I had the same issue. It seems like Apple Silicon requires a different config file to build the kernel compared to what is in the source. I've managed to build libkrunfw by just making a simple override that replaces config-libkrunfw_aarch64 file with my kernel config and adding all the required dependencies to build it on aarch64-linux:

    (final: prev: {
      libkrunfw = prev.libkrunfw.overrideAttrs (old: {
        nativeBuildInputs = (old.nativeBuildInputs or []) ++ (with pkgs; [perl openssl]);
        postPatch =
          old.postPatch
          + ''
            zcat /proc/config.gz > config-libkrunfw_aarch64
          '';
        meta.platforms = (old.meta.platform or []) ++ ["aarch64-linux"];
      });
    })

[slp]

That's weird, because libkrunfw does build fine on Apple Silicon with Fedora Asahi Linux. Which compiler is NixOS using?

[RossComputerGuy]

@slp It can use Clang but defaults to GCC.

[Lemm1]

@slp nixpkgs builds libkrunfw with gcc

[liarokapisv]

Changing the bundled aarch64 config is not correct since it is fine-tuned for virtualization.

The problem here is that nixpkgs' gcc wrapper, adds -march=armv8-a to all builds on Apple Silicon. This conflicts with linux/crypto/aegis128-neon-inner.c's appended -mcpu=generic+crypto flag. It seems using genericfor the mcpu with a different march trips up the gnu assembler.
Adding NIX_CFLAGS_COMPILE="-march=armv8-a+crypto" to the libkrunfw package build along with the missing cpio makes things work for me.