Skip to content

Runtime suite is 2-63x slower per test on macOS than Linux; a full local run does not finish in 7 hours #1060

Description

@mrthankyou

Summary

The runtime suite is dramatically slower on macOS than on Linux, on the same commit and the same tests. A full local run on main (8b985b90) was stopped after 7 hours without finishing; the Linux CI lanes for the same suite complete in 24-46 minutes each.

This only became visible after #1019 and #1025. Before those, most of these tests aborted in milliseconds on the symlinked os.tmpdir() (#1017), so the cost was hidden behind a failure. Nothing here is a regression from that work -- it is the bill arriving.

Per-test comparison, same commit

Darwin timings from the local run; Linux timings extracted from the CI runtime lanes on the same commit.

test darwin linux ratio
syncRun accepts the runner's correlation envelope and rejects a mismatched one 32.2 min 1.59 min 20x
syncRun accepts the pinned 0.35.0 usage payload and still bounds its new fields 29.8 min 13.76 min 2x
getRunHealth rejects every noncurrent status envelope without fallback or filtering 23.7 min 2.86 min 8x
refreshed controller rejects a drifted retained prompt snapshot 21.7 min 0.34 min 63x
syncRun binds an optional prerequisite digest before a final-boundary manifest swap 18.3 min 2.54 min 7x
syncRun succeeds when only an explicitly nonblocking specialist fails 17.7 min 0.61 min 29x

The slowest test on the Linux side of the whole sample is about a minute. On darwin, 147 tests exceed 60 seconds.

Aggregate

wall clock                 7h 00m, stopped before completion
measured test time        23.5 hours (tests run concurrently across files)
tests over 60s               147
slowest single test        32.2 min

It is not CPU

The machine was idle throughout. Sampled repeatedly during the run:

total CPU across test processes    7.6% - 18.8%

An M-series laptop at ~90% idle taking 63x a CI container points at IO or syscall behaviour rather than compute. During the run the system temporary directory held 378,807 live files across 149 roots, which is a plausible contributor but has not been isolated.

What this is not

Why it matters

A contributor on macOS cannot run the runtime suite as part of normal work. Seven hours without finishing means the practical options are to push and let CI answer, or to run a narrow selection and hope the rest is unaffected. Both weaken the suite as a local safety net, and it is the platform the project now supports for campaigns (#1011, #1016).

Not diagnosed

I have not isolated the cause and am not proposing a fix here. The obvious next step is to profile one of the extreme cases -- refreshed controller rejects a drifted retained prompt snapshot at 63x is the cleanest candidate, since it is only 20 seconds on Linux -- and determine whether the time is in filesystem syscalls, process spawning, or something else specific to darwin.

Environment

  • macOS (Darwin 25.6.0) arm64, node v26.3.1, pnpm 11.1.1
  • ultrafuzz main @ 8b985b90
  • Linux figures from the CI runtime lanes on the same commit
  • Default TMPDIR; temporary directory cleared before the run

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions