Skip to content

feat(sweep): run isolation fans concurrently on the device-farm fleet #874

Description

@jonathaneoliver

Summary

Run a sweep isolation fan (control + single-axis-flip variants) concurrently across the Appium Device Farm — one device per arm, all sharing the isolation group/pattern — instead of serially on one operator-pinned sim. This makes cross-platform/protocol flips (the design's Tier-1 isolation strategy) actually parallel and cuts wall-clock per fan.

Motivation

sweep's IsolationFan (tools/harness-cli/internal/sweep/isolate.go) already builds a control + one variant per axis-flip (OFAT attribution). But TestSweepProbe (modes/sweep_probe_test.go) resolves one operator-pinned device (PickMode → appium.Discover → CHARACTERIZATION_DEVICE_UDID) and never touches DeviceFarmEnabled()/resolveFleetDeviceFarm() — so the fan runs serially on one sim. The skill explicitly wants the opposite: "Tier 1 = platform/protocol → different devices → simultaneous."

char-matrix already has the machinery: TestCharMatrixFleet (modes/char_matrix_fleet_test.go) → resolveFleetresolveFleetDeviceFarm (modes/fleet.go) allocates N devices by capability, assigns per-arm platform (CHAR_ARM_<i>_PLATFORM), pins real-iPhone hardware off-farm (runner/appium.go:287), and binds all arms to one proxy group (binding is by player_id at the proxy). Post-#869 both probes converge on the typed charplan.RunPlan (go-proxy/pkg/charplan/plan.go).

Proposed

When a fan has ≥2 arms, route it onto the fleet path: build a charplan.RunPlan from the fan and resolve via resolveFleetDeviceFarm, so control + variants launch on one device each, sharing the isolation group/pattern (control arms the pattern, variants bind). Keep the single-device path as the default for depth-1 / single-experiment ticks (cheap, one sim — the loop's cost-per-tick discipline).

Scope

  • Teach the sweep probe a fleet entry point — prefer reusing the TestCharMatrixFleet path via charplan.RunPlan over duplicating allocation logic.
  • Fan → RunPlan: each arm = one ArmConfig (recipe + PlayerID + shared Group/PatternMaster).
  • Gate: fans (≥2 arms) → fleet; singletons → existing single-device probe.

Key files

internal/sweep/isolate.go (IsolationFan), cmd/harness/sweep.go (apply/agenda), modes/{sweep_probe_test.go,fleet.go,char_matrix_fleet_test.go} (probe + DF allocation), go-proxy/pkg/charplan/plan.go (RunPlan), runner/appium.go (hybrid real-iOS routing, withBaselineTestFlags).

Constraints / gotchas

  • Mixed-platform fans (sim + real iPhone) use the appium.go mixed-master design: pin the iPhone hardware UDID, leave sims unpinned (fleet.go:117). The iPhone arm runs off-farm on :4799, farm-locked while sims stay on the farm at :4723 — the ADF cannot WDA-launch a real iOS 26 device (see memory reference_real_iphone_usbmux_pairing_wipe). The fan must tolerate one arm on the hybrid path.
  • Preserve Fleet char-matrix runs have no graceful-interrupt teardown → orphaned appium sessions block the next run #853 device-release-on-interrupt (t.Cleanup/SIGTERM dance) and the n=1 confirm-reps gate.
  • Keep the queue fileless / CH-master — thread the RunPlan in-memory / temp-file (as char.go already does); don't introduce a spec file just to reach the fleet path. (If the YAML-bridge sibling lands first, the fleet path may reuse its RunPlan adapter.)
  • Stale device-farm locks after an abrupt kill must still be reaped (the documented reapDeviceFarm / unblock recovery).

Verification

  • Unit: fan → RunPlan golden — arm count, shared group, single-axis diffs preserved.
  • Live: seed a platform-flip notable → control + variant launch on two devices at once, share one pattern (identical bandwidth per tick), both land verdicts → analyze attributes correctly. Plus a mixed sim + real-iPhone fan exercising the off-farm hybrid arm.
  • Build gate: go build ./... && go vet ./... across go-proxy, tools/harness-cli, tests/characterization.

Acceptance criteria

Relationship

Sibling of the sweep↔YAML-bridge issue, which introduces the Experiment ↔ charmatrix/RunPlan adapter this path reuses. Land the YAML bridge first; this builds on its RunPlan mapping.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions