Skip to content

hal0 v0.9.1

Choose a tag to compare

@github-actions github-actions released this 05 Jul 15:36
ab21010

ROCmFPX llama.cpp runner support, plus a safer notes-aware self-update.

Highlights

  • ROCmFPX runner support: GGUF quant-family detection, rocmfpx-rocm / vkfpx-moe seed profiles, and a build/quantize agent skill.
  • hal0 update now shows cosign-verified release notes and asks before applying (staged prepare → commit).

Added

  • Updater prepare / commit split. hal0 update downloads + cosign-verifies + extracts a release and shows its notes — with breaking/migration callouts — before activating anything; --yes skips the prompt for headless/cron. Adds POST /api/updates/prepare + /commit (#1075).
  • Release notes in the update. The release build bundles RELEASE_NOTES.md + release.json into the cosign-verified tarball; a CHANGELOG section's ### Highlights / ### Breaking / ### Migrations subsections become the hal0 update callouts (#1078).
  • ROCmFPX quant detection — the registry classifies the ROCmFPX / ROCmFP{3,4,6,8} quant family from a GGUF filename so FPX slots resolve their launch command (#1068).
  • ROCmFPX seed profiles rocmfpx-rocm (ROCm0 dense) and vkfpx-moe (Vulkan0 MoE) for the custom ROCmFPX runner (#1069, renamed in #1076).
  • vkfpx-dense seed profile — the Vulkan0 lane for DENSE ROCmFP4, for prefill-bound dense workloads (Vulkan wins prompt-processing); complements the decode-optimal ROCm0 rocmfpx-rocm.
  • hal0-quantize agent skill — build the ROCmFPX toolchain and quantize a model to ROCmFP4/FPX (#1071).
  • ROCmFPX bench tooling: server-ab (MTP / concurrency) aggregation in the benchmark SUMMARY, FPX sweep cells, run provenance (#1072).
  • Continuous batching — per-slot parallel field. A slot can now set
    llama-server's --parallel / -np sequence-slot count so concurrent
    requests share the once-loaded weights instead of serializing through a
    single sequence and thrashing one prompt cache (the win the shared-slot
    architecture already earns but never harvested — every seed profile pins
    --parallel 1). None inherits the profile; a value >1 also emits
    --kv-unified so --ctx-size stays a SHARED pool (each request may use the
    full context) rather than being silently split to ctx/N per slot. Emitted as
    a slot override (beats the profile, loses to hand-authored extra_args);
    surfaced in the slot drawer with a shared-pool hint. The dead haloai
    workers field is deprecated (inert; a non-default value now logs at
    launch). MTP x batching runs but logs mtp.batched_speculation (unproven on
    gfx1151, bench-gated). Seed-profile defaults stay --parallel 1 pending the
    on-box -np sweep (server_ab.py --mode batch). See the
    concurrency-batching plan handoff.

Changed

  • The plain rocm / vulkan seed profiles are reduced to basic flags
    (-ngl 999 -fa on --jinja); per-model KV/batch tuning now lives in the
    model's defaults.extra_args (#1076).
  • Seed-profile intent labels normalised to terse structural tags
    (e.g. ROCmFPX · DENSE · MTP, VULKFPX · MOE · MTP, ROCm, Embeddings) —
    no served-model names, no filler.
  • Slot units are re-rendered through the new code during an update's commit
    step, so a subsequent restart uses current argv (#1075).

Removed

  • Legacy MTP toolbox seed profiles rocm-moe and rocm-dnse (superseded by the
    ROCmFPX profiles); rocmfpx-moe renamed to vkfpx-moe to indicate its Vulkan
    lane (#1076).

Migrations

  • Slots pinned to a removed/renamed seed profile (rocm-moe, rocm-dnse, rocmfpx-moe) auto-fall-back to the backend's basic profile (rocm / vulkan) on launch — existing slots keep working with no operator action (#1076).