Windows compatibility for launcher, CLI, and config paths#11
Open
offbyonebit wants to merge 9 commits into
Open
Windows compatibility for launcher, CLI, and config paths#11offbyonebit wants to merge 9 commits into
offbyonebit wants to merge 9 commits into
Conversation
- Split monolithic chat.html into chat.css + chat.js modules - Render assistant messages with marked.js + highlight.js syntax highlighting - Add copy buttons to code blocks - Redesign agent tool cards as a step timeline with step numbers and target chips - Render markdown in agent assistant messages - Improve assistant message typography (no global monospace)
Adds a Windows code path alongside the existing Linux one: process lifecycle (CREATE_NEW_PROCESS_GROUP + CTRL_BREAK_EVENT/taskkill tree-kill instead of setsid/killpg), %APPDATA%/%LOCALAPPDATA% config paths, and graceful degradation in `doctor`/`init`/`serve`/`systemd-unit` for Linux-only diagnostics. Also fixes 13 pre-existing ruff lint errors (unused imports/locals) that have been silently failing CI's lint step since the mtp-clean merge, so Test never actually ran. Adds windows-latest to the CI matrix to get real validation on a Windows runner.
The windows-latest CI job (added in the previous commit) caught 7 real
failures the Linux-only local verification couldn't:
- search_files now emits POSIX-style relative paths (Path.as_posix())
instead of native separators, so matches are consistent across platforms.
- test_run_command_cwd_is_root used `pwd`, which on the Windows runner
resolves to Git Bash's pwd.exe and prints an MSYS-style path
("/c/Users/...") instead of a native one even though cwd was correct.
Swapped for a python -c cwd check, which is honest on both platforms.
- make_sysfs_gpu fakes Linux's /sys/bus/pci/devices/<slot> layout, and
slot names contain colons (e.g. "0000:03:00.0"), which are illegal in
Windows path components. This fixture is inherently Linux-only (Windows
has no sysfs to fake), so it now skips on win32 instead of failing.
test_find_drm_card_found builds its own /sys/bus/pci/devices/0000:03:00.0 path inline instead of going through the make_sysfs_gpu fixture, so it slipped past the skip added there. Same root cause: colon in the PCI slot name is illegal in a Windows path component, and there's no real sysfs to fake on Windows anyway.
…t export/import Runtime robustness: - Router refuses to load a model if its estimated VRAM (weights + KV cache + overhead) won't fit on the target GPU, accounting for co-resident models in multi-resident mode. - LlamaServer.tail_log() surfaces the last 40 log lines in the error when a model fails its health check, instead of just a log line nobody sees. - load_config now runs migrate_config()/validate_config(): rejects configs newer than the supported schema version, fills in missing top-level sections, and raises clear errors on structural problems. Operations/observability: - GET /admin/metrics: load/stop/error counters, active models, GPU info. - GET /health now reports uptime and currently loaded models. - llama-server subprocess logs rotate at 50MB, keeping 3 backups. - ARC_LLAMA_LOG_JSON=1 switches logging to single-line JSON. API/UX: - GET /v1/chats/export and POST /v1/chats/import, plus Export/Import buttons in the chat history panel. Reuses the existing path-sanitizing _chat_path, so imported chat ids can't escape the chat directory. Streaming in the web UI and agent tool confirmation were already implemented prior to this change; no work was needed there.
Adds a Backend enum (sycl/vulkan) with per-GPU backend config, branching build_env() to strip SYCL/oneAPI vars and set GGML_VK_VISIBLE_DEVICES on the Vulkan path. New binary.py detects which backend a llama-server build actually supports (via strings or a pure-Python scan) and surfaces mismatches in `doctor`/`add`/`init`. Also expands filename-based kv_class detection to cover Qwen2.5, Qwen3 dense, Phi-4, Llama 3/4, and DeepSeek-R1 distills. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…fault-safe admin auth Interactive agent TUI, MCP client support, repo map/semantic index, and per-run file checkpoints with restore. Checkpoint store now enforces retention (max count, max age, optional total-size cap) so .arc_llama_checkpoints/ doesn't grow unbounded. Admin/agent HTTP surface now requires a bearer token: _require_admin gates every destructive admin route plus auto_confirm agent runs, closing the gap where a caller could set auto_confirm:true in the request body and skip tool confirmation entirely. admin_token is no longer opt-in -- load_config auto-generates and persists one the first time a config has none, ARC_LLAMA_ADMIN_TOKEN overrides without persisting, and `serve` takes --admin-token and prints the active token at startup. The bundled web UI bootstraps its own token from /admin/session-token, which is gated on the connection being loopback rather than on the token itself, so it stays zero-friction locally but refuses to hand out the secret to non-loopback callers. Also: Vulkan-adjacent config/CLI polish, dropped two stale roadmap items now superseded by shipped work, and a test-isolation fix (three server tests were implicitly reading the real ~/.config/arc-llama/config.toml via a bare create_app()).
The auto-recipe forced -ub 8 for every MTP model on the assumption it prevented SSM compute-buffer OOM. Live benchmarking on real hardware showed this made prompt-eval ~9.5x slower than doing nothing (45.6 vs 435.7 tok/s on Qwen3.6-27B-MTP) while the OOM risk was never actually confirmed. Removed the forced ubatch size from both add_local_model and register_discovered and the launcher fallback; re-benchmarked after the fix and prompt-eval recovered to ~360 tok/s with generation throughput unchanged, matching hand-tuned production numbers. Also fixes MoE detection silently failing on real Gemma-4-26B-A4B and Qwen3-Coder-30B-A3B GGUFs (expert_count metadata was never read), and benchmark.py not sending the admin bearer token after admin auth landed. Bumps to 0.4.0 for the accumulated Windows support, Vulkan backend, agent TUI/MCP/checkpoints, and default-safe admin auth since 0.3.0.
Refocuses the default CLI and web UI on the core goal: fastest way to set up and run inference on Intel Arc/Battlemage cards. The agent, code, and agent-tui commands are still there, but hidden from --help and the bundled chat UI unless ARC_LLAMA_EXPERIMENTAL_AGENT=1 is set. Drops the standalone `arcllama` script entry point since it's now redundant with `arc-llama agent-tui`, and fixes the README/cli.py docstring references that still pointed at it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
%APPDATA%/%LOCALAPPDATA%), and CLI (doctor/init/serve/systemd-unitdegrade gracefully instead of crashing on Windows).windows-latestto the CI matrix for real validation on a Windows runner.Test plan
ruff check .passes locallymypyclean on changed files