Skip to content

README.md

Latest commit

 

History

History
147 lines (122 loc) · 7.24 KB

README.md

File metadata and controls

147 lines (122 loc) · 7.24 KB
Raw
╔══════════════╗		██╗      ██████╗  ██████╗ █████╗ ██╗     ██████╗  ██████╗ ██╗  ██╗
║ ╔═══╗        ║		██║     ██╔═══██╗██╔════╝██╔══██╗██║     ██╔══██╗██╔═══██╗╚██╗██╔╝
║ ║███║  ████  ║║		██║     ██║   ██║██║     ███████║██║     ██████╔╝██║   ██║ ╚███╔╝
║ ╚═══╝        ║║		██║     ██║   ██║██║     ██╔══██║██║     ██╔══██╗██║   ██║ ██╔██╗
╚══════════════╝║		███████╗╚██████╔╝╚██████╗██║  ██║███████╗██████╔╝╚██████╔╝██╔╝ ██╗
 ╚══════════════╝		╚══════╝ ╚═════╝  ╚═════╝╚═╝  ╚═╝╚══════╝╚═════╝  ╚═════╝ ╚═╝  ╚═╝
  Put a local LLM behind the Claude Code / LocalPilot harness

LocalBox

Run local GGUF models through a real coding-agent harness.

Documentation · Install · Troubleshooting · LocalX

version 2.3.0 Windows, Linux, macOS llama.cpp runtime GitHub stars

LocalBox turns a local model into something you can actually use for coding. It starts llama-server, chooses safe settings for your hardware and model, and connects the result to Claude Code, Codex, or LocalPilot.

At a glance
Use it when You have a GGUF model and want an agent-ready local runtime
It handles Server lifecycle, chat templates, parsers, sampling, context, KV cache, VRAM fit hints, and harness setup
You control Model, quant, context size, runtime mode, and target harness
Runs on Windows, Linux, and macOS — a single native binary, run from any shell

Privacy by design

LocalBox runs your model on hardware you control and keeps the normal inference path local.

  • No usage telemetry is sent. LocalBox does not report your prompts, code, models, hardware measurements, or usage to us.
  • Your runtime data stays yours. Models, profiles, logs, and generated configuration remain on the machine and paths you choose.
  • Network access is deliberate. Model or runtime downloads happen only when you request them; exposing a server beyond loopback is an explicit, guarded action.
  • You remain in control. The configuration is readable, portable, and yours to inspect, back up, move, or delete.

Important

LocalBox uses llama-server. Ollama support ended after the ollama-classic tag.

Quick start

LocalBox is a single native binary — no PowerShell, .NET, or Python needed. (On Windows, AMD GPU-name detection may call the vendor's powershell.exe as a last-resort probe; the NVIDIA path and everything else are pure native.) Build it from this repository (or use a release binary):

cargo install --path crates/localbox --locked

Then launch the guided model picker:

localbox

That is the shortest path. The first run seeds ~/.local-llm with the defaults and an editable model catalog, never overwriting anything you already have. See the installation guide for details.

Everyday commands

Goal Command
Pick a model interactively localbox
Guided launcher with plain-text menus localbox --plain
Launch a model into Claude Code localbox launch <model>
Run through LocalPilot localbox launch <model> --agent localpilot
Run through Codex localbox launch <model> --agent codex
Serve headless (no agent) localbox serve <model>
List the configured models localbox info
Serve health and the remedy localbox status
Stop everything localbox stop

Model aliases come from the catalog, so the exact list on your machine may be different. localbox help is the source of truth for the installed command surface.

What LocalBox does for you

  • Chooses the correct chat template, parser, sampler, stop set, and reasoning policy for each supported model family.
  • Flags a weight-vs-VRAM fit estimate for each model in the picker (green/yellow/red) so you can spot a combination likely to run out of VRAM before you launch. The estimate is advisory — it colours the choice, it does not block the launch.
  • Keeps every launch path — launch, serve, and guided launches alike — predictable with single-session server defaults and prompt-cache reuse.
  • Sets up one consistent dispatch path for Claude Code, Codex, LocalPilot, and plain server mode.
  • Saves measured LocalBench recommendations as reusable AutoBest profiles.
GGUF model ──> LocalBox ──> llama-server ──> Claude Code / Codex / LocalPilot
                    │
                    └── VRAM fit hints, templates, parsers, cache and sampling

Choose your next guide

I want to… Read
Install or repair LocalBox Install
Upgrade from a 1.x install Install → Upgrading from 1.x
Learn the day-to-day flags Usage
Connect a coding-agent harness Harness mode
Add or size a model Model management
Tune a model automatically Auto-tuner and AutoBest profiles
Configure a machine Settings
Choose a llama.cpp runtime mode llama.cpp modes
Understand the repository Architecture
Fix a problem Troubleshooting

LocalX

LocalBox is the runtime layer in the LocalX toolchain:

Project Role
LocalBox Run local models
LocalBench Find fast, stable settings
LocalPilot Code through the agent harness
LocalMind Turn reviewed sessions into reusable project memory

Release history lives in CHANGELOG.md.