Skip to content

v0.1.0

Latest
Latest

Choose a tag to compare

View all tags
@agunapal agunapal released this 14 May 19:31
· 62 commits to main since this release
v0.1.0
f5c7f58
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

aevyra-forge is an autonomous vLLM deployment optimizer — give it a model, a GPU, and a workload trace and it tunes your serving config overnight with a full audit trail of every experiment.

What's included in v0.1.0

Core loop

  • Orchestrator — autonomous tune loop: propose → boot vLLM → bench → keep/revert, repeat
  • ForgeConfig — budget knobs: max_experiments, max_hours, max_dollars, accuracy_floor, dry_run
  • Layer escalation: config → quant → kernel (quant/kernel stubs return gracefully when reached)
  • Convergence detection and periodic re-exploration from global best
  • Resume interrupted runs (aevyra-forge tune resume) — all state read from disk, no args needed

Recipe

  • VLLMConfig — 14 tunable serving args covering batching, caching, parallelism, and prefill
  • QuantRecipe — Layer 2 schema (method, kv_cache_quant, calibration_dataset) — scaffolded for v0.2
  • Recipe.diff() — human-readable change summary between any two recipes
  • Unique recipe IDs derived from config hash; full lineage via parent_id and generation

Benchmarking

  • bench.py — concurrent workload replay against live vLLM; returns throughput, P50/P99 latency, TTFT, TPOT, and accuracy
  • Dry-run mode with synthetic bench results for pipeline testing without a GPU

Workload

  • workload_from_jsonl — load real traffic traces (prompt + expected_output_tokens per line)
  • workload_synthetic / workload_shared_prefix / workload_concurrent_synthetic — generators for testing and dry-runs

Playbook

  • Bundled playbook.md with T4/A10/A100/H100 heuristics, safe search space, and forbidden combinations
  • --playbook flag for custom playbooks
  • aevyra-forge playbook show|validate

Persistence

  • ForgeStore / ForgeRun — run directory layout with config.json, experiments.jsonl, experiments.json, experiments.tsv, best_recipe.yaml
  • Interrupted runs detected automatically; resume restores full state from disk

CLI

  • aevyra-forge tune — start a new run
  • aevyra-forge tune resume — resume latest interrupted run
  • aevyra-forge report <dir> — human-readable TSV summary
  • aevyra-forge report <dir> --format json — machine-readable JSON for dashboards and tooling
  • aevyra-forge doctor — pre-flight checks: GPU, vLLM, API keys
  • aevyra-forge playbook show|validate

LLM backends

Anthropic (default), OpenAI, OpenRouter, Together AI, Groq, Ollama, and any OpenAI-compatible endpoint via the provider/model flag format.

Docs

Full documentation at aevyra.mintlify.app/forge/introduction

Install

pip install aevyra-forge
Assets 2
Loading