Skip to content

Commit f5c7f58

Browse files
authored
Merge pull request #10 from aevyraai/fix/add-introduction-doc
fix: add missing introduction.mdx to docs
2 parents ead63f5 + 3fec9d4 commit f5c7f58

1 file changed

Lines changed: 129 additions & 0 deletions

File tree

docs/introduction.mdx

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: Introduction
3+
description: Autonomous vLLM deployment optimizer — tunes your serving config overnight against your real workload.
4+
---
5+
6+
aevyra-forge tunes your vLLM deployment overnight. Give it a model, a GPU,
7+
and a workload trace. It runs an autonomous loop — propose a config, boot the
8+
server, benchmark against your real workload, keep or revert, repeat. By
9+
morning you have a deployment recipe that beats hand-tuned defaults, with a
10+
full audit trail of every experiment.
11+
12+
```bash
13+
pip install aevyra-forge
14+
```
15+
16+
vLLM exposes roughly 40 serving args. The defaults are conservative — designed
17+
to work safely on any GPU, not to max out any specific one. On a T4 with a
18+
chat workload the defaults leave significant throughput on the table:
19+
20+
```
21+
Baseline (defaults): 2718 tok/s P99: 241 ms
22+
After Forge (8 exps): 3421 tok/s P99: 187 ms (+26%)
23+
```
24+
25+
Forge finds that gain by searching the joint space of batching, caching, and
26+
memory knobs — and keeping only the changes that improve the score on your
27+
actual workload.
28+
29+
## Where Forge fits
30+
31+
```mermaid
32+
flowchart LR
33+
W[Witness\ncaptures trace]:::w
34+
V[Verdict\ngrades it]:::v
35+
O[Origin\nfinds culprits]:::o
36+
R[Reflex\nfixes prompts]:::r
37+
F[Forge\ntunes deployment]:::f
38+
39+
W --> V --> O --> R
40+
W --> F
41+
42+
classDef w fill:#3FBFFF,color:#fff,stroke:none
43+
classDef v fill:#6E3FF3,color:#fff,stroke:none
44+
classDef o fill:#FF6B6B,color:#fff,stroke:none
45+
classDef r fill:#2ECC71,color:#fff,stroke:none
46+
classDef f fill:#FF6B35,color:#fff,stroke:none
47+
```
48+
49+
Forge operates on the infrastructure layer. Where Reflex rewrites prompts and
50+
Origin diagnoses agent failures, Forge maximises throughput and minimises
51+
latency for a model that's already doing the right thing.
52+
53+
## The loop
54+
55+
```mermaid
56+
flowchart LR
57+
WL[Workload\nJSONL trace]:::data
58+
HW[Hardware\nauto-detected]:::data
59+
PB[Playbook\nheuristics]:::forge
60+
AG[Agent\n1 LLM call]:::forge
61+
VL[vLLM\nserver]:::bench
62+
BN[bench\nreplay workload]:::bench
63+
SC([Score\nthroughput × accuracy]):::score
64+
RY[best_recipe.yaml]:::output
65+
66+
WL & HW --> AG
67+
PB --> AG
68+
AG -->|mutated recipe| VL
69+
VL --> BN
70+
BN -->|BenchResult| SC
71+
SC -->|keep / revert| AG
72+
SC --> RY
73+
74+
classDef data fill:#6E3FF3,color:#fff,stroke:none
75+
classDef forge fill:#9B6BFF,color:#fff,stroke:none
76+
classDef bench fill:#3FBFFF,color:#fff,stroke:none
77+
classDef score fill:#FF6B35,color:#fff,stroke:none
78+
classDef output fill:#2ECC71,color:#fff,stroke:none
79+
```
80+
81+
Each iteration: the agent reads the playbook and experiment history, proposes
82+
one targeted change, and Forge measures whether it actually helps. The audit
83+
trail captures every decision — config, result, rationale — so you can see
84+
exactly how the winning recipe was found.
85+
86+
## Tuning layers
87+
88+
| Layer | What it tunes | Status |
89+
|---|---|---|
90+
| **1. Config** | vLLM serving args: batching, caching, parallelism | ✅ v0.1 |
91+
| **2. Quantization** | INT4/FP8/INT8, KV cache precision | 🔧 v0.2 |
92+
| **3. Kernel synthesis** | Custom kernels via AutoKernel | 🚧 v0.3 |
93+
94+
Layer 1 has the highest leverage per experiment because it requires no
95+
recompilation. Forge escalates to Layer 2 when Layer 1 converges.
96+
97+
## Works with any GPU and LLM
98+
99+
Auto-detects NVIDIA and AMD GPUs via `nvidia-smi` / `rocm-smi`. Works with
100+
any OpenAI-compatible LLM for the agent.
101+
102+
```bash
103+
pip install aevyra-forge # Claude included by default
104+
pip install aevyra-forge[openai] # add OpenAI / OpenRouter / Together / Groq
105+
```
106+
107+
| Provider | Env var |
108+
|---|---|
109+
| **Anthropic** | `ANTHROPIC_API_KEY` |
110+
| **OpenAI** | `OPENAI_API_KEY` |
111+
| **OpenRouter** | `OPENROUTER_API_KEY` |
112+
| **Ollama** ||
113+
114+
Python 3.10+. Apache-2.0 licensed.
115+
116+
<CardGroup cols={2}>
117+
<Card title="Quick start" icon="bolt" href="/forge/quickstart">
118+
Run your first tuning session in 15 minutes
119+
</Card>
120+
<Card title="Tutorial" icon="book-open" href="/forge/tutorial-colab-quickstart">
121+
Dry-run walkthrough with real log output
122+
</Card>
123+
<Card title="Concepts: Recipe" icon="file-code" href="/forge/concepts/recipe">
124+
The artifact Forge proposes, mutates, and keeps or reverts
125+
</Card>
126+
<Card title="Concepts: Playbook" icon="book" href="/forge/concepts/playbook">
127+
The agent's instruction manual
128+
</Card>
129+
</CardGroup>

0 commit comments

Comments
 (0)