Skip to content

Commit 5090000

Browse files
adilhafeezclaude
adilhafeez
and
claude
committed
Add OpenClaw + Plano intelligent routing demo
Demonstrates preference-based routing for personal AI assistants: Kimi K2.5 handles conversation and agentic tasks, Claude handles code generation, testing, and complex reasoning — with zero application code changes and ~48% cost savings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 473996d commit 5090000

5 files changed

Lines changed: 297 additions & 0 deletions

File tree

‎demos/llm_routing/openclaw_routing/README.md‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# OpenClaw + Plano: Smart Model Routing for Personal AI Assistants
2+
3+
OpenClaw is an open-source personal AI assistant that connects to WhatsApp, Telegram, Slack, and Discord. By pointing it at Plano instead of a single LLM provider, every message is automatically routed to the best model — conversational requests go to Kimi K2.5 (cost-effective), while code generation, testing, and complex reasoning go to Claude (most capable) — with zero application code changes.
4+
5+
## Architecture
6+
7+
```
8+
[WhatsApp / Telegram / Slack / Discord]
9+
|
10+
[OpenClaw Gateway]
11+
ws://127.0.0.1:18789
12+
|
13+
[Plano :12000] ──────────────> Kimi K2.5 (conversation, agentic tasks)
14+
| $0.60/M input tokens
15+
|──────────────────────> Claude (code, tests, reasoning)
16+
|
17+
[Arch-Router 1.5B]
18+
(local via Ollama, ~200ms)
19+
```
20+
21+
Plano's 1.5B [Arch-Router](https://arxiv.org/abs/2506.16655) model analyzes each prompt locally and selects the best backend based on configured routing preferences.
22+
23+
## Prerequisites
24+
25+
- **Docker** running
26+
- **Ollama** installed ([ollama.com](https://ollama.com))
27+
- **Plano CLI**: `uv tool install planoai` or `pip install planoai`
28+
- **OpenClaw**: `npm install -g openclaw@latest`
29+
- **API keys**:
30+
- `MOONSHOT_API_KEY` — from [Moonshot AI](https://platform.moonshot.cn/)
31+
- `ANTHROPIC_API_KEY` — from [Anthropic](https://console.anthropic.com/)
32+
33+
## Quick Start
34+
35+
### 1. Set Environment Variables
36+
37+
```bash
38+
export MOONSHOT_API_KEY="your-moonshot-key"
39+
export ANTHROPIC_API_KEY="your-anthropic-key"
40+
```
41+
42+
### 2. Start the Demo
43+
44+
```bash
45+
cd demos/llm_routing/openclaw_routing
46+
bash run_demo.sh
47+
```
48+
49+
This will:
50+
- Pull the Arch-Router model into Ollama
51+
- Start Jaeger for tracing
52+
- Start Plano on port 12000
53+
54+
### 3. Configure OpenClaw
55+
56+
In `~/.openclaw/openclaw.json`, set:
57+
58+
```json
59+
{
60+
"agent": {
61+
"model": "kimi-k2.5",
62+
"baseURL": "http://127.0.0.1:12000/v1"
63+
}
64+
}
65+
```
66+
67+
Then run:
68+
69+
```bash
70+
openclaw onboard --install-daemon
71+
```
72+
73+
### 4. Test Routing
74+
75+
Run the test script to verify routing decisions:
76+
77+
```bash
78+
bash test_routing.sh
79+
```
80+
81+
## Demo Scenarios
82+
83+
| # | Message | Expected Route | Why |
84+
|---|---------|---------------|-----|
85+
| 1 | "Hey, what's up? Tell me something interesting." | **Kimi K2.5** | General conversation — cheap and fast |
86+
| 2 | "Remind me tomorrow at 9am and ping Slack about the deploy" | **Kimi K2.5** | Agentic multi-step task orchestration |
87+
| 3 | "Write a Python rate limiter with the token bucket algorithm" | **Claude** | Code generation — needs precision |
88+
| 4 | "Write unit tests for the auth middleware, cover edge cases" | **Claude** | Testing & evaluation — needs thoroughness |
89+
| 5 | "Compare WebSockets vs SSE vs polling for 10K concurrent users" | **Claude** | Complex reasoning — needs deep analysis |
90+
91+
OpenClaw's code doesn't change at all. It points at `http://127.0.0.1:12000/v1` instead of a direct provider URL. Plano's Arch-Router analyzes each prompt in ~200ms and picks the right backend.
92+
93+
## Monitoring
94+
95+
### Routing Decisions
96+
97+
Watch Plano logs for model selection:
98+
99+
```bash
100+
docker logs plano 2>&1 | grep MODEL_RESOLUTION
101+
```
102+
103+
### Jaeger Tracing
104+
105+
Open [http://localhost:16686](http://localhost:16686) to see full traces of each request, including which model was selected and the routing latency.
106+
107+
## Cost Impact
108+
109+
For a personal assistant handling ~1000 requests/day with a 60/40 conversation-to-code split:
110+
111+
| Without Plano (all Claude) | With Plano (routed) |
112+
|---|---|
113+
| 1000 req x Claude pricing | 600 req x Kimi K2.5 + 400 req x Claude |
114+
| ~$3.00/day input tokens | ~$0.36 + $1.20 = **$1.56/day** (~48% savings) |
115+
116+
Same quality where it matters (code, tests), lower cost where it doesn't (chat).
117+
118+
## Stopping the Demo
119+
120+
```bash
121+
bash run_demo.sh down
122+
```

‎demos/llm_routing/openclaw_routing/config.yaml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
version: v0.1.0
2+
3+
routing:
4+
model: Arch-Router
5+
llm_provider: arch-router
6+
7+
listeners:
8+
egress_traffic:
9+
address: 0.0.0.0
10+
port: 12000
11+
message_format: openai
12+
timeout: 30s
13+
14+
llm_providers:
15+
16+
# Arch Router - the 1.5B preference-aligned routing model (runs locally via Ollama)
17+
- name: arch-router
18+
model: arch/hf.co/katanemo/Arch-Router-1.5B.gguf:Q4_K_M
19+
base_url: http://host.docker.internal:11434
20+
21+
# Kimi K2.5 — Moonshot AI's open model (1T MoE, 32B active params)
22+
# Great for general conversation, agentic tasks, and multimodal work
23+
# OpenAI-compatible API at $0.60/M input, $2.50/M output tokens
24+
- model: openai/kimi-k2.5
25+
access_key: $MOONSHOT_API_KEY
26+
base_url: https://api.moonshot.ai/v1
27+
provider_interface: openai
28+
default: true
29+
routing_preferences:
30+
- name: general conversation
31+
description: general chat, greetings, casual conversation, Q&A, and everyday questions
32+
- name: agentic tasks
33+
description: coordinating multi-step workflows, device automation, scheduling, and task orchestration across channels
34+
35+
# Claude — Anthropic's most capable model
36+
# Best for complex reasoning, code, tool use, and evaluation
37+
- model: anthropic/claude-sonnet-4-5
38+
access_key: $ANTHROPIC_API_KEY
39+
routing_preferences:
40+
- name: testing and evaluation
41+
description: writing tests, running evaluations, QA checks, verifying correctness, and debugging failures
42+
- name: code generation
43+
description: generating code, writing scripts, implementing functions, and building tool integrations
44+
- name: complex reasoning
45+
description: multi-step analysis, planning, architectural decisions, and deep problem-solving
46+
47+
tracing:
48+
random_sampling: 100

‎demos/llm_routing/openclaw_routing/docker-compose.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
services:
2+
jaeger:
3+
build:
4+
context: ../../shared/jaeger
5+
ports:
6+
- "16686:16686"
7+
- "4317:4317"
8+
- "4318:4318"

‎demos/llm_routing/openclaw_routing/run_demo.sh‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
#!/bin/bash
2+
set -e
3+
4+
echo "=== OpenClaw + Plano Routing Demo ==="
5+
6+
# Check prerequisites
7+
command -v docker >/dev/null || { echo "Error: Docker not found"; exit 1; }
8+
command -v ollama >/dev/null || { echo "Error: Ollama not found. Install from https://ollama.com"; exit 1; }
9+
10+
# Check/create .env file
11+
if [ -f ".env" ]; then
12+
echo ".env file already exists"
13+
else
14+
if [ -z "${MOONSHOT_API_KEY:-}" ]; then
15+
echo "Error: MOONSHOT_API_KEY not set"
16+
exit 1
17+
fi
18+
if [ -z "${ANTHROPIC_API_KEY:-}" ]; then
19+
echo "Error: ANTHROPIC_API_KEY not set"
20+
exit 1
21+
fi
22+
echo "Creating .env file..."
23+
echo "MOONSHOT_API_KEY=$MOONSHOT_API_KEY" > .env
24+
echo "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY" >> .env
25+
fi
26+
27+
# Pull Arch-Router model if needed
28+
echo "Pulling Arch-Router model..."
29+
ollama pull hf.co/katanemo/Arch-Router-1.5B.gguf:Q4_K_M
30+
31+
start_demo() {
32+
# Start Jaeger for tracing
33+
echo "Starting Jaeger..."
34+
docker compose up -d
35+
36+
# Start Plano gateway
37+
echo "Starting Plano..."
38+
planoai up --service plano --foreground
39+
}
40+
41+
stop_demo() {
42+
docker compose down
43+
planoai down
44+
}
45+
46+
if [ "${1:-}" == "down" ]; then
47+
stop_demo
48+
else
49+
start_demo
50+
echo ""
51+
echo "=== Plano is running on http://localhost:12000 ==="
52+
echo "=== Jaeger UI at http://localhost:16686 ==="
53+
echo ""
54+
echo "Configure OpenClaw to use Plano as its LLM endpoint:"
55+
echo ' In ~/.openclaw/openclaw.json, set:'
56+
echo ' { "agent": { "model": "kimi-k2.5", "baseURL": "http://127.0.0.1:12000/v1" } }'
57+
echo ""
58+
echo "Then run: openclaw onboard --install-daemon"
59+
fi

‎demos/llm_routing/openclaw_routing/test_routing.sh‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
PLANO_URL="http://localhost:12000/v1/chat/completions"
5+
6+
echo "=== Testing Plano Routing Decisions ==="
7+
echo ""
8+
9+
# Scenario 1: General conversation -> should route to Kimi K2.5
10+
echo "--- Scenario 1: General Conversation (expect: Kimi K2.5) ---"
11+
curl -s "$PLANO_URL" \
12+
-H "Content-Type: application/json" \
13+
-d '{
14+
"model": "kimi-k2.5",
15+
"messages": [{"role": "user", "content": "Hey! What is the weather like today? Can you tell me a fun fact?"}]
16+
}' | jq '{model: .model, content: .choices[0].message.content[:100]}'
17+
echo ""
18+
19+
# Scenario 2: Agentic task -> should route to Kimi K2.5
20+
echo "--- Scenario 2: Agentic Task (expect: Kimi K2.5) ---"
21+
curl -s "$PLANO_URL" \
22+
-H "Content-Type: application/json" \
23+
-d '{
24+
"model": "kimi-k2.5",
25+
"messages": [{"role": "user", "content": "Schedule a reminder for tomorrow at 9am to review the pull request, then send a message to the team Slack channel about the deployment."}]
26+
}' | jq '{model: .model, content: .choices[0].message.content[:100]}'
27+
echo ""
28+
29+
# Scenario 3: Code generation -> should route to Claude
30+
echo "--- Scenario 3: Code Generation (expect: Claude) ---"
31+
curl -s "$PLANO_URL" \
32+
-H "Content-Type: application/json" \
33+
-d '{
34+
"model": "kimi-k2.5",
35+
"messages": [{"role": "user", "content": "Write a Python function that implements a rate limiter using the token bucket algorithm with async support."}]
36+
}' | jq '{model: .model, content: .choices[0].message.content[:100]}'
37+
echo ""
38+
39+
# Scenario 4: Testing/evaluation -> should route to Claude
40+
echo "--- Scenario 4: Testing & Evaluation (expect: Claude) ---"
41+
curl -s "$PLANO_URL" \
42+
-H "Content-Type: application/json" \
43+
-d '{
44+
"model": "kimi-k2.5",
45+
"messages": [{"role": "user", "content": "Write unit tests for this authentication middleware. Test edge cases: expired tokens, malformed headers, missing credentials, and concurrent requests."}]
46+
}' | jq '{model: .model, content: .choices[0].message.content[:100]}'
47+
echo ""
48+
49+
# Scenario 5: Complex reasoning -> should route to Claude
50+
echo "--- Scenario 5: Complex Reasoning (expect: Claude) ---"
51+
curl -s "$PLANO_URL" \
52+
-H "Content-Type: application/json" \
53+
-d '{
54+
"model": "kimi-k2.5",
55+
"messages": [{"role": "user", "content": "Analyze the trade-offs between using WebSockets vs SSE vs long-polling for real-time notifications in a distributed messaging system with 10K concurrent users."}]
56+
}' | jq '{model: .model, content: .choices[0].message.content[:100]}'
57+
echo ""
58+
59+
echo "=== Check Plano logs for MODEL_RESOLUTION details ==="
60+
echo "Run: docker logs plano 2>&1 | grep MODEL_RESOLUTION"

0 commit comments

Comments
 (0)