Skip to content

Commit 7a52932

Browse files
authored
[codex] Add optimization intelligence reports (#129)
* Add optimization intelligence reports * Improve optimization TUI readability
1 parent d0ea3d6 commit 7a52932

35 files changed

Lines changed: 2814 additions & 22 deletions

README.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,11 @@ tokenleak focus --provider codex --days 30
142142
tokenleak nutrition
143143
tokenleak nutrition --days 30 --format json
144144

145+
# Optimization intelligence
146+
tokenleak simulate-routing --days 30
147+
tokenleak waste --severity high
148+
tokenleak behavior-diff --provider claude-code,codex --days 30 --format json
149+
145150
# Authenticate Cursor and sync its local cache
146151
tokenleak cursor login --name work
147152

@@ -151,7 +156,7 @@ tokenleak --list-providers
151156

152157
### Analysis commands
153158

154-
Tokenleak ships three dedicated investigation commands in addition to the main dashboard flow:
159+
Tokenleak ships dedicated investigation and optimization commands in addition to the main dashboard flow:
155160

156161
```bash
157162
# Explain what drove a specific day
@@ -181,12 +186,24 @@ tokenleak nutrition --days 30
181186

182187
# Emit the AI ROI report as JSON
183188
tokenleak nutrition --format json --output ai-roi.json
189+
190+
# Estimate savings from model routing
191+
tokenleak simulate-routing --days 30
192+
193+
# Detect agent waste signals with evidence and recipes
194+
tokenleak waste --days 30
195+
196+
# Compare two agent/provider/model cohorts
197+
tokenleak behavior-diff --provider claude-code,codex --days 30
184198
```
185199

186200
- `tokenleak explain <date>` builds a narrative day report with top providers, sessions, projects, models, and anomaly flags.
187201
- `tokenleak focus` ranks sessions by a deep-work score derived from duration, token density, and project streak.
188202
- `tokenleak replay [date]` shows a chronological timeline of all sessions for a day, clustering events into flow blocks with a pulse chart and flow/think ratio. Defaults to today. Pass `--interactive` (or `-i`) to open a browser scrub UI on `http://localhost:3567` — drag the timeline, press space to play the day at 60–600× speed, watch the cumulative cost odometer tick up. Combine with `--open` to launch the browser automatically.
189203
- `tokenleak nutrition` powers the TUI **AI ROI** view. It resolves local Git repo roots from provider project paths, runs read-only `git log --numstat`, and reports tokens/cost per commit and changed line. `No Git signal` means Tokenleak saw AI usage for a repo path but found no commits in the selected date window; switch to a wider window or ensure the project path exists locally as a Git worktree.
204+
- `tokenleak simulate-routing` re-prices historical events under conservative downgrade rules so pro users can estimate savings before changing model habits or team guidance.
205+
- `tokenleak waste` detects deterministic waste signals such as context drag, repeated prompt clusters, model churn, cache misses, and premium models used for small tasks.
206+
- `tokenleak behavior-diff` compares cohorts such as provider-vs-provider or model-vs-model and emits deterministic takeaways for engineering teams.
190207

191208
### Cursor commands
192209

packages/cli/src/cli.test.ts

Lines changed: 65 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -797,7 +797,9 @@ describe('CLI invocation', () => {
797797
expect(stdout).toContain('--more');
798798
expect(stdout).toContain('tokenleak explain <date>');
799799
expect(stdout).toContain('focus');
800-
expect(stdout).not.toContain('tokenleak waste');
800+
expect(stdout).toContain('tokenleak simulate-routing');
801+
expect(stdout).toContain('tokenleak waste');
802+
expect(stdout).toContain('tokenleak behavior-diff');
801803
expect(stdout).toContain('interactive launcher');
802804
expect(stdout).toContain('Examples:');
803805
});
@@ -874,16 +876,74 @@ describe('CLI invocation', () => {
874876
expect(stderr).toContain('--days must be a positive number');
875877
});
876878

877-
test('waste is not exposed as a standalone command', async () => {
879+
test('waste --help exits with code 0 and prints waste usage', async () => {
878880
const proc = Bun.spawn(['bun', cliPath, 'waste', '--help'], {
879881
stdout: 'pipe',
880882
stderr: 'pipe',
881883
});
882884
const exitCode = await proc.exited;
883-
const stderr = await new Response(proc.stderr).text();
885+
const stdout = await new Response(proc.stdout).text();
884886

885-
expect(exitCode).toBe(1);
886-
expect(stderr).toContain('Advisor view for Waste Patterns');
887+
expect(exitCode).toBe(0);
888+
expect(stdout).toContain('tokenleak waste');
889+
expect(stdout).toContain('agent waste signals');
890+
});
891+
892+
test('waste emits a JSON agent waste report', async () => {
893+
const { env, cleanup } = createProviderFixtureEnv();
894+
895+
try {
896+
const waste = Bun.spawn(['bun', cliPath, 'waste', '--format', 'json', '--provider', 'pi'], {
897+
stdout: 'pipe',
898+
stderr: 'pipe',
899+
env,
900+
});
901+
const wasteExit = await waste.exited;
902+
const wasteStdout = await new Response(waste.stdout).text();
903+
expect(wasteExit).toBe(0);
904+
expect(JSON.parse(wasteStdout).summary).toBeDefined();
905+
} finally {
906+
cleanup();
907+
}
908+
});
909+
910+
test('simulate-routing emits a JSON routing report', async () => {
911+
const { env, cleanup } = createProviderFixtureEnv();
912+
913+
try {
914+
const simulate = Bun.spawn(['bun', cliPath, 'simulate-routing', '--format', 'json', '--provider', 'pi'], {
915+
stdout: 'pipe',
916+
stderr: 'pipe',
917+
env,
918+
});
919+
const simulateExit = await simulate.exited;
920+
const simulateStdout = await new Response(simulate.stdout).text();
921+
expect(simulateExit).toBe(0);
922+
expect(JSON.parse(simulateStdout).strategy).toBe('conservative');
923+
} finally {
924+
cleanup();
925+
}
926+
});
927+
928+
test('behavior-diff emits a JSON behavior diff report', async () => {
929+
const { env, cleanup } = createProviderFixtureEnv();
930+
931+
try {
932+
const diff = Bun.spawn(
933+
['bun', cliPath, 'behavior-diff', '--format', 'json', '--provider', 'pi,claude-code'],
934+
{
935+
stdout: 'pipe',
936+
stderr: 'pipe',
937+
env,
938+
},
939+
);
940+
const diffExit = await diff.exited;
941+
const diffStdout = await new Response(diff.stdout).text();
942+
expect(diffExit).toBe(0);
943+
expect(JSON.parse(diffStdout).takeaways).toBeArray();
944+
} finally {
945+
cleanup();
946+
}
887947
});
888948

889949
test('--version prints version', async () => {

0 commit comments

Comments
 (0)