Expand AI Coach analytics and trends

AI_COACH.md

@@ -155,18 +155,28 @@ GET /api/analytics/sessions/:id
 
 Returns `404` if the session hasn't been analyzed yet (unknown harness, or no logs found in the session's time window).
 
+## Per-session view: gauges, trends, and examples
+
+The per-session **Analytics** tab renders each of the 4 scorecards as a circular gauge with a grade label (Great/Good/Fair/Poor), a WoW/MoM trend chip, and a 10-point sparkline. Trends are computed live, not stubbed: the API compares this session's score against the rolling average of the *same repo's* other analyzed sessions in the prior 7 days (WoW) and prior 30 days (MoM). Anti-patterns are grouped by category into collapsible sections, and each one that has matched examples (real flagged prompt/response snippets, capped by the rule's `examples` config) shows an expandable "N examples" row.
+
 ## Cross-session dashboard (AI Coach Analytics tab)
 
-While the per-session view above answers "how did this one session go," the **AI Coach Analytics** tab aggregates every analyzed session over a date range you pick:
+While the per-session view above answers "how did this one session go," the **AI Coach Analytics** tab aggregates every analyzed session over a date range you pick, across 6 sub-tabs: Dashboard, Patterns, Timeline, SDLC, Skill Finder, and Context Health.
 
 ```
-GET /api/ai-coach/dashboard?from=YYYY-MM-DD&to=YYYY-MM-DD   # avg score, per-category trends, top anti-patterns, daily activity, harness mix
-GET /api/ai-coach/patterns?from=...&to=...                  # hour × weekday request heatmap, calendar, per-repo project stats
-GET /api/ai-coach/timeline?from=...&to=...                  # most recent 200 sessions in range, with title/duration/score/cost
-GET /api/ai-coach/sdlc?from=...&to=...                       # work-type split (bug fix / feature / refactor / docs / config / code review / other) per request, overall and per-repo
+GET /api/ai-coach/dashboard?from=YYYY-MM-DD&to=YYYY-MM-DD     # avg score, per-category trends, top anti-patterns, daily activity, harness mix, token output/burndown
+GET /api/ai-coach/patterns?from=...&to=...                    # hour × weekday request heatmap, calendar, per-repo project stats
+GET /api/ai-coach/timeline?from=...&to=...                    # most recent 200 sessions in range, with title/duration/score/cost
+GET /api/ai-coach/sdlc?from=...&to=...                        # work-type split (bug fix / feature / refactor / docs / config / code review / other) per request, overall and per-repo
+GET /api/ai-coach/skill-finder?from=...&to=...                # underused harness features (skills, slash commands, plan mode, custom instructions, devcontainers, spec-driven workflows), ranked by occurrence
+GET /api/ai-coach/context-health?from=...&to=...              # single aggregate score + findings for context-engineering rules (AGENTS.md/CLAUDE.md, file references, devcontainers, spec structure)
 ```
 
-All four routes join `session_analytics` with `sessions` (and the `session_summary` artifact, for session titles), scoped to `s.created_at BETWEEN from AND to`, and exclude bare `zsh`/`bash` shell sessions. `workType` is a per-request classification already present on the parsed `SessionRequest` from the rule engine — no extra parsing pass needed.
+All six routes join `session_analytics` with `sessions` (and the `session_summary` artifact, for session titles), scoped to `s.created_at BETWEEN from AND to`, and exclude bare `zsh`/`bash` shell sessions. `workType` is a per-request classification already present on the parsed `SessionRequest` from the rule engine — no extra parsing pass needed.
+
+**Output / Burndown**: the Dashboard sub-tab also surfaces total input/output tokens and cost from the existing `sessions.estimated_input_tokens`/`estimated_output_tokens`/`estimated_cost_usd` columns (already populated by statusline parsing — see Spend Analytics), plus `budget_tokens`/`budget_usd` where a budget was set, giving a used/budget burndown view scoped to the same date range.
+
+**Skill Finder** and **Context Health** are computed entirely from anti-patterns already in `session_analytics.anti_patterns` — no new detectors. Skill Finder filters to feature-adoption rules (`no-skills`, `no-slash-commands`, `no-plan-mode`, `no-custom-instructions`, `no-devcontainer`, `no-spec-driven-development`, `agent-mode-for-asks`); Context Health filters to context-engineering rules (`context-engineering-gaps`, `no-file-context`, `excessive-file-context`, `no-custom-instructions`, `no-devcontainer`, `no-spec-structure`) and scores them with the same severity-weighted formula as the 4 main scorecards.
 
 ## Privacy & data flow
 

README.md

@@ -12,11 +12,11 @@ Local-first “mission control” for AI coding agent CLIs (and any shell comman
 
 ## ✨ Recently Shipped
 - **AI Coach analytics** (latest)
-  - New **Analytics** tab: per-session practice scorecards across 4 categories — Prompt Quality, Session Hygiene, Code Review, Tool Mastery
-  - 45 built-in detection rules (ported from [AI Engineering Coach](https://github.com/microsoft/AI-Engineering-Coach)) flag anti-patterns like runaway agent loops, late-night coding, lazy prompting, auto-model avoidance — each with severity, occurrence count, and a concrete suggestion
+  - New **Analytics** tab: per-session practice scorecards across 4 categories — Prompt Quality, Session Hygiene, Code Review, Tool Mastery — rendered as circular gauges with real WoW/MoM trend chips and sparklines (computed from the repo's own session history, not stubbed)
+  - 45 built-in detection rules (ported from [AI Engineering Coach](https://github.com/microsoft/AI-Engineering-Coach)) flag anti-patterns like runaway agent loops, late-night coding, lazy prompting, auto-model avoidance — each with severity, occurrence count, a concrete suggestion, and expandable real examples
   - Parses your own `~/.claude/projects` or `~/.codex` logs and scopes them to the exact AgentFleet session window — no duplicate/global stats
   - Backfillable: `pnpm --filter @agents_fleet/server exec tsx scripts/backfill-analytics.ts` analyzes any historical stopped sessions that predate this feature
-  - New **AI Coach Analytics** tab: cross-session dashboard over a date range — avg practice score, per-category trends, top anti-patterns across all sessions, an hour×weekday activity heatmap, a per-repo project breakdown, a session timeline, and an SDLC work-type split (bug fix / feature / refactor / docs / etc.)
+  - New **AI Coach Analytics** tab: cross-session dashboard over a date range, across 6 sub-tabs — Dashboard (avg practice score, per-category trends, top anti-patterns, daily activity, harness mix, token output/burndown), Patterns (hour×weekday activity heatmap, session calendar, per-repo breakdown), Timeline, SDLC work-type split, Skill Finder (underused harness features), and Context Health (AGENTS.md/file-context/devcontainer gaps)
   - Full breakdown: [AI_COACH.md](AI_COACH.md)
 - **AI session summary** (latest)
   - One-click plain-English summary of any session — title, what the agent did, and token/cost breakdown for the summary call

apps/server/src/routes/aiCoachAnalytics.ts

@@ -31,6 +31,10 @@ interface AnalyticsRow {
   s_repo_path: string;
   s_command: string;
   s_estimated_cost_usd: number;
+  s_estimated_input_tokens: number;
+  s_estimated_output_tokens: number;
+  s_budget_tokens: number | null;
+  s_budget_usd: number | null;
   summary_content: string | null;
 }
 
@@ -43,12 +47,16 @@ const JOIN_QUERY = `
     sa.group_scores,
     sa.parsed_requests,
     sa.created_at,
-    s.created_at         AS s_created_at,
-    s.ended_at           AS s_ended_at,
-    s.repo_path          AS s_repo_path,
-    s.command            AS s_command,
-    s.estimated_cost_usd AS s_estimated_cost_usd,
-    art.content          AS summary_content
+    s.created_at                AS s_created_at,
+    s.ended_at                  AS s_ended_at,
+    s.repo_path                 AS s_repo_path,
+    s.command                   AS s_command,
+    s.estimated_cost_usd        AS s_estimated_cost_usd,
+    s.estimated_input_tokens    AS s_estimated_input_tokens,
+    s.estimated_output_tokens   AS s_estimated_output_tokens,
+    s.budget_tokens             AS s_budget_tokens,
+    s.budget_usd                AS s_budget_usd,
+    art.content                 AS summary_content
   FROM session_analytics sa
   JOIN sessions s ON s.id = sa.session_id
   LEFT JOIN session_artifacts art
@@ -57,6 +65,32 @@ const JOIN_QUERY = `
     AND s.command NOT IN ('zsh', 'bash')
 `;
 
+// Rules that flag under-use of harness features (Skill Finder).
+const SKILL_FINDER_RULE_IDS = new Set([
+  "no-skills",
+  "no-slash-commands",
+  "no-plan-mode",
+  "no-custom-instructions",
+  "no-devcontainer",
+  "no-spec-driven-development",
+  "agent-mode-for-asks",
+]);
+
+// Rules that measure how well context is engineered for the agent (Context Health).
+const CONTEXT_HEALTH_RULE_IDS = new Set([
+  "context-engineering-gaps",
+  "no-file-context",
+  "excessive-file-context",
+  "no-custom-instructions",
+  "no-devcontainer",
+  "no-spec-structure",
+]);
+const CONTEXT_HEALTH_SEVERITY_PENALTY: Record<string, number> = {
+  high: 12,
+  medium: 7,
+  low: 3,
+};
+
 export function aiCoachAnalyticsRouter(): Router {
   const router = createRouter();
 
@@ -76,6 +110,13 @@ export function aiCoachAnalyticsRouter(): Router {
         topAntiPatterns: [],
         dailyActivity: [],
         harnessBreakdown: [],
+        tokenStats: {
+          totalInputTokens: 0,
+          totalOutputTokens: 0,
+          totalCostUsd: 0,
+          totalBudgetTokens: 0,
+          totalBudgetUsd: 0,
+        },
       });
     }
 
@@ -155,13 +196,34 @@ export function aiCoachAnalyticsRouter(): Router {
       avgScore: scoreCount > 0 ? Math.round(scoreSum / scoreCount) : null,
     }));
 
+    // Output / Burndown token measures
+    let totalInputTokens = 0;
+    let totalOutputTokens = 0;
+    let totalCostUsd = 0;
+    let totalBudgetTokens = 0;
+    let totalBudgetUsd = 0;
+    for (const row of rows) {
+      totalInputTokens += row.s_estimated_input_tokens || 0;
+      totalOutputTokens += row.s_estimated_output_tokens || 0;
+      totalCostUsd += row.s_estimated_cost_usd || 0;
+      totalBudgetTokens += row.s_budget_tokens || 0;
+      totalBudgetUsd += row.s_budget_usd || 0;
+    }
+
     res.json({
       sessionCount: rows.length,
       avgPracticeScore,
       groupAverages,
       topAntiPatterns,
       dailyActivity,
       harnessBreakdown,
+      tokenStats: {
+        totalInputTokens,
+        totalOutputTokens,
+        totalCostUsd,
+        totalBudgetTokens,
+        totalBudgetUsd,
+      },
     });
   });
 
@@ -318,5 +380,89 @@ export function aiCoachAnalyticsRouter(): Router {
     });
   });
 
+  // ── Skill Finder ────────────────────────────────────────────────────────────
+  // Surfaces harness features (skills, slash commands, plan mode, custom
+  // instructions, devcontainers, spec-driven workflows) that are underused
+  // across the date range, ranked by how often they were flagged.
+  router.get("/ai-coach/skill-finder", (req: Request, res: Response) => {
+    const range = parseDateRange(req);
+    if (!range) return jsonError(res, 400, "from and to query params required (YYYY-MM-DD)");
+
+    const db = getDb();
+    const rows = db.prepare(JOIN_QUERY).all(range.from, range.to) as AnalyticsRow[];
+
+    const totals: Record<
+      string,
+      { id: string; name: string; group: string; severity: string; suggestion: string; totalOccurrences: number; sessionCount: number }
+    > = {};
+    for (const row of rows) {
+      let patterns: { id: string; name: string; group: string; severity: string; suggestion: string; occurrences: number }[] = [];
+      try { patterns = JSON.parse(row.anti_patterns); } catch { continue; }
+      for (const p of patterns) {
+        if (!SKILL_FINDER_RULE_IDS.has(p.id)) continue;
+        if (!totals[p.id]) {
+          totals[p.id] = { id: p.id, name: p.name, group: p.group, severity: p.severity, suggestion: p.suggestion, totalOccurrences: 0, sessionCount: 0 };
+        }
+        totals[p.id].totalOccurrences += p.occurrences;
+        totals[p.id].sessionCount += 1;
+      }
+    }
+
+    const skills = Object.values(totals).sort(
+      (a, b) => b.sessionCount - a.sessionCount || b.totalOccurrences - a.totalOccurrences,
+    );
+
+    res.json({ sessionCount: rows.length, skills });
+  });
+
+  // ── Context Health ──────────────────────────────────────────────────────────
+  // Aggregates the context-engineering rule group into a single score, the
+  // same severity-weighted formula used for the per-session scorecards.
+  router.get("/ai-coach/context-health", (req: Request, res: Response) => {
+    const range = parseDateRange(req);
+    if (!range) return jsonError(res, 400, "from and to query params required (YYYY-MM-DD)");
+
+    const db = getDb();
+    const rows = db.prepare(JOIN_QUERY).all(range.from, range.to) as AnalyticsRow[];
+
+    const totals: Record<
+      string,
+      { id: string; name: string; severity: string; description: string; suggestion: string; totalOccurrences: number; sessionCount: number }
+    > = {};
+    let penalty = 0;
+    let sessionsWithFindings = 0;
+
+    for (const row of rows) {
+      let patterns: { id: string; name: string; severity: string; description: string; suggestion: string; occurrences: number }[] = [];
+      try { patterns = JSON.parse(row.anti_patterns); } catch { continue; }
+      let rowHasFinding = false;
+      for (const p of patterns) {
+        if (!CONTEXT_HEALTH_RULE_IDS.has(p.id)) continue;
+        rowHasFinding = true;
+        penalty += CONTEXT_HEALTH_SEVERITY_PENALTY[p.severity] ?? 5;
+        if (!totals[p.id]) {
+          totals[p.id] = { id: p.id, name: p.name, severity: p.severity, description: p.description, suggestion: p.suggestion, totalOccurrences: 0, sessionCount: 0 };
+        }
+        totals[p.id].totalOccurrences += p.occurrences;
+        totals[p.id].sessionCount += 1;
+      }
+      if (rowHasFinding) sessionsWithFindings += 1;
+    }
+
+    const maxPenalty = Math.max(rows.length, 1) * CONTEXT_HEALTH_RULE_IDS.size * 12;
+    const score = rows.length > 0 ? Math.max(0, Math.round(100 * (1 - penalty / maxPenalty))) : null;
+
+    const findings = Object.values(totals).sort(
+      (a, b) => b.sessionCount - a.sessionCount || b.totalOccurrences - a.totalOccurrences,
+    );
+
+    res.json({
+      sessionCount: rows.length,
+      sessionsWithFindings,
+      score,
+      findings,
+    });
+  });
+
   return router;
 }

apps/server/src/routes/analytics.ts

@@ -15,6 +15,69 @@ interface SessionAnalyticsRow {
   anti_patterns: string;
   group_scores: string | null;
   created_at: string;
+  repo_path: string;
+}
+
+interface HistoryRow {
+  group_scores: string | null;
+  practice_score: number | null;
+  created_at: string;
+}
+
+interface StoredGroupScore {
+  group: string;
+  score: number;
+  topIssue: string | null;
+  improvements: string[];
+  patternCount: number;
+}
+
+const HISTORY_WINDOW_DAYS = 30;
+const SPARKLINE_LENGTH = 10;
+
+function average(values: number[]): number | null {
+  if (values.length === 0) return null;
+  return Math.round(values.reduce((sum, v) => sum + v, 0) / values.length);
+}
+
+function buildTrends(
+  currentGroupScores: StoredGroupScore[],
+  history: HistoryRow[],
+  currentCreatedAt: string,
+) {
+  const now = new Date(currentCreatedAt).getTime();
+  const sevenDaysAgo = now - 7 * 24 * 60 * 60 * 1000;
+  const thirtyDaysAgo = now - HISTORY_WINDOW_DAYS * 24 * 60 * 60 * 1000;
+
+  // history is ordered most-recent-first; reverse for chronological sparklines
+  const chronological = [...history].reverse();
+
+  return currentGroupScores.map((g) => {
+    const wowScores: number[] = [];
+    const momScores: number[] = [];
+    const sparkline: number[] = [];
+
+    for (const h of chronological) {
+      const ts = new Date(h.created_at).getTime();
+      const parsed: StoredGroupScore[] = h.group_scores ? JSON.parse(h.group_scores) : [];
+      const match = parsed.find((p) => p.group === g.group);
+      if (!match) continue;
+      if (ts >= thirtyDaysAgo) momScores.push(match.score);
+      if (ts >= sevenDaysAgo) wowScores.push(match.score);
+      sparkline.push(match.score);
+    }
+    sparkline.push(g.score);
+
+    const wowAvg = average(wowScores);
+    const momAvg = average(momScores);
+
+    return {
+      ...g,
+      wowPct: wowAvg !== null ? g.score - wowAvg : 0,
+      momPct: momAvg !== null ? g.score - momAvg : 0,
+      sparkline: sparkline.slice(-SPARKLINE_LENGTH),
+    };
+  });
 }
 
 export function analyticsRouter(): Router {
@@ -23,19 +86,37 @@ export function analyticsRouter(): Router {
   router.get("/analytics/sessions/:id", (req, res) => {
     const db = getDb();
     const row = db
-      .prepare("SELECT * FROM session_analytics WHERE session_id = ?")
+      .prepare(
+        `SELECT sa.*, s.repo_path AS repo_path
+         FROM session_analytics sa
+         JOIN sessions s ON s.id = sa.session_id
+         WHERE sa.session_id = ?`,
+      )
       .get(req.params.id) as SessionAnalyticsRow | undefined;
 
     if (!row) {
       return jsonError(res, 404, "No analytics available for this session");
     }
 
+    const history = db
+      .prepare(
+        `SELECT sa.group_scores, sa.practice_score, s.created_at
+         FROM session_analytics sa
+         JOIN sessions s ON s.id = sa.session_id
+         WHERE s.repo_path = ? AND s.created_at < ? AND sa.session_id != ?
+         ORDER BY s.created_at DESC
+         LIMIT 30`,
+      )
+      .all(row.repo_path, row.created_at, row.session_id) as HistoryRow[];
+
+    const groupScores: StoredGroupScore[] = row.group_scores ? JSON.parse(row.group_scores) : [];
+
     res.json({
       sessionId: row.session_id,
       harness: row.harness,
       practiceScore: row.practice_score,
       antiPatterns: JSON.parse(row.anti_patterns),
-      groupScores: row.group_scores ? JSON.parse(row.group_scores) : [],
+      groupScores: buildTrends(groupScores, history, row.created_at),
       requests: JSON.parse(row.parsed_requests),
       createdAt: row.created_at,
     });