docs: add rule/metric authoring guide (#25)

CONTRIBUTING.md

@@ -32,3 +32,13 @@ request features. Before filing a new issue, please check if one already exists.
 
 If you discover a security vulnerability, please follow the instructions in [SECURITY.md](SECURITY.md).
 **Do not** report security vulnerabilities through public GitHub issues.
+
+## Creating Rules and Metrics
+
+Detection rules and metrics are the primary extensibility surface of AI Engineer Coach. Each one is
+a self-contained markdown file with YAML frontmatter and a small DSL — no code changes required to
+ship a new one. Built-in rules live in [`src/core/rules/`](src/core/rules/) and metrics in
+[`src/core/metrics/`](src/core/metrics/).
+
+See [docs/AUTHORING_RULES.md](docs/AUTHORING_RULES.md) for the full authoring guide: file format,
+annotated rule and metric examples, the local testing workflow, and links to the DSL reference.

docs/AUTHORING_RULES.md

@@ -0,0 +1,145 @@
+# Authoring Rules and Metrics
+
+Detection rules and metrics are the primary extensibility surface of AI Engineer Coach. Every rule
+and metric is a self-contained markdown file with YAML frontmatter and a small DSL — no code changes
+required to ship a new one.
+
+This guide covers contributing a built-in rule or metric to this repository. For the in-extension
+authoring flow (live-test, threshold sliders, AI-assisted drafting), see the
+[Rule Editor guide](https://microsoft.github.io/AI-Engineering-Coach/improve/rule-editor/).
+
+## Where rules and metrics live
+
+| Layer | Location | Trust | Use when |
+|---|---|---|---|
+| Built-in | [`src/core/rules/`](../src/core/rules/), [`src/core/metrics/`](../src/core/metrics/) | Trusted | Contributing a rule for everyone via this repo |
+| Personal | `~/.ai-engineer-coach/rules/`, `~/.ai-engineer-coach/metrics/` | Prompted on first load | Private rules shared across all your workspaces |
+| Project | `<workspace>/.ai-engineer-coach/rules/`, `<workspace>/.ai-engineer-coach/metrics/` | Prompted on first load | Workspace-specific rules checked into a repo |
+
+Personal and project rules follow the same file format as built-in rules but are loaded at runtime
+through the trust gate in [`src/core/rule-trust.ts`](../src/core/rule-trust.ts).
+
+For the full DSL reference — field schema, function catalog, and metric primitives — open the
+**DSL Reference** modal inside the extension (Rule Editor → DSL Reference) or the
+[Rule Playground](https://microsoft.github.io/AI-Engineering-Coach/improve/rule-playground/).
+
+## Anatomy of a rule
+
+A rule file lives at `src/core/rules/<rule-id>.md`. Below is
+[`lazy-prompting.md`](../src/core/rules/lazy-prompting.md) annotated section by section:
+
+````markdown
+---
+id: lazy-prompting              # Stable identifier, must match the filename
+name: Lazy Prompting             # Human-readable name shown in the UI
+group: prompt-quality            # Practice category: prompt-quality | session-hygiene |
+                                 #   code-review | tool-mastery | context-management
+severity: medium                 # low | medium | high — drives score weighting
+scope: requests                  # requests (per-message) or sessions (per-session)
+version: 1                       # Bump when changing detection logic
+tags: [prompt, quality, short]   # Free-form tags surfaced in the rule filter
+thresholds:                      # Tunable knobs. Referenced as thresholds.<name> in DSL.
+  minChars: 30                   #   Sliders for these appear in the Rule Editor.
+  maxRatio: 0.3
+  minSample: 10
+---
+
+# Description
+Short, user-facing summary of what this rule detects.
+
+# When Triggered
+One-line finding text. Supports template variables: {{count}}, {{total}}, {{pct}},
+and anything you emit via extra.<key> in the detection logic.
+
+# How to Improve
+Concrete, actionable recommendation shown alongside the finding.
+
+# Examples
+Template for the per-occurrence example list. {{message}} and any field on the
+matched row (e.g. {{extra.charCount}}) are available.
+
+# Detection Logic
+```detect
+scan: requests                                    # What to iterate over
+match: messageLength < thresholds.minChars        # Predicate; uses field schema
+       AND messageLength > 0                      #   + thresholds.* + DSL helpers
+aggregate: ratio                                  # count | ratio | sum | someWhere | ...
+check: ratio > thresholds.maxRatio                # The trigger condition
+       AND count > thresholds.minSample
+examples: "{{messageText | truncate:80}}" ({{messageLength}} chars)
+```
+
+# Tests
+```test
+{messageText: "fix bug", messageLength: 7} -> triggered
+{messageText: "Refactor the authentication middleware to use JWT tokens", messageLength: 60} -> clean
+{messageText: "", messageLength: 0} -> clean
+```
+````
+
+The `# Tests` block is optional but strongly encouraged — each line is a synthetic row evaluated
+against the rule. Tests run as part of the standard `npm test` suite.
+
+## Anatomy of a metric
+
+Metrics are simpler than rules — they emit a single named value that rules and dashboards can
+consume. A metric file lives at `src/core/metrics/<metric-id>.metric.md`. Below is
+[`weekend-requests.metric.md`](../src/core/metrics/weekend-requests.metric.md):
+
+```markdown
+---
+id: weekend-requests             # Stable identifier; reference as metrics.weekendRequests
+name: Weekend Requests
+scope: requests                  # requests or sessions
+version: 1
+tags: [wellbeing, time]
+---
+
+# Filter
+dayOfWeek(timestamp) == 0 OR dayOfWeek(timestamp) == 6   # Predicate over the scope
+
+# Metric
+ratio                            # Aggregation: count | ratio | sum | avg
+
+# Examples
+{{messageText | truncate:60}} ({{dayOfWeek(timestamp)}})
+```
+
+Once defined, a metric is automatically available in the Rule Playground metric list and can be
+referenced from any rule's detection logic.
+
+## Local testing workflow
+
+Before opening a pull request:
+
+1. **Install and build**
+   ```bash
+   npm install
+   npm run build
+   ```
+2. **Run the rule and metric tests** — these load every markdown file under `src/core/rules/` and
+   `src/core/metrics/`, validate the frontmatter, and execute any `# Tests` blocks:
+   ```bash
+   npm test
+   ```
+3. **Live-test against your own data.** Launch the extension in the VS Code Extension Development
+   Host (`F5`), open the **Rule Editor**, find your new rule, and click **Test Rule**. Use the
+   threshold sliders to sweep values without re-editing the markdown.
+4. **Prototype DSL expressions** in the **Rule Playground** before committing them — it shares the
+   same runtime as the rule engine, so anything that evaluates there will behave identically in a
+   rule.
+5. **Lint**
+   ```bash
+   npm run lint
+   ```
+
+## References
+
+- Built-in rules: [`src/core/rules/`](../src/core/rules/) — 45 worked examples across all categories
+- Built-in metrics: [`src/core/metrics/`](../src/core/metrics/)
+- A minimal rule with tests: [`src/core/rules/lazy-prompting.md`](../src/core/rules/lazy-prompting.md)
+- A rule using a custom aggregator and `extra.*` fields: [`src/core/rules/yolo-mode.md`](../src/core/rules/yolo-mode.md)
+- A minimal metric: [`src/core/metrics/weekend-requests.metric.md`](../src/core/metrics/weekend-requests.metric.md)
+- Anti-Patterns overview: [content/improve/anti-patterns.md](content/improve/anti-patterns.md)
+- Rule Editor guide: [content/improve/rule-editor.md](content/improve/rule-editor.md)
+- Rule Playground guide: [content/improve/rule-playground.md](content/improve/rule-playground.md)