Skip to content

Commit a693539

Browse files
Davin Hillsclaude
andcommitted
Add README with usage, concepts, architecture, and development guide
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 708bd07 commit a693539

1 file changed

Lines changed: 223 additions & 0 deletions

File tree

README.md

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
# RealityCheck
2+
3+
**Intent enforcement for agentic coding systems.**
4+
5+
RealityCheck verifies whether an implementation faithfully realizes the declared intent of a system as expressed in a `SPEC.md` (contractual obligations) and a `PLAN.md` (declared execution steps). It answers one question:
6+
7+
> *Did the code do what we said we would do, and only that?*
8+
9+
---
10+
11+
## Core Concepts
12+
13+
| Term | Definition |
14+
|---|---|
15+
| **Drift** | Code behavior that exists without spec or plan authorization |
16+
| **Violation** | Code behavior that contradicts a declared constraint |
17+
| **Coverage** | Whether each spec/plan item is implemented in the code |
18+
19+
If behavior exists without authorization, it is drift.
20+
If authorization exists without behavior, it is failure.
21+
If behavior contradicts authorization, it is violation.
22+
23+
---
24+
25+
## Pipeline Position
26+
27+
```
28+
SPEC.md → SpecCritic → PLAN.md → PlanCritic → CODE → RealityCheck → Prism
29+
```
30+
31+
RealityCheck runs after planning and before code quality review.
32+
33+
---
34+
35+
## Installation
36+
37+
```bash
38+
go install github.com/dshills/realitycheck/cmd/realitycheck@latest
39+
```
40+
41+
Or build from source:
42+
43+
```bash
44+
git clone https://github.com/dshills/realitycheck
45+
cd realitycheck
46+
go build ./cmd/realitycheck
47+
```
48+
49+
Requires `ANTHROPIC_API_KEY` to be set.
50+
51+
---
52+
53+
## Usage
54+
55+
```bash
56+
realitycheck check [path] [flags]
57+
```
58+
59+
### Required flags
60+
61+
```
62+
--spec <file> Path to SPEC.md
63+
--plan <file> Path to PLAN.md
64+
```
65+
66+
### Common flags
67+
68+
```
69+
--code-root <dir> Root directory to analyze (default: cwd)
70+
--format json|md Output format (default: json)
71+
--out <file> Write output to file instead of stdout
72+
--profile <name> Enforcement profile: general, strict-api, data-pipeline, library
73+
--strict No inferred intent; escalate drift severities
74+
--fail-on <verdict> Exit 2 if verdict >= level (ALIGNED|PARTIALLY_ALIGNED|DRIFT_DETECTED|VIOLATION)
75+
--severity-threshold <s> Filter output to findings at or above INFO|WARN|CRITICAL
76+
--model <id> Anthropic model ID (default: claude-opus-4-6)
77+
--offline Skip ANTHROPIC_API_KEY pre-flight check
78+
--verbose Print execution trace to stderr
79+
--debug Dump assembled prompt to stderr
80+
```
81+
82+
### Example
83+
84+
```bash
85+
realitycheck check \
86+
--spec specs/SPEC.md \
87+
--plan specs/PLAN.md \
88+
--code-root . \
89+
--format md \
90+
--fail-on DRIFT_DETECTED
91+
```
92+
93+
---
94+
95+
## Output
96+
97+
### Verdicts
98+
99+
| Verdict | Meaning |
100+
|---|---|
101+
| `ALIGNED` | Code matches spec and plan |
102+
| `PARTIALLY_ALIGNED` | Gaps or incomplete implementation |
103+
| `DRIFT_DETECTED` | Unauthorized behavior present |
104+
| `VIOLATION` | Code contradicts a declared constraint |
105+
106+
### Scoring
107+
108+
Score starts at 100 and decreases deterministically:
109+
110+
- **−20** per CRITICAL finding
111+
- **−7** per WARN finding
112+
- **−2** per INFO finding
113+
- Clamped to `[0, 100]`
114+
115+
Scoring is always computed locally — never by the LLM.
116+
117+
### Exit codes
118+
119+
| Code | Meaning |
120+
|---|---|
121+
| `0` | Success |
122+
| `2` | `--fail-on` threshold met |
123+
| `3` | Input error (missing flags, file not found) |
124+
| `4` | LLM / provider error |
125+
| `5` | LLM produced unrecoverable invalid output |
126+
127+
### JSON output (excerpt)
128+
129+
```json
130+
{
131+
"tool": "realitycheck",
132+
"version": "0.1.0",
133+
"summary": {
134+
"verdict": "DRIFT_DETECTED",
135+
"score": 80,
136+
"critical_count": 0,
137+
"warn_count": 1,
138+
"info_count": 0
139+
},
140+
"drift": [
141+
{
142+
"id": "DRIFT-001",
143+
"severity": "WARN",
144+
"description": "Undocumented retry loop in HTTP client",
145+
"evidence": [{ "path": "internal/client/client.go", "symbol": "retryRequest" }],
146+
"why_unjustified": "No spec or plan item authorizes automatic retries.",
147+
"recommendation": "Add to spec or remove."
148+
}
149+
]
150+
}
151+
```
152+
153+
---
154+
155+
## Profiles
156+
157+
Profiles modulate how the LLM interprets the spec and plan.
158+
159+
| Profile | Description |
160+
|---|---|
161+
| `general` | Default balanced analysis |
162+
| `strict-api` | Any undeclared HTTP handler or outbound call is CRITICAL drift |
163+
| `data-pipeline` | Any undeclared write to an external store is CRITICAL drift |
164+
| `library` | Drift evaluated only on exported symbols |
165+
166+
---
167+
168+
## Strict Mode
169+
170+
`--strict` enables adversarial analysis:
171+
172+
- Unclear coverage → `NOT_IMPLEMENTED`
173+
- Missing evidence → absent
174+
- WARN drift → CRITICAL, INFO drift → WARN
175+
176+
---
177+
178+
## Architecture
179+
180+
```
181+
cmd/realitycheck/ CLI entry point (cobra)
182+
internal/schema/ Canonical data types
183+
internal/spec/ SPEC.md parser
184+
internal/plan/ PLAN.md parser
185+
internal/codeindex/ Code inventory (symbols, tests, manifests)
186+
internal/profile/ Enforcement profiles
187+
internal/llm/ LLM provider, prompt builder, response validator
188+
internal/coverage/ Coverage analysis helpers
189+
internal/drift/ Drift severity helpers
190+
internal/verdict/ Scoring and verdict logic
191+
internal/render/ JSON and Markdown renderers
192+
```
193+
194+
Symbol extraction is regex-based (no full AST). Supported languages: Go, JavaScript/TypeScript, Python, Rust.
195+
196+
---
197+
198+
## Development
199+
200+
```bash
201+
# Run all tests
202+
go test ./...
203+
204+
# Run with race detector
205+
go test -race ./...
206+
207+
# Run integration tests (uses mock LLM, no API key required)
208+
go test -race -tags=integration ./...
209+
210+
# Build binary
211+
go build ./cmd/realitycheck
212+
213+
# Lint
214+
go vet ./...
215+
```
216+
217+
---
218+
219+
## Security & Privacy
220+
221+
- No telemetry emitted by default
222+
- Raw code is **never** sent to the LLM — only file paths, symbol names, and dependency manifest text
223+
- `--debug` prints the assembled prompt to stderr (no redaction needed since code content is absent)

0 commit comments

Comments
 (0)