Add --catalog json discovery contract, AGENTS.md, llms.txt (v0.1.7)

Wire extract-cli fully into the contract-ops suite's agent conventions:

- extract --catalog json: the suite-wide discovery contract
  {name, bin, version, description, commands[], exitCodes}, mirroring
  nda-review-cli/docx2pdf/sign. Agents call it at startup instead of
  hardcoding commands/flags. A test asserts it never drifts from the
  real argparse parser. Added to bash/zsh completion flag lists.
- AGENTS.md: agent contract in the canonical section order
  (output contract / exit codes / discovery / failure -> recovery).
- llms.txt: machine-readable tool summary at the repo root.
- pyproject: suite-standard keywords (contract-ops, agent-first,
  legal-tech); ship AGENTS.md + llms.txt in the sdist.
- README opens with Run this / Where to go next; --catalog documented
  in README + docs/INTEROP.md.

No schema or extraction-logic change (extractor_version unchanged).
mypy --strict clean; 110 passed / 2 skipped; spec-check OK.

Author: DrBaher

AGENTS.md

@@ -0,0 +1,87 @@
+# Agents
+
+Drive `extract-cli` from an LLM agent or non-interactive client. Same agent
+contract as the rest of the contract-ops suite: a stable machine-readable
+catalog, JSON on stdout, humans on stderr, and a small documented exit-code set.
+
+`extract-cli` is the suite's **open-loop front door**: hand it any contract
+(`.md` / `.txt` / `.html` / `.docx` / `.pdf`, yours or a counterparty's) and it
+returns structured JSON the rest of the pipeline can consume. Every field
+carries a `confidence` and a `source` — **verify, don't trust**.
+
+## Output contract
+
+- **Success**: a single JSON object to **stdout**, exit `0`. This is the machine
+  payload; it's the default (no `--json` needed, though `--json` forces it).
+- Every extracted scalar is the envelope `{value, confidence, source}`;
+  "not found" is the canonical `{value: null, confidence: 0.0, source: "none"}`.
+  Lists (`parties`, `clauses`, `defined_terms`) carry per-item
+  `confidence`/`source`. `source ∈ {deterministic, llm, none}`.
+- `_meta` records `extractor_version`, `tiers_used`, and `llm_used`.
+- The output shape is locked by a JSON Schema —
+  [`docs/spec/extract-output.schema.json`](docs/spec/extract-output.schema.json),
+  also printed by `extract schema`. Validate against it instead of trusting
+  field shapes by convention. (Note: the `--no-confidence` projection is a
+  reduced convenience view, **not** governed by the schema.)
+- **stderr** is for humans only: `--why` rationale, warnings, and errors.
+  stdout stays clean JSON even under `--why`.
+- **Failure**: a one-line `error: <message>` on **stderr**, non-zero exit.
+  The error shape is a flat string (the suite is not uniform on error-object
+  shape) — **branch on the exit code, never on the human-readable message.**
+
+## Exit codes
+
+| Code | Meaning |
+|------|---------|
+| `0` | Success. |
+| `1` | Low-signal document — no high-signal fields (parties/clauses/dates) could be extracted; e.g. a scanned/image-only or empty file. A **finding**, not a crash: valid JSON is still emitted on stdout. |
+| `2` | Bad usage / user-actionable error (unreadable path, bad flag value, unsupported completion shell). |
+
+## Discovery
+
+Never hardcode command or flag names — call the catalog at startup:
+
+```bash
+extract --catalog json    # {name, bin, version, description, commands[], exitCodes}
+```
+
+`--catalog json` is the suite-wide discovery contract (parallel to
+`nda-review-cli --catalog json`, `docx2pdf --catalog json`,
+`sign --catalog json`). It is **complete, accurate, and stable across minor
+versions** — a test asserts it never drifts from the real parser.
+
+Tool-specific discovery extras:
+
+```bash
+extract schema            # the output JSON Schema (the cross-CLI data contract)
+extract fields            # extractable fields and the tier that produces each
+extract fields --json     # ...as JSON
+extract demo              # run on a bundled fixture (zero-config first run)
+extract --version
+```
+
+## Failure → recovery
+
+| Symptom | Diagnose | Recover |
+|---|---|---|
+| Exit `1`, warning "no high-signal fields" | The document is likely scanned/image-only or has no recognizable structure. JSON is still emitted. | OCR the source first, or feed a text/`.docx`/`.md` version. The empty-but-valid JSON is safe to pass downstream. |
+| Exit `2`, `error: ...` | `extract --catalog json` (or `extract <cmd> --help`) for the real surface. | Fix the path/flag and retry. |
+| `clauses: []` on a real contract | The `.docx` likely auto-numbers via Word's numbering with no heading style (its numbers live only in `numbering.xml`), so the deterministic cascade sees no headings. | Re-run with `--llm` (opt-in): when no clauses are detected, the LLM is asked for section headings, normalized through the same canonical vocabulary and emitted with `tier: "llm"`, `source: "llm"`, and a modest confidence. Requires `~/.config/contract-ops/llm.json`. |
+| Low-fidelity `.docx`/`.pdf` text | The stdlib best-effort reader ran (no extras installed). | `pip install "extract-cli[docx]"` and/or `"extract-cli[pdf]"` for higher fidelity. The core always works without them. |
+| `--llm` only printed a warning | No LLM config found. | Copy [`config/llm.json.example`](config/llm.json.example) to `~/.config/contract-ops/llm.json`. Without it, deterministic output is still returned in full. |
+
+## Recommended usage
+
+```bash
+# Inspect any contract's structure, one tool for five formats.
+extract counterparty.docx | jq '{parties: [.parties[].name],
+  governing_law: .governing_law.value, clauses: [.clauses[].canonical_title]}'
+
+# Gate a workflow on extraction confidence (non-zero exit if any clause is shaky).
+extract draft.docx | jq -e '.clauses | all(.confidence > 0.7)' && echo ok
+```
+
+The integration contract is the **output schema** + the **shared canonical
+clause vocabulary** (`canonical_title` values match what `template-vault-cli`
+detects and `nda-review-cli` keys policy on) — not per-tool flags. See
+[`docs/INTEROP.md`](docs/INTEROP.md).

CHANGELOG.md

@@ -6,6 +6,25 @@ to [Semantic Versioning](https://semver.org/). Per the suite convention
 (see [`docs/INTEROP.md`](docs/INTEROP.md)), **backward-incompatible changes to
 the output schema require a major version bump**; new optional fields are minor.
 
+## [0.1.7] - 2026-05-22
+
+### Added
+- **`extract --catalog json` — the suite's shared discovery contract.** Emits
+  `{name, bin, version, description, commands[], exitCodes}` (mirroring
+  `nda-review-cli --catalog json` / `docx2pdf --catalog json` /
+  `sign --catalog json`) so agents can learn every command and flag at startup
+  instead of hardcoding them. A test asserts the catalog never drifts from the
+  real argparse parser. Also added to the bash/zsh completion flag lists.
+- **`AGENTS.md`** — the agent contract in the suite's canonical section order
+  (output contract / exit codes / discovery / failure → recovery).
+- **`llms.txt`** — machine-readable tool summary at the repo root.
+
+### Changed
+- Packaging: added the suite-standard keywords (`contract-ops`, `agent-first`,
+  `legal-tech`); README now opens with `## Run this` / `## Where to go next`;
+  `--catalog json` documented in the README and `docs/INTEROP.md`. No schema or
+  extraction-logic change (`extractor_version` unchanged).
+
 ## [0.1.6] - 2026-05-21
 
 ### Docs

README.md

@@ -23,6 +23,30 @@ ingest (extract) → review → diff → convert → sign
    ^you are here
 ```
 
+## Run this
+
+```bash
+pipx run extract-cli demo        # zero-config: extract a bundled NDA → structured JSON
+# or, installed:  pip install extract-cli && extract demo
+```
+
+That prints the full output contract — parties, dates, term, governing law, and
+a clause map normalized onto the suite's canonical vocabulary — for a bundled
+fixture, with no setup and no network. Point it at your own file with
+`extract path/to/contract.docx`.
+
+## Where to go next
+
+- **New here?** Keep reading — [What it does](#what-it-does) and
+  [The two extraction tiers](#the-two-extraction-tiers).
+- **Driving it from an agent?** See [`AGENTS.md`](AGENTS.md) and call
+  `extract --catalog json` at startup to discover commands/flags. The output
+  shape is locked by [`docs/spec/extract-output.schema.json`](docs/spec/extract-output.schema.json).
+- **Wiring it into the pipeline?** See [`docs/INTEROP.md`](docs/INTEROP.md) — the
+  contract is the output schema + the shared clause vocabulary.
+- **Contributing / building a sibling CLI?** [`CONTRIBUTING.md`](CONTRIBUTING.md)
+  and [ARCHITECTURE.md](ARCHITECTURE.md).
+
 ## What it does
 
 Give it a contract in **`.md` / `.txt` / `.html`** (native), **`.docx`**, or
@@ -77,6 +101,7 @@ for them.
 
 ```bash
 extract <path>            # parse a document → structured JSON on stdout (default)
+extract --catalog json    # machine-readable catalog of commands/flags (agents call at startup)
 extract schema            # print the output JSON Schema (the cross-CLI contract)
 extract fields            # list extractable fields and their tier
 extract demo              # run on a bundled fixture and show the narrative
@@ -87,6 +112,7 @@ extract completion bash   # emit a shell-completion script (bash|zsh)
 
 | Flag | Meaning |
 |---|---|
+| `--catalog json` | Print the machine-readable command/flag catalog and exit (the suite discovery contract; agents call this at startup) |
 | `--llm` | Opt-in LLM enrichment of fuzzy fields (off by default) |
 | `--fields a,b,c` | Emit only a subset of top-level fields (e.g. `parties,clauses`) |
 | `--format json\|table` | Output format (default `json`) |

docs/INTEROP.md

@@ -118,6 +118,7 @@ only stdlib `urllib`, so there is no runtime dependency.
 | Concern | Convention |
 |---|---|
 | Primary result | **stdout** (JSON payload, default) |
+| Discovery | `extract --catalog json` (commands/flags, the suite contract) + `extract schema` / `extract fields --json` |
 | `--why`, warnings, errors | **stderr** |
 | `--why` envelope | plain-text `[why] <header>` block (as in template-vault-cli / draft-cli) |
 | Quiet | `-q` / `--silent` / `--quiet` aliases |

extract_cli.py

@@ -43,7 +43,7 @@
 from pathlib import Path
 from typing import Any, Dict, List, Optional, Tuple
 
-__version__ = "0.1.6"
+__version__ = "0.1.7"
 
 # Bumped independently of the package version when the *extraction logic*
 # changes in a way downstream consumers should notice. Embedded in `_meta`.
@@ -995,7 +995,9 @@ def _read_docx_stdlib(raw: bytes) -> str:
     paras: List[str] = []
     # iter over w:p in document order (includes paragraphs inside table cells).
     for p in root.iter(w + "p"):
-        style = _docx_paragraph_style(p.find(w + "pPr"), w)
+        ppr = p.find(w + "pPr")
+        style = _docx_paragraph_style(ppr, w)
+        numbered = ppr is not None and ppr.find(w + "numPr") is not None
         run_texts: List[str] = []
         any_text = False
         all_bold = True
@@ -1012,17 +1014,21 @@ def _read_docx_stdlib(raw: bytes) -> str:
         if not line:
             paras.append("")
             continue
-        # Word heading styles carry the clause structure (their numbers are
-        # auto-generated, so absent from text). Emit them as H2 so the clause
-        # cascade's strongest tier detects them; keep any run-in body too.
-        if _is_heading_style(style):
+        # Clause structure in real Word contracts lives in heading STYLES
+        # (Heading1-9/Title) or auto-NUMBERED paragraphs (w:numPr) -- in both the
+        # visible number is auto-generated and absent from the text. Emit such a
+        # paragraph as an H2 heading (strongest cascade tier) when its lead looks
+        # like a heading; _docx_heading_title rejects full-sentence body items
+        # (e.g. deep numbered sub-points), so this stays conservative. Keep any
+        # run-in body as a following paragraph.
+        if _is_heading_style(style) or numbered:
             title = _docx_heading_title(line)
             if title is not None:
                 paras.append(f"## {title}")
                 if len(title) < len(line):
                     paras.append(line[len(title):].lstrip(" .:\t"))
                 continue
-            # Sentence carrying a heading style -> treat as ordinary body text.
+            # Not heading-like -> treat as ordinary body text.
         if any_text and all_bold:
             line = f"**{line}**"
         paras.append(line)
@@ -1851,7 +1857,8 @@ def cmd_demo(args: argparse.Namespace) -> int:
 _SUBCOMMANDS = ("schema", "fields", "demo", "completion")
 _GLOBAL_FLAGS = (
     "--json", "--why", "-q", "--silent", "--no-color", "--llm",
-    "--format", "--fields", "--no-confidence", "-V", "--version", "-h", "--help",
+    "--format", "--fields", "--no-confidence", "--catalog",
+    "-V", "--version", "-h", "--help",
 )
 
 _BASH_COMPLETION = r"""# extract-cli bash completion
@@ -1860,7 +1867,7 @@ def cmd_demo(args: argparse.Namespace) -> int:
     local cur prev
     cur="${COMP_WORDS[COMP_CWORD]}"
     local cmds="schema fields demo completion"
-    local flags="--json --why -q --silent --no-color --llm --format --fields --no-confidence -V --version -h --help"
+    local flags="--json --why -q --silent --no-color --llm --format --fields --no-confidence --catalog -V --version -h --help"
     if [ "$COMP_CWORD" -eq 1 ]; then
         COMPREPLY=( $(compgen -W "${cmds}" -- "${cur}") $(compgen -f -- "${cur}") )
         return 0
@@ -1886,7 +1893,7 @@ def cmd_demo(args: argparse.Namespace) -> int:
     )
     flags=(
         '--json' '--why' '-q' '--silent' '--no-color' '--llm'
-        '--format' '--fields' '--no-confidence' '-V' '--version'
+        '--format' '--fields' '--no-confidence' '--catalog' '-V' '--version'
     )
     if (( CURRENT == 2 )); then
         _describe 'command' cmds
@@ -1925,6 +1932,102 @@ def _completion_handler(argv: List[str]) -> int:
     return 0
 
 
+# ---------------------------------------------------------------------------
+# Machine-readable catalog (`extract --catalog json`)
+# ---------------------------------------------------------------------------
+# The suite's shared discovery contract: agents call `extract --catalog json`
+# at startup to learn every command and flag instead of hardcoding them
+# (parallel to `nda-review-cli --catalog json`, `docx2pdf --catalog json`,
+# `sign --catalog json`). It is a STABLE contract — keep it complete and
+# accurate; `tests/test_cli.py` asserts it never drifts from the real parser.
+
+
+def _flag(name: str, *, aliases: Optional[List[str]] = None, help: str = "",
+          default: Any = None, choices: Optional[List[str]] = None,
+          required: bool = False) -> JSON:
+    return {
+        "name": name,
+        "aliases": aliases if aliases is not None else [],
+        "help": help,
+        "required": required,
+        "default": default,
+        "choices": choices,
+    }
+
+
+# Output flags shared by `extract` and `demo` (mirror _add_common_output_flags).
+_CATALOG_OUTPUT_FLAGS: Tuple[JSON, ...] = (
+    _flag("--json", help="Force JSON output to stdout (the default)."),
+    _flag("--format", default="json", choices=["json", "table"],
+          help="Output format (default: json)."),
+    _flag("--no-confidence",
+          help="Omit confidence/source markers (reduced convenience view)."),
+    _flag("--why", help="Print a rationale block to stderr."),
+    _flag("--silent", aliases=["-q", "--quiet"],
+          help="Suppress non-error diagnostics (and the human table)."),
+)
+
+
+def build_catalog() -> JSON:
+    """The machine-readable catalog emitted by `extract --catalog json`."""
+    extract_flags: List[JSON] = [
+        _flag("--llm",
+              help="Opt-in LLM enrichment of fuzzy fields (renewal mechanics, "
+                   "obligations, and a clause-map fallback). Off by default; the "
+                   "deterministic core is fully useful without it."),
+        _flag("--fields", default="",
+              help="Comma-separated subset of top-level fields to emit "
+                   "(e.g. parties,clauses,governing_law)."),
+        *_CATALOG_OUTPUT_FLAGS,
+    ]
+    return {
+        "name": CLI_NAME,
+        "bin": "extract",
+        "version": __version__,
+        "description": (
+            "Open-loop front door of the contract-ops CLI suite: ingest any contract "
+            "(.md/.txt/.html/.docx/.pdf) and emit structured JSON."
+        ),
+        "commands": [
+            {
+                "name": "extract",
+                "help": "Parse a document into structured JSON. The default action: "
+                        "`extract <path>` works without naming the subcommand. "
+                        "Positional: path to a .md/.txt/.html/.docx/.pdf file.",
+                "flags": extract_flags,
+            },
+            {
+                "name": "schema",
+                "help": "Print the output JSON Schema — the cross-CLI output contract.",
+                "flags": [],
+            },
+            {
+                "name": "fields",
+                "help": "List extractable fields and the tier that produces each.",
+                "flags": [_flag("--json", help="Emit the field list as JSON.")],
+            },
+            {
+                "name": "demo",
+                "help": "Run extraction on a bundled fixture (zero-config first run).",
+                "flags": list(_CATALOG_OUTPUT_FLAGS),
+            },
+            {
+                "name": "completion",
+                "help": "Emit a shell-completion script. Positional: bash | zsh.",
+                "flags": [],
+            },
+        ],
+        "exitCodes": {
+            "0": "success",
+            "1": "low-signal document — no high-signal fields (parties/clauses/dates) "
+                 "could be extracted; e.g. a scanned/image-only or empty file. "
+                 "A finding, not a crash.",
+            "2": "bad usage / user-actionable error (unreadable path, bad flag value, "
+                 "unsupported completion shell).",
+        },
+    }
+
+
 # ---------------------------------------------------------------------------
 # Argument parsing + main
 # ---------------------------------------------------------------------------
@@ -2025,6 +2128,23 @@ def main(argv: Optional[List[str]] = None) -> int:
     if argv and argv[0] == "__complete":
         return _completion_handler(argv[1:])
 
+    # `extract --catalog json` (or `--catalog=json`): the suite discovery
+    # contract. Intercepted before routing so it works as a bare global flag.
+    catalog_fmt: Optional[str] = None
+    for i, a in enumerate(argv):
+        if a == "--catalog":
+            catalog_fmt = argv[i + 1] if i + 1 < len(argv) else "json"
+            break
+        if a.startswith("--catalog="):
+            catalog_fmt = a.split("=", 1)[1] or "json"
+            break
+    if catalog_fmt is not None:
+        if catalog_fmt != "json":
+            _eprint(_red("error:") + f" unknown --catalog format {catalog_fmt!r}; supported: json")
+            return 2
+        print(json.dumps(build_catalog(), indent=2, ensure_ascii=True))
+        return 0
+
     if not argv:
         build_parser().print_help()
         return 0