Finding
scripts/ contains 10 Python entry points. Eight have a matching tests/unit/test_*.py. Two did not:
| script |
lines |
CI gate |
scripts/generate_skill_index.py |
165 |
.github/workflows/docs-validate.yml runs it with --check |
scripts/update_coverage_snapshot.py |
186 |
.github/workflows/pr-validate.yml runs it with --check |
Both are merge gates. Their failure modes are quiet:
generate_skill_index.collect() enforces nine front-matter validations (required fields, category/status enums, entry_point self-match, id == name). A regression that drops a check lets a malformed skill into index.json with a green build.
generate_skill_index.main(--check) compares index.md against render_md(skills, current['generated_at']) — deliberately reusing the committed date rather than date.today(). Nothing pinned that behaviour; switching it to today's date would make the docs gate fail every day after the catalog is committed (the same class of failure tracked in dakota-iso#137).
update_coverage_snapshot.parse_scenarios() implements non-obvious Gherkin tag semantics: feature tags inherit, scenario tags reset after each scenario, and Rule:/Background:/Examples: discard pending tags. classify() layers a precedence order on top (quarantine > hardware_blocked/future/pending > active). A silent miscount here corrupts the suite-map coverage table that this repo uses as its coverage source of truth — and the whole point of that script is to stop humans hand-editing those numbers.
update_file() returns three distinct exit codes (0 ok, 1 stale, 2 markers missing) that CI branches on; none were pinned.
Recommendation
Add unit tests for both scripts. Done in PR #766 (hold-gated):
tests/unit/test_generate_skill_index.py — 36 tests
tests/unit/test_update_coverage_snapshot.py — 49 tests
85 tests, all passing. No production code changed.
Priority
- Impact: high (both scripts are required merge gates with silent failure modes)
- Effort: low (both are pure functions plus a
--repo-root seam; no mocking of external services needed)
Filed by quality agent (hold-gated mode)
🐝 Hive Agent: quality | Instance: hosted-projectbluefin-knuckle-gjvq | SHA: f368ee5
— hive: agent=quality backend=copilot model=claude-opus-5
Finding
scripts/contains 10 Python entry points. Eight have a matchingtests/unit/test_*.py. Two did not:scripts/generate_skill_index.py.github/workflows/docs-validate.ymlruns it with--checkscripts/update_coverage_snapshot.py.github/workflows/pr-validate.ymlruns it with--checkBoth are merge gates. Their failure modes are quiet:
generate_skill_index.collect()enforces nine front-matter validations (required fields, category/status enums,entry_pointself-match,id == name). A regression that drops a check lets a malformed skill intoindex.jsonwith a green build.generate_skill_index.main(--check)comparesindex.mdagainstrender_md(skills, current['generated_at'])— deliberately reusing the committed date rather thandate.today(). Nothing pinned that behaviour; switching it to today's date would make the docs gate fail every day after the catalog is committed (the same class of failure tracked in dakota-iso#137).update_coverage_snapshot.parse_scenarios()implements non-obvious Gherkin tag semantics: feature tags inherit, scenario tags reset after each scenario, andRule:/Background:/Examples:discard pending tags.classify()layers a precedence order on top (quarantine>hardware_blocked/future/pending> active). A silent miscount here corrupts the suite-map coverage table that this repo uses as its coverage source of truth — and the whole point of that script is to stop humans hand-editing those numbers.update_file()returns three distinct exit codes (0 ok, 1 stale, 2 markers missing) that CI branches on; none were pinned.Recommendation
Add unit tests for both scripts. Done in PR #766 (hold-gated):
tests/unit/test_generate_skill_index.py— 36 teststests/unit/test_update_coverage_snapshot.py— 49 tests85 tests, all passing. No production code changed.
Priority
--repo-rootseam; no mocking of external services needed)Filed by quality agent (hold-gated mode)
🐝 Hive Agent:
quality| Instance:hosted-projectbluefin-knuckle-gjvq| SHA:f368ee5— hive: agent=quality backend=copilot model=claude-opus-5