What happened
In PR #1781, the author introduced a new public helper function lacks_effective_anchor(pattern) in policy/lib/rule_data/rule_data.rego. The function was tested indirectly through anchoring_errors tests — callers of the helper were tested, but the helper itself had no direct unit tests exercising its edge cases.
The review agent ran 7 successful reviews across the PR's lifetime and never flagged the missing direct tests. Human reviewer st3penta requested direct unit tests for lacks_effective_anchor on 2026-08-10. The author added tests covering unanchored patterns, ^.* prefixes, ^.+ prefixes, properly anchored patterns, and non-string values.
St3penta also identified that the pattern != "" branch in sbom.rego for empty/absent URLs had no test coverage — another gap that indirect testing did not reveal.
What could go better
The existing AGENTS.md states that 100% test coverage is enforced, and the PR technically met that bar through indirect coverage. But the human reviewer's request for direct unit tests reflects a stronger convention: new helper functions that encapsulate non-trivial logic should have their own focused unit tests that exercise edge cases independently of their callers.
This is a medium-confidence finding. The convention clearly exists in practice (the human reviewer explicitly asked for it, and the codebase generally follows it — most helpers in policy/lib/ have direct test suites). But it is not documented in AGENTS.md, so the review agent had no basis to flag it.
The risk of not documenting this is that future PRs introduce helper functions with only indirect coverage, masking edge-case bugs that would be caught by direct unit tests. The lacks_effective_anchor function had 4 distinct logical branches (no anchor, ^.* prefix, ^.+ prefix, valid anchor) — indirect tests through anchoring_errors might not exercise all combinations.
Proposed change
Add guidance to the testing section of AGENTS.md distinguishing coverage percentage from test quality for helper functions. Specifically:
## Testing new helper functions
When a PR introduces a new public helper function in `policy/lib/`, it must have
direct unit tests in the corresponding `_test.rego` file — not just indirect
coverage through consuming rules. Direct tests should exercise:
- Each logical branch of the function
- Edge cases (empty strings, non-string values, boundary conditions)
- The function's contract independent of any specific caller
100% coverage through indirect tests alone is not sufficient for functions with
multiple logical branches.
Also update .claude/rules/policy-rules.md to include: "New public helper functions in policy/lib/ require direct unit tests, not just indirect coverage through callers."
Validation criteria
On the next 3 PRs in conforma/policy that introduce new public helper functions in policy/lib/, the review agent should flag any that lack direct unit tests. Verify by checking whether the agent's review comments mention direct testing when a new helper is added with only indirect coverage.
Generated by retro agent from #1781
What happened
In PR #1781, the author introduced a new public helper function
lacks_effective_anchor(pattern)inpolicy/lib/rule_data/rule_data.rego. The function was tested indirectly throughanchoring_errorstests — callers of the helper were tested, but the helper itself had no direct unit tests exercising its edge cases.The review agent ran 7 successful reviews across the PR's lifetime and never flagged the missing direct tests. Human reviewer st3penta requested direct unit tests for
lacks_effective_anchoron 2026-08-10. The author added tests covering unanchored patterns,^.*prefixes,^.+prefixes, properly anchored patterns, and non-string values.St3penta also identified that the
pattern != ""branch insbom.regofor empty/absent URLs had no test coverage — another gap that indirect testing did not reveal.What could go better
The existing AGENTS.md states that 100% test coverage is enforced, and the PR technically met that bar through indirect coverage. But the human reviewer's request for direct unit tests reflects a stronger convention: new helper functions that encapsulate non-trivial logic should have their own focused unit tests that exercise edge cases independently of their callers.
This is a medium-confidence finding. The convention clearly exists in practice (the human reviewer explicitly asked for it, and the codebase generally follows it — most helpers in
policy/lib/have direct test suites). But it is not documented in AGENTS.md, so the review agent had no basis to flag it.The risk of not documenting this is that future PRs introduce helper functions with only indirect coverage, masking edge-case bugs that would be caught by direct unit tests. The
lacks_effective_anchorfunction had 4 distinct logical branches (no anchor,^.*prefix,^.+prefix, valid anchor) — indirect tests throughanchoring_errorsmight not exercise all combinations.Proposed change
Add guidance to the testing section of
AGENTS.mddistinguishing coverage percentage from test quality for helper functions. Specifically:Also update
.claude/rules/policy-rules.mdto include: "New public helper functions inpolicy/lib/require direct unit tests, not just indirect coverage through callers."Validation criteria
On the next 3 PRs in conforma/policy that introduce new public helper functions in
policy/lib/, the review agent should flag any that lack direct unit tests. Verify by checking whether the agent's review comments mention direct testing when a new helper is added with only indirect coverage.Generated by retro agent from #1781