Skip to content

Outcome scorer: CJK implementation-summary pattern coverage gap #25

@tznthou

Description

@tznthou

Context

Counterpart to #23 (English corpus gap). Discovered during 2026-05-06 dogfood corpus audit on 39 v=2 committed/tested sessions.

Finding

After PR fixing extractOutcome to skip save-t process reports (preventing it from capturing process meta instead of real outcomes), audit shows real implementation outcomes now reach the scorer — but stay sub-threshold (0/39 hit) due to coverage gaps in 5 categories (decision-language / impl-facts / constraints / cause-effect / validation).

Common CJK implementation-summary patterns NOT covered

Sample observed in audit (post-fix, real outcomes captured but score < 2):

Pattern Sample Current score
Chinese commit confirmation Commit \6de0666` 落地` 0
Bug fix commit notation Bug fix commit = \768d95e`` 1 (decision-language only)
修復總結 / 完工總結 table ## 修復總結 | 變動 | 檔案 | 1 (偶中)
Phase milestone ## Phase 1 完成 ✅ | Step | Commit | 0

Why this matters

  • 0 hit / 39 ground-truth committed-tested sessions = hook auto-harvest path effectively dead for CJK dogfood users
  • Counterpart to EN corpus gap (English-language scorer pattern coverage (post-#18) #23) — same root issue (regex-based scorer can't keep up with semantic richness), different language coverage

Acceptance criteria

  • 3+ corroborating reports converging on the same category gap before extending anchor patterns (per English-language scorer pattern coverage (post-#18) #23 protocol — avoid dataset bias by adding patterns prematurely)
  • Patterns extended must not over-broaden (false positive on noise) — verify against existing audit corpus
  • Re-run audit after extension to confirm hit rate improvement

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions