feat(ci): token-limits gate (tiktoken, no key) + byte/token partition

[JacobPEvans-personal]

Summary

Finishes the long-started transition from a raw byte file-size gate to a token-based limit for AI-read docs — using a no-auth, offline tokenizer (tiktoken) instead of atc (which needs ANTHROPIC_API_KEY). Both gates now coexist with mutually-exclusive coverage: token budgets for Markdown docs, byte limits for everything else.

Changes

• _token-limits.yml (new, reusable): counts tokens with public/offline tiktoken (pip install tiktoken, no secret), per-repo config in .token-limits.yaml; sparse-checkouts the shared counter (same pattern as the watchdog).
• scripts/check-token-limits.py (new): fnmatch budgets (most-restrictive match wins), exclude globs, skips non-token-gated + binary files; exit 1 on violation. A file is token-gated iff it matches a limits pattern.
• _file-size.yml: when a .token-limits.yaml is present, drops .md from its scan so Markdown docs are governed only by the token gate — no file is double-gated. Repos without .token-limits.yaml are unaffected.
• _ci-gate.yml: new token_limits input + Token Limits job wired into the Merge Gate needs/allowed-skips.

Config shape

# .token-limits.yaml
defaults: { max_tokens: 2000 }
exclude: ['TERRAFORM.md']      # machine-generated reference: neither gate
limits:
  AGENTS.md: 2000
  '*README.md': 1500
  '*.md': 2000                  # catch-all so all .md are budgeted

Test plan

• actionlint clean on all three workflows (shellcheck included — no SC2053/SC2254 suppressions; the partition uses an extension drop, not dynamic globs).
• Counter verified locally with real tiktoken (o200k_base): correct per-pattern budgets, exclude honored, non-.md skipped, exit 1 on over-budget.
• No secrets required anywhere.

First consumer: dryvist/tofu-unifi (follow-up PR) flips file_size-only → file_size + token_limits.

[gemini-code-assist]

Code Review

This pull request introduces a new offline token-limit checker script (scripts/check-token-limits.py) that uses tiktoken to enforce token budgets defined in .token-limits.yaml. Feedback was provided to make the YAML parsing logic more robust by explicitly validating the types of the parsed configuration elements, preventing potential CI crashes if the configuration file is malformed.