Skip to content

Phase 3: embedding-based intent resolver (opt-in, backward-compatible) - #7

Merged
shreyanshjain7174 merged 1 commit into
mainfrom
feat/embedding-resolver
May 2, 2026
Merged

shreyanshjain7174 merged 1 commit into
mainfrom
feat/embedding-resolver

Conversation

@shreyanshjain7174

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in embedding-based intent resolver (`--resolver embedding`) that generalizes beyond literal keyword matches. Pure Go, dependency-free, deterministic, offline-capable. Drop-in replacement behind the existing `Resolver` interface.

Why

The keyword resolver works perfectly for the S1-S5 benchmark scenarios (because those intents use literal keywords like 'query', 'email', 'slack'). But production agent intents are paraphrased. The embedding resolver covers the variation without sending intent text to an external API.

Side-by-side smoke

Live server, four paraphrased intents:

Intent keyword embedding
query the customer database 1 action 2 actions
pull last week revenue numbers ERR no caps 1 action
drop a message in the channel 1 action 2 actions
deliver the report by email 2 actions 2 actions

How it works

  1. Each capability has 4-6 training phrases (`DefaultExamples()`).
  2. Hash each phrase token into 256 buckets, weight by IDF, average per-capability, L2-normalize -> centroid.
  3. At query time vectorize the intent the same way; pick capabilities with cosine >= threshold (default 0.20).
  4. Falls back to the keyword resolver when no centroid passes the threshold so adversarial intents still resolve.

Backward compatibility

`--resolver keyword` is the default. The S1-S5 benchmark numbers are unaffected. Embedding is opt-in.

Validation

  • `go vet ./...`: clean.
  • `go test -race ./...`: 9 packages green.
  • internal/resolver coverage: 97.4% (was 100% before; embedding adds 2 small math helpers at 87.5/83.3%).
  • staticcheck + govulncheck: clean.
  • pytest --cov-fail-under=85: 65 passed, 96.91% coverage.
  • 9 new resolver test cases (known phrases, paraphrase generalization, determinism over 50 iterations, hint passthrough, threshold fallback, error surfacing, empty input, defaults, malformed-example skipping).

Phase 3 of the 'all in order' rollout. Adds an opt-in embedding-based
intent resolver that generalizes beyond literal keyword matches while
staying deterministic, dependency-free, and offline.

internal/resolver/embedding.go (BSL 1.1):

- EmbeddingResolver implements the existing Resolver interface
  (Resolve(intent, hints) -> caps). Drop-in replacement.
- Hash-based TF-IDF embedding (default 256 buckets).
- Capability centroids are L2-normalized averages of training-phrase
  vectors. Cosine similarity at or above Threshold (default 0.20)
  selects capabilities.
- DefaultExamples() seeds five capabilities (sql, audit, template,
  email, messaging) with 4-6 paraphrased phrases each. 24 examples
  total.
- Falls back to a configurable Resolver (default keyword) when no
  centroid passes the threshold so adversarial intents still resolve
  via substring matching.
- Pure Go, no model downloads, no Python, no GPU. Same vector for
  the same input on every machine.
- Goroutine-safe after construction.

internal/resolver/embedding_test.go (BSL 1.1):

- 9 test cases covering known-phrase recognition, paraphrase
  generalization (4 intents the keyword resolver misses or
  under-resolves), determinism (50 iterations identical),
  hint passthrough, threshold-driven fallback, no-fallback error
  surfacing, empty-input handling, default option values, and
  malformed-example skipping.

cmd/acp-server/main.go:

- New --resolver flag: 'keyword' (default, backward-compatible) or
  'embedding'. Unknown values log a warning and fall back to keyword
  so a typo never takes the server offline.
- pickResolver() helper chooses the resolver and is logged in the
  startup info line.

Side-by-side smoke (live server, four paraphrased intents):

  intent                              keyword       embedding
  query the customer database         1 action      2 actions
  pull last week revenue numbers      ERR no caps   1 action
  drop a message in the channel       1 action      2 actions
  deliver the report by email         2 actions     2 actions

Embedding catches all four; keyword fails one outright and
under-resolves three. The S1-S5 benchmark numbers are unaffected
because those intents already use literal keyword matches; the
embedding resolver is for production deployments where intent
vocabulary is more varied than the benchmark scenarios.

Validation:

- go vet ./...: clean.
- go test -race -count=1 -timeout 60s ./...: 9 packages green.
- internal/resolver coverage: 97.4%.
- staticcheck, govulncheck: clean.
- pytest --cov-fail-under=85: 65 passed, 96.91% coverage.
- docs check: ok.

Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com>
@shreyanshjain7174
shreyanshjain7174 merged commit 93a284f into main May 2, 2026
6 checks passed
@shreyanshjain7174
shreyanshjain7174 deleted the feat/embedding-resolver branch May 2, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant