Skip to content

feat: show active embedding provider in memory stats#1623

Open
marioja wants to merge 2 commits intoruvnet:mainfrom
marioja:DOCUMENT-EMBEDDING-CHOICE
Open

feat: show active embedding provider in memory stats#1623
marioja wants to merge 2 commits intoruvnet:mainfrom
marioja:DOCUMENT-EMBEDDING-CHOICE

Conversation

@marioja
Copy link
Copy Markdown

@marioja marioja commented Apr 17, 2026

Replaces #1621 (closed due to branch rename)

Summary

  • ruflo memory stats now shows which embedding provider is active, its vector dimensions, and whether the HNSW index is available
  • Adds an "Embedding" section to both table and --format json output
  • Two new tests covering the embedding info output

Justification

There is currently no way to know which embedding provider ruflo selected for a project without re-running ruflo memory init --verbose --force (which reinitializes the database). The embedding provider directly affects:

  • Search quality: semantic (384/768-dim ONNX) vs keyword-only (128-dim hash fallback)
  • Search speed: HNSW-indexed (~1ms) vs linear scan
  • Vector compatibility: entries stored with different providers have different dimensions and can't be compared

When memory stats shows Provider: hash-fallback, that immediately tells the user their semantic search is degraded and they need to install an ONNX provider. Without this info, memory search silently returns poor results with no indication why.

Possible provider values

Provider Package Dimensions Quality
Xenova/all-MiniLM-L6-v2 @xenova/transformers 384 Full semantic
agentic-flow/reasoningbank agentic-flow 768 Full semantic
ruvector/onnx ruvector 384 Full semantic
agentic-flow agentic-flow 768 Full semantic
hash-fallback (built-in) 128 No semantic understanding
none 0 memory-initializer unavailable

Example output

Embedding
+------------+---------------+
| Metric     |         Value |
+------------+---------------+
| Provider   | hash-fallback |
| Dimensions |           128 |
| HNSW Index |        active |
+------------+---------------+

Changes

File Change
v3/@claude-flow/cli/src/commands/memory.ts Probe loadEmbeddingModel() and getHNSWIndex() in stats command, display Embedding section
v3/@claude-flow/cli/__tests__/commands.test.ts Two new tests: embedding info in JSON output, graceful defaults when unavailable
.gitignore **/node_modules/ to catch nested workspace dirs
v3/package.json Bump packageManager to pnpm@10.33.0

Test plan

  • npx vitest run -t "memory stats" — both new tests pass
  • node v3/@claude-flow/cli/bin/cli.js memory stats — Embedding section renders correctly
  • --format json includes embedding.provider, embedding.dimensions, embedding.hnswAvailable
  • Graceful fallback to provider: "none" when memory-initializer is unavailable

🤖 Generated with claude-flow

`ruflo memory stats` now displays which embedding provider is active,
its vector dimensions, and whether the HNSW fast-search index is
available. This makes it easy to diagnose which fallback the system
selected without re-running `memory init --verbose`.

The embedding provider is resolved via a priority fallback chain in
`memory-initializer.ts:loadEmbeddingModel()` (line 1498). Possible
values for the Provider field:

  1. "cached"                   — model already loaded in this process
  2. "Xenova/all-MiniLM-L6-v2"  — @xenova/transformers ONNX (384-dim)
  3. "agentic-flow/reasoningbank" — agentic-flow ReasoningBank (768-dim)
  4. "ruvector/onnx"             — ruvector bundled MiniLM (384-dim)
  5. "agentic-flow"              — agentic-flow core embeddings (768-dim)
  6. "hash-fallback"             — no AI, deterministic hash (128-dim)
  7. "none"                      — memory-initializer unavailable

The new Embedding section appears in both table and --format json output.

Other changes:
- .gitignore: use **/node_modules/ to catch nested workspace dirs
- v3/package.json: bump packageManager to pnpm@10.33.0
- Two new tests for embedding info in stats output

Co-Authored-By: claude-flow <ruv@ruv.net>
Adds an optional Line 5 to the Claude Code statusline showing the active
embedding provider, vector dimensions, and HNSW indexing status. The line
is hidden until .swarm/embedding-provider.json is present, so existing
users see no change until memory is exercised.

Changes:
- memory-initializer.ts: add _writeEmbeddingProviderFile() helper called at
  every non-cache return site in loadEmbeddingModel(). Writes provider,
  dimensions, and preserved hnswAvailable on first model resolution, covering
  all callers (session-start hooks, memory init/store/search/stats).
- memory.ts (statsCommand): remove the misplaced file-write block; stats
  continues to display embedding info from the loadEmbeddingModel() return
  value. Removes debug console.log.
- statusline.cjs + statusline-generator.ts: add getEmbeddingProviderStats()
  reading .swarm/embedding-provider.json, and a conditional Line 5 rendered
  as: 🧬 Embeddings  Provider ●<name>  │  Dims <n>-dim  │  HNSW ⚡active|inactive
  Also exposes embedding field in --json output.

Co-Authored-By: claude-flow <ruv@ruv.net>
@marioja
Copy link
Copy Markdown
Author

marioja commented Apr 19, 2026

Here is what it looks like with the pull request. Look at the last green Embeddings line:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant