Memory Performance

⚡ Performance & SIMD

Spector Memory is engineered for microsecond-scale latency. This page documents the benchmark results and the key performance techniques that make it possible.

---

Benchmark Summary

Measured on Intel Core Ultra 9 285K, Java 25, AVX2 256-bit (8 float lanes), ZGC:

Benchmark	Result	Notes
SIMD L2 Distance (128-dim)	0.8 µs/vector	1.2M vectors/sec
SIMD L2 Distance (384-dim)	1.5 µs/vector	2.6M vectors/sec
SIMD L2 Distance (768-dim)	2.2 µs/vector	1.4M vectors/sec
SIMD L2 Distance (1024-dim)	3.0 µs/vector	1.0M vectors/sec
Reverse Index Lookup	180 ns/lookup	O(1) packed-key map
Cognitive Scorer (10K × 128-dim)	2.9 ms total	Full 6-phase pipeline
Batch Habituation (1K IDs)	101 µs total	100 ns per penalty computation
Tier Count Query	17 ms / 100K calls	170 ns per call
Full Pipeline (1K ingest + 100 recall)	< 50 ms/query	End-to-end latency
Real Embedding (qwen3-embedding 4096-dim)	31 ms/embed	Via Ollama (network bound)

---

Key Techniques

O(1) Reverse Index

Memory IDs are resolved in constant time using a packed-key map. The key packs (type, offset) into a single 64-bit long — zero string concatenation, zero hashing overhead.

This yields 180 ns lookups at 50K entries.

---

SIMD Euclidean Distance

Quantized INT8 Euclidean distance uses the Java Vector API for hardware acceleration:

flowchart LR
    READ["Read INT8 bytes<br/>from MemorySegment"] --> CAST["Cast INT8 → float32<br/><i>vectorized</i>"]
    CAST --> DEQUANT["Affine dequantize<br/><i>float = byte × scale + min</i>"]
    DEQUANT --> L2["Fused multiply-add<br/><i>accumulate squared diff</i>"]
    L2 --> RESULT["L2 distance<br/><b>2.2 µs/768-dim</b>"]

    style READ fill:#4a6fa5,color:white
    style L2 fill:#0984e3,color:white
    style RESULT fill:#00b894,color:white

Loading

The entire computation runs in SIMD registers — no intermediate Java objects are created.

Throughput: 2.2 µs/vector at 768 dimensions (1.4M vectors/sec on AVX2).

---

Batch Habituation

The habituation penalty module computes all penalties in a single batch call with amortized map access, processing 1K penalties in 101 µs total.

---

Inline Header Capture

Scored records capture the cognitive header inline during scoring, eliminating N×8 off-heap re-reads per recall query.

---

Direct Tier Access

Tier count queries use direct field access to typed store references rather than iteration, completing 100K calls in 17 ms (170 ns/call).

---

Parallel Tier Scanning

Each memory tier is scanned on a dedicated Virtual Thread:

gantt
    title Parallel Recall: 5 concurrent scans
    dateFormat X
    axisFormat %L ms
    
    section Working (100 records)
    Scan     :a1, 0, 1
    section Episodic P1 (5K records)
    Scan     :a2, 0, 3
    section Episodic P2 (3K records)
    Scan     :a3, 0, 2
    section Semantic (200 headers)
    Scan     :a4, 0, 1
    section Procedural (50 records)
    Scan     :a5, 0, 1
    section Merge + Rank
    Top-K    :a6, 3, 4

Loading

Key insight: Episodic partitions use disjoint memory segments — each partition's mmap is a separate off-heap buffer. This guarantees zero contention between virtual threads, enabling perfect parallel scaling.

Fallback: If parallel scanning fails (e.g., thread pool exhaustion), the pipeline falls back to sequential scanning with identical results.

---

Memory Footprint

Component	Formula	10K memories (768-dim)
Episodic partition	64B header + N × (64B + vecBytes)	64B + 10K × 832B = 8.1 MB
Working memory	capacity × (64B + vecBytes)	100 × 832B = 81 KB
Semantic headers	capacity × 64B	5K × 64B = 312 KB
Procedural store	capacity × (64B + vecBytes)	500 × 832B = 406 KB
Forward index	~120B per entry	10K × 120B = 1.2 MB
Reverse index	~60B per entry	10K × 60B = 600 KB
Total		~10.7 MB

tip: vs. Python Memory Layers A Python memory system stores each memory as a Python object (~500-800 bytes overhead) plus the vector in NumPy (~3KB for 768-dim float32). Spector stores the same memory in 832 bytes (64B header + 768B INT8 vector) — a 4-8× reduction.

---

Test Suite

spector-core:   276 tests ✅   (includes 15 SIMD kernel verification tests)
spector-memory: 167 tests ✅   (includes performance benchmarks + index tests)
                + 10 Ollama real embedding E2E tests (gated by OLLAMA_LIVE=true)
Total: 443 tests, 0 failures

Running Benchmarks

# Run all memory tests (includes benchmark assertions)
mvn test -pl spector-memory

# Run only performance benchmarks
mvn test -pl spector-memory -Dtest=PerformanceBenchmarkTest

# Run Ollama real embedding E2E tests
OLLAMA_LIVE=true mvn test -pl spector-memory -Dtest=OllamaRealEmbeddingTest

---

Next Steps

• :material-memory: [[Off-Heap Panama Design|Memory--Panama-Design]] — zero-GC architecture
• :material-lightning-bolt: [[6-Phase Scoring Pipeline|Memory--Scoring-Pipeline]] — the SIMD hot-loop
• :material-brain: [[Architecture|Memory--Architecture]] — system-level design