Skip to content

Commit 3f4fcff

Browse files
committed
feat(docs,scripts,bolt): consumer skills, migration scripts, unified commit-failure wrapper
Three things land in one pass. 1. Bolt wire contract — single commit-failure shape on both commit paths. The implicit-autocommit path at pkg/cypher/executor.go:1978 used to wrap commit failures with "failed to commit implicit transaction: ..." while the explicit BEGIN/COMMIT path at pkg/cypher/transaction.go:181 used "commit failed: ...". Downstream Bolt classifiers that match on the substring saw two shapes for the same race. Aligned both to "commit failed: ..." and updated three downstream test sites that synthesized the legacy wrapper. Pinned with five new regression tests covering the consumer-visible error contract (optimistic conflict shapes, commit-time UNIQUE wrapping, MERGE idempotency under concurrent writers, namespaced-engine constraint refresh, parsed-default values) and one-line `// Wire contract:` annotations at every source-of-truth line. See docs/plans/consumer-pinned-error-contract-plan.md. 2. docs/skills/ — agent-ready skill files for every consumer-facing surface: hot-path Cypher cookbook, Bolt client, gRPC (Qdrant + NornicSearch), Qdrant migration, Neo4j migration, knowledge policies, decay tuning, promotion policies, managed embeddings, vector & full- text search, RAG procedures. Knowledge-policy user guides rewritten with disambiguated bundle-vs-binding language. Documentation site consolidated from 14 top-level groups to 8 in mkdocs.yml; every published page is in nav: so search-clicks always have sidebar context, and canonical URLs emit on every built page. 3. scripts/migration/ — runnable Neo4j → NornicDB and Qdrant → NornicDB migrations in Python, Go, and Node. The Neo4j scripts run schema-first (constraints + indexes including fulltext and vector), then nodes via UnwindSimpleMergeBatch, then edges via UnwindMultiMatchCreateBatch, keyset-paginated by elementId. The Qdrant scripts use the gRPC compat surface end-to-end (REST in Node), replicate collection vector configs, scroll and upsert points in batches, and verify counts. All Go scripts vet and build clean.
1 parent 6ec42e9 commit 3f4fcff

34 files changed

Lines changed: 3880 additions & 91 deletions

CHANGELOG.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Latest Changes]
99

10-
- See `docs/latest-untagged.md` for the untagged `latest` image changelog.
10+
- See `docs/latest-untagged.md` for the full untagged changelog with rationale and file cites.
11+
12+
### Fixed
13+
14+
- **Bolt commit-failure wrapper unified across both commit paths.** The implicit-autocommit path now produces `"commit failed: ..."` (matching the explicit `BEGIN/COMMIT` path) instead of the legacy `"failed to commit implicit transaction: ..."`. Downstream Bolt classifiers that match on the substring no longer need to handle two shapes for the same race.
15+
16+
### Internal
17+
18+
- **Consumer-pinned error contract pinned with regression tests.** Five new tests cover the optimistic-conflict message shapes, commit-time UNIQUE wrapping, MERGE idempotency under concurrent writers, namespaced-engine constraint refresh, and parsed-default values. See `docs/plans/consumer-pinned-error-contract-plan.md`.
19+
20+
### Documentation
21+
22+
- **`docs/skills/` — agent-ready skill files** for every consumer-facing surface: hot-path Cypher cookbook, Bolt client, gRPC (Qdrant + NornicSearch), Qdrant migration, Neo4j migration, knowledge policies, decay tuning, promotion policies, managed embeddings, vector & full-text search, RAG procedures.
23+
- **Knowledge-policy user guides rewritten** with disambiguated bundle-vs-binding language: bundles are inert parameter packages; bindings carry the `FOR` target; decay is pure time math evaluated on read; `ON ACCESS` belongs to promotion only; `LAST_ACCESSED` decay only reads access metadata.
24+
- **Documentation site consolidated to 8 top-level groups** (was 14). Every published `.md` is now in `nav:`, fixing the bug where clicking a search result loaded the content but left the sidebar empty. Canonical URLs emit on every page; URL ↔ sidebar ↔ content stay synchronized when navigating from a search result.
25+
26+
### Tools & Scripts
27+
28+
- **`scripts/migration/neo4j/`** — runnable Neo4j → NornicDB migrations in Python, Go, and Node. Schema-first (constraints + indexes including fulltext and vector), then nodes via `UnwindSimpleMergeBatch`, then edges via `UnwindMultiMatchCreateBatch`. Keyset-paginated, idempotent.
29+
- **`scripts/migration/qdrant/`** — runnable Qdrant → NornicDB migrations in Python, Go, and Node, using the Qdrant-compatible gRPC surface end-to-end (REST in the Node script). Replicates collection vector configs, scrolls and upserts points in batches, verifies counts.
1130

1231
## [v1.1.0] - 2026-05-14
1332

README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,8 @@ with driver.session() as session:
208208
- **Drop-in compatibility** via Bolt + Cypher for existing applications.
209209
- **Canonical graph ledger model** for temporal validity, tritemporal fact modeling, as-of reads, and audit-oriented mutation tracking.
210210

211+
> 🚚 **Migrating?** [Neo4j → NornicDB skill](docs/skills/neo4j-migration.skill.md) · runnable Python/Go/Node scripts in [`scripts/migration/neo4j/`](scripts/migration/neo4j/).
212+
211213
## Why Switch from Qdrant?
212214

213215
- **Graph + vector in one engine**: combine semantic retrieval with native graph traversal and Cypher queries.
@@ -216,6 +218,8 @@ with driver.session() as session:
216218
- **Canonical truth modeling**: versioned facts, temporal validity windows, tritemporal facts, and as-of reads for governance-heavy use cases.
217219
- **Protocol flexibility**: use REST, GraphQL, Bolt/Cypher, Qdrant-compatible gRPC, and additive Nornic gRPC on one platform.
218220

221+
> 🚚 **Migrating?** [Qdrant → NornicDB skill](docs/skills/qdrant-migration.skill.md) · [gRPC surface skill](docs/skills/grpc.skill.md) · runnable Python/Go/Node scripts in [`scripts/migration/qdrant/`](scripts/migration/qdrant/).
222+
219223
## Features
220224

221225
### Retention Policies
@@ -233,6 +237,8 @@ Designed to work with existing Neo4j drivers and Bolt/Cypher workflows, with min
233237
- **Schema Management** — Constraints, indexes, vector indexes
234238
- **Qdrant gRPC API Compatible** — Works with Qdrant-style gRPC vector workflows
235239

240+
> 🤖 **Agent skill:** [Bolt Client](docs/skills/bolt-client.skill.md) — connection defaults, retry classification, MERGE under concurrent writers, batch sizing.
241+
236242
### 🧠 Knowledge-Layer Scoring
237243

238244
Profile-driven decay and promotion scoring with the [Ebbinghaus-Roynard four-layer decomposition](https://arxiv.org/pdf/2604.11364). The engine does not hardcode cognitive tiers. Operators model their own labels and lifecycle rules using Cypher DDL.
@@ -486,6 +492,7 @@ Start with the docs hub for role/task navigation, then use the issue index for s
486492
| [User Guides](docs/user-guides/README.md) | Complete examples & patterns |
487493
| [Performance](docs/performance/README.md) | Benchmarks vs Neo4j |
488494
| [Neo4j Migration](docs/neo4j-migration/README.md) | Compatibility & feature parity |
495+
| [Migration Scripts](scripts/migration/README.md) | Runnable Neo4j and Qdrant → NornicDB migrations (Python, Go, Node) |
489496
| [Architecture](docs/architecture/README.md) | System design & internals |
490497
| [Docker Guide](docker/README.md) | Build & deployment |
491498
| [Development](docs/development/README.md) | Contributing & development |

docs/latest-untagged.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,60 @@
11
# Latest (Untagged) Changelog
2+
3+
Changes that have landed on `main` since the last tagged release. Promoted into a versioned section of `CHANGELOG.md` when the next tag cuts.
4+
5+
## Bolt wire contract — single commit-failure wrapper across both commit paths
6+
7+
The implicit-autocommit path at `pkg/cypher/executor.go:1978` historically wrapped commit failures with `"failed to commit implicit transaction: ..."`, while the explicit `BEGIN/COMMIT` path at `pkg/cypher/transaction.go:181` used `"commit failed: ..."`. Downstream Bolt consumers classify retryable transients on the substring; the two-shape wire surface meant a classifier that retried one path silently surfaced the other as a permanent failure.
8+
9+
Aligned both paths to the `"commit failed: ..."` shape. Updated three downstream test sites (`pkg/bolt/server_test.go`, `pkg/server/server_db_error_mapping_test.go`, `testing/e2e/vector_traversal_shapes_bench_test.go`) plus a doc comment in `pkg/storage/badger_mvcc_clock_skew_test.go`.
10+
11+
### Pinned with regression tests
12+
13+
Added five new contract tests so the wire shape can't drift again:
14+
15+
- `pkg/storage/constraint_validation_namespaced_test.go``RefreshUniqueConstraintValuesForEngine` keeps storage-prefixed IDs under `NamespacedEngine`. Pins the fix at `pkg/storage/constraint_validation.go:71` against a known false-UNIQUE failure mode.
16+
- `pkg/storage/conflict_message_contract_test.go` — five canonical optimistic-conflict messages (node, node-with-version-detail, edge, edge-alternate, adjacent-edge) plus `errors.Is(err, ErrConflict)` chain preservation.
17+
- `pkg/cypher/commit_failure_message_contract_test.go` — node and relationship variants of the `commit failed: constraint violation: … already exists` shape.
18+
- `pkg/cypher/merge_idempotency_under_concurrent_test.go` — two writers MERGE on the same uid; the loser sees the pinned wire shape and a retry succeeds with the loser's `SET` visible.
19+
- `pkg/config/defaults_consumer_contract_test.go``LoadDefaults()` returns the documented values for `Database.AsyncWritesEnabled=true`, `Database.PersistSearchIndexes=false`, `Auth.Enabled=false`, `Server.BoltPort=7687`, `Server.HTTPPort=7474`.
20+
21+
Each pinned source-of-truth line gained a one-line `// Wire contract:` comment pointing back to the plan.
22+
23+
## Documentation — agent-ready skills
24+
25+
New `docs/skills/` directory containing self-contained, agent-ready skill files for every consumer-facing surface:
26+
27+
- `cypher-queries.skill.md` — hot-path Cypher cookbook (point lookup, batch retrieval, pagination, search, traversal, batched UNWIND/MERGE writes, cleanup, multi-tenant isolation).
28+
- `bolt-client.skill.md` — Bolt connection defaults, retry classification, MERGE under concurrent writers, batch sizing.
29+
- `grpc.skill.md` — Qdrant-compatible gRPC + `NornicSearch.SearchText`. RPC catalog, collection→database mapping, point→node mapping, limits, auth.
30+
- `qdrant-migration.skill.md` — Qdrant → NornicDB end-to-end through the gRPC compat layer.
31+
- `neo4j-migration.skill.md` — Neo4j → NornicDB over Bolt, with phase order and exact UNWIND/MERGE shapes pinned to the cookbook.
32+
- `knowledge-policies.skill.md`, `decay-tuning.skill.md`, `promotion-policies.skill.md` — full DDL surface for decay/promotion, with disambiguated bundle-vs-binding language.
33+
- `managed-embeddings.skill.md`, `vector-search.skill.md`, `rag-procedures.skill.md` — search and RAG surfaces.
34+
35+
Knowledge-policy user guides (`docs/user-guides/knowledge-layer-policies.md`, `decay-profiles.md`, `promotion-policies.md`) rewritten in the same precise vocabulary: bundles are inert parameter packages, bindings carry the `FOR` target, decay is pure time math evaluated on read, `ON ACCESS` belongs to promotion only, `LAST_ACCESSED` decay only reads access metadata.
36+
37+
## Migration scripts — Neo4j and Qdrant
38+
39+
New `scripts/migration/` directory with runnable migrations in three languages each:
40+
41+
- `scripts/migration/neo4j/{migrate.py,migrate.go,migrate.mjs}` — Bolt → Bolt, schema first (constraints + indexes including fulltext and vector), then nodes via `UnwindSimpleMergeBatch`, then edges via `UnwindMultiMatchCreateBatch`. Keyset-paginated by `elementId`. Preserves source IDs on `_neo4j_id` and original labels on `_neo4j_labels` for an operator-driven label promotion pass.
42+
- `scripts/migration/qdrant/{migrate.py,migrate.go,migrate.mjs}` — gRPC → gRPC (Node uses REST since there's no maintained JS gRPC client). Replicates collection vector configs, scrolls and upserts points in batches, verifies counts. Idempotent on re-run.
43+
44+
All Go scripts vet and build clean against `github.com/qdrant/go-client v1.18.1` and `github.com/neo4j/neo4j-go-driver/v5 v5.28.4`.
45+
46+
## Documentation site — consolidated navigation, fixed search routing
47+
48+
Rewrote `mkdocs.yml`:
49+
50+
- Top-level navigation collapsed from 14 groups to 8: Home, Getting Started, Concepts & Guides, Skills (Agent-Ready), Migrating, API Reference, Operating, Building, Changelog. Sub-grouped within each (e.g. Concepts & Guides → Cypher / Search & RAG / Knowledge Layer / Temporal & Ledger / Topology & Multi-DB / Heimdall / Plugins).
51+
- **Every published `.md` file is registered in `nav:`.** This is the fix for the bug where clicking a search result loaded the page content but left the sidebar empty — Material renders the page on direct hit, but a page outside `nav:` has no anchor for the sidebar to highlight.
52+
- `use_directory_urls: true` made explicit; `site_url` produces a `<link rel="canonical">` on every built page.
53+
- Theme features tightened: `navigation.instant.progress`, `navigation.tracking`, `navigation.path`, `navigation.indexes`, `navigation.prune`, `search.share`, `toc.integrate`. The combination keeps URL ↔ sidebar ↔ content synchronized when navigating from a search result.
54+
- New top-level entries link the Skills directory and the Migrating section (Neo4j + Qdrant) so they're discoverable from the global nav, not just deep-linked from the README.
55+
56+
Root README cross-links the cookbook, the skills directory, and the migration scripts in the relevant feature and "why switch from" sections.
57+
58+
## Plan landed
59+
60+
`docs/plans/consumer-pinned-error-contract-plan.md` — single-pass plan documenting the wire-contract pinning above. Default values reconciled against `LoadDefaults()` truth (the original draft had `AsyncWritesEnabled` and `PersistSearchIndexes` defaults inverted; both are now correct).

0 commit comments

Comments
 (0)