feat: Add micronaut-data-nitrite embedded document store module - #3770
Closed
sfkamath wants to merge 73 commits into
Closed
feat: Add micronaut-data-nitrite embedded document store module#3770sfkamath wants to merge 73 commits into
sfkamath wants to merge 73 commits into
Conversation
sfkamath
changed the title
sfkamath
force-pushed
the
4.14.x-nitrite
branch
2 times, most recently
from
March 23, 2026 12:54
23f65cc to
037765b
Compare
Contributor
|
this might make sense in a separate project all together under micronaut-projects |
sfkamath
force-pushed
the
4.14.x-nitrite
branch
from
March 25, 2026 01:45
d1563a7 to
b72d867
Compare
Contributor
If it stays in micronaut-data I think it should target 5.0.x branch. |
sfkamath
force-pushed
the
4.14.x-nitrite
branch
3 times, most recently
from
March 27, 2026 21:09
18efc85 to
7094057
Compare
Author
Happy to target 5.0.x eventually, but the implementation is complete and passing all document TCK tests on 4.14.x today. Since 4.14.x is the current release and 5.0.x isn't out yet, would it make sense to land it on 4.14.x first and forward-port to 5.0.x as part of that release cycle? That way users on the current release get access to it sooner, and the 5.0.x port can account for any API changes in that branch. |
sfkamath
marked this pull request as ready for review
April 22, 2026 08:25
- Add micronaut-data-nitrite and micronaut-data-nitrite-test modules. - Implement robust transaction management using PropagatedContext. - Migrate to JPA Criteria API (QueryBuilder2) for JSON-based query building. - Fix Instant precision and sorting precedence issues. - Support type-agnostic numeric equality and UUID string conversion. - Add sync-nitrite.sh for ongoing source synchronization. - Centralize dependencies in libs.versions.toml.
- Implement TCK repository interfaces (Author, Book, BasicTypes, etc.). - Align NitriteBookRepository with TCK abstract class requirements. - Add comprehensive Javadoc to test models and repositories to satisfy Checkstyle. - Fix constructor whitespace for consistency with project standards. - Enable regex support signals in NitriteQueryBuilder.
NitriteDB integration fixes and documentation: - Fix JSON parsing so ':name' placeholders are preserved and bound correctly for user-authored JSON @query methods. - Handle JSON update documents that use a Mongo-style '' wrapper by unwrapping '' before Nitrite collection.update(...). - Ensure criteria UpdateQueryDefinition '' values that are BindingParameters are encoded as :N placeholders and included in query bindings. - Make equality comparisons tolerant to numeric representation differences (Long/Integer/Double vs BigDecimal) to stabilize findById(Long) and Instant equality queries. - Remove misuse of QueryResult additionalRequiredParameters as a metadata channel (prevents implicit CrudRepository method compilation failures). - Add module-level Nitrite implementation notes and link from repo README. - Improve test diagnostics (logback-test.xml with line numbers) and clarify ignored placeholder tests. Co-authored-by: Gemini <gemini@google.com> Co-authored-by: Codex <codex@openai.com>
- Document Nitrite query encoding/runtime contracts (Instant, numeric coercion, $set updates) - Explain key regression tests and ignored placeholders - Fix checkstyle WhitespaceAround and reduce Javadoc warnings - Keep supportsRegex() without @OverRide for Micronaut Data 4.x compatibility
- Move Nitrite verification from removed :data-nitrite-test into :data-nitrite (Spock repo specs, document TCK spec, and tx management specs) and wire test dependencies into data-nitrite/build.gradle so :data-nitrite:test is the entry point - Add composite/embedded id coverage (Project/ProjectId + ProjectRepository + ProjectRepositorySpec) and align doc-examples with the same pattern - Extend DefaultNitriteRepositoryOperations to handle document-processor SQL-like SELECT/DELETE alongside JSON filters, bind placeholders and named params, and apply Nitrite update semantics correctly for JSON payloads - Add Nitrite guide pages and TOC entries plus Java/Groovy/Kotlin nitrite doc-example projects and include them from settings.gradle - Tame noisy test logging via logback-test.xml (root INFO, Nitrite DEBUG, suppress env/classpath discovery spam) Co-authored-by: Codex <codex@openai.com>
…class size Extensive refactoring of the Nitrite module to improve architectural alignment with MongoDB and satisfy Checkstyle constraints: - Extracted entity and document mapping logic into 'NitriteEntityMapper', centralizing ID handling, numeric coercion (Long/BigDecimal), and normalization for Instant/UUID. - Introduced 'NitriteStoredQuery' and 'NitritePreparedQuery' decorators (with 'Default*' implementations) to pre-calculate Nitrite filters and state, shifting complexity away from execution time. - Extracted raw query parsing for both JSON and SQL-like strings into 'NitriteQueryParser'. - Moved JSON-to-Nitrite Filter construction into 'NitriteFilterBuilder'. - Delegated update and merge execution to 'NitriteUpdateExecutor', improving support for optional field aliases in SQL-like UPDATE statements. - Reduced 'DefaultNitriteRepositoryOperations' from ~2,500 to ~1,370 lines, well below the 2,000-line limit. - Fixed regressions in named parameter resolution for SQL statements and improved '' key filtering for Nitrite document updates. - Added 'NitriteRefactorRegressionSpec' to verify fixes for identified bugs. Verified with 'checkstyleMain' and full suite of 163 Nitrite tests including TCK. Co-authored-by: Gemini <gemini@google.com>
Optimized the Nitrite module performance by aligning its query execution model with MongoDB and introduced a new JMH benchmark project: - Refactored 'DefaultNitriteRepositoryOperations' to implement 'PreparedQueryDecorator' and 'MethodContextAwareStoredQueryDecorator'. - Enabled automatic caching of decorated 'StoredQuery' and 'PreparedQuery' objects, avoiding redundant JSON/SQL parsing on every repository method execution. - Updated 'DefaultNitriteStoredQuery' and 'DefaultNitritePreparedQuery' to extend standard 'DefaultBindableParameters*' base classes for improved consistency and reduced boilerplate. - Created ':micronaut-benchmarks:micronaut-benchmark-micronaut-data-nitrite' project to establish a performance baseline (measured at ~163k ops/s for simple finders). - Configured 'spotless' and 'checkstyle' for the new benchmark project via the 'io.micronaut.build.internal.data-module' plugin. - Fixed Checkstyle violations in benchmark source code (missing Javadoc, non-final classes). Co-authored-by: Gemini <gemini@google.com>
…rator
Introduced support for multiple storage engines and refined configuration to follow Nitrite conventions:
- Added 'StorageMode' enum (MVSTORE, IN_MEMORY, ROCKSDB) to 'NitriteConfiguration'.
- Implemented 'Pure' In-Memory mode by skipping storage module loading when no path is provided, as per Nitrite defaults.
- Added support for 'nitrite.field-separator' to handle complex property nesting conflicts.
- Integrated 'nitrite-rocksdb-adapter' for persistent RocksDB-backed storage.
- Added 'NitriteStorageModeSpec' to verify CRUD functionality across all storage engines.
- Added 'NitriteInMemoryVerificationSpec' to strictly confirm no disk I/O in Pure In-Memory mode.
- Updated JMH benchmark to measure all three modes, establishing a verified performance baseline (~232k ops/s for Pure In-Memory).
- Updated documentation ('nitriteConfiguration.adoc') to cover new settings.
Results (ops/s):
- IN_MEMORY: ~232,523
- MVSTORE: ~223,536
- ROCKSDB: ~101,983
Co-authored-by: Gemini <gemini@google.com>
- Corrected AsciiDoc header levels to resolve sequence errors. - Fixed snippet project-base and source paths to correctly locate example code. - Resolved missing file warnings for TransactionService and BookRepository snippets. Co-authored-by: Gemini <gemini@google.com>
…orage - Updated 'nitriteConfiguration.adoc' to include 'storage-mode' and 'field-separator' properties. - Refined defaults in documentation to match the new non-opinionated configuration (absent path = in-memory). - Updated 'nitriteQuickStart.adoc' example to use 'IN_MEMORY' mode. - Added note about adapter dependencies to 'nitriteLimitations.adoc'. - Standardized all headers to Level 1 for proper documentation sequencing. Co-authored-by: Gemini <gemini@google.com>
Refined documentation to show the most concise way to configure Nitrite:
- Updated 'nitriteQuickStart.adoc' to show that 'nitrite: {}' (empty path) defaults to in-memory.
- Clarified in 'nitriteConfiguration.adoc' that 'IN_MEMORY' mode is equivalent to leaving 'db-path' empty.
- Simplified examples to follow convention over configuration.
Co-authored-by: Gemini <gemini@google.com>
Implemented automated index management for Nitrite repositories: - Added logic to 'DefaultNitriteRepositoryOperations' to automatically create Nitrite indexes based on Micronaut Data '@Index' and '@indexes' annotations (supports compound and property-level indexes). - Automatically creates a unique index on the 'id' field for the identity property to ensure optimal lookup performance. - Exposed 'nitrite.create-indexes' configuration (defaults to 'true') to control this behavior. - Added 'NitriteIndexSpec' and 'IndexedBook' test model to verify index creation. - Updated documentation with a new 'Automated Index Creation' section and updated TOC. Co-authored-by: Gemini <gemini@google.com>
Enabled advanced indexing capabilities for Nitrite repositories: - Introduced '@FullTextIndex' and '@SpatialIndex' annotations. - Updated 'DefaultNitriteRepositoryOperations' to automatically create full-text and spatial indexes alongside standard unique/non-unique indexes. - Made RocksDB and Spatial modules optional using 'compileOnly' and dynamic reflection-based loading in 'NitriteOperationsFactory' to avoid forcing transitive dependencies. - Added 'nitrite-spatial' to test dependencies and updated 'NitriteIndexSpec' to verify all index types. - Updated 'libs.versions.toml' with correct artifact IDs. Additional changes: - Renamed 'presentTest' source set to 'rocksDbPresentTest' for clarity - Added 'spatialPresentTest' source set for optional spatial module testing - Moved 'IndexedBook' entity and repository to 'spatialPresentTest' source set - Added 'IndexedBook' examples to doc-examples (Java, Groovy, Kotlin) for documentation - Fixed Javadoc tag error in 'DefaultNitriteRepositoryOperations' - Added Javadoc comments to test classes to satisfy checkstyle Co-authored-by: Gemini <gemini@google.com> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Fixed spatial filter ($near, $within, $intersects) implementation and added complete documentation for optional Nitrite modules. Bug fixes: - Fixed NitriteFilterBuilder to correctly handle Geometry types in spatial filters - Changed $near to use Coordinate.extract() from Geometry instead of raw Geometry - Fixed $within and $intersects to pass Geometry directly with correct method signatures - Added SLF4J logging (log.debug) instead of System.out for debugging - Fixed test coordinates to use correct JTS (longitude, latitude) order Documentation: - Added nitriteSpatial.adoc with comprehensive spatial query documentation - Documented all 3 optional modules: nitrite-spatial, rocksdb adapter, jackson mapper - Added Java, Kotlin, and Groovy examples for all spatial filters ($near, $within, $intersects) - Included important note about JTS Coordinate (longitude, latitude) order - Updated nitriteConfiguration.adoc with optional modules section - Added complete usage examples with GeometryFactory and JTS types - Fixed all external links to use verified URLs Example files: - Added IndexedBookRepository with @query spatial methods (Java, Kotlin, Groovy) - Added SpatialExample demonstrating $near, $within, $intersects usage (Java, Kotlin, Groovy) - Updated IndexedBook entity with @SpatialIndex annotation tags (Java, Kotlin, Groovy) Build configuration: - Added jackson-datatype-jts v2.21.0 for JTS Geometry JSON serialization - Configured compileOnly and spatialPresentTest source sets for optional dependencies All tests pass: test (168), spatialPresentTest (1), rocksDbPresentTest (1) Documentation builds successfully with ./gradlew docs Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…teEntityOperations Refactor write operations to use the new ObjectRepositoryWriter and CollectionWriter strategy classes for entity-to-Document conversion. Changes: - NitriteEntityOperations: Use ObjectRepositoryWriter.toDocument() instead of NitriteEntityMapper.toDocument() for single entity operations - NitriteEntitiesOperations: Use ObjectRepositoryWriter.toDocument() instead of NitriteEntityMapper.toDocument() for batch entity operations Benefits: - Centralized entity-to-Document conversion logic - Consistent with READ operation strategy pattern - Easier to maintain and test write operations - Clear separation between conversion (Writer) and persistence (Operations) The Operations classes still handle: - Event lifecycle (pre/post persist, update, remove) - Optimistic locking (version checks) - Cascade operations - Context management (tracking persisted entities) While Writer classes handle: - Entity to Document conversion - Version preparation (prepareForInsert, prepareForUpdate) - Batch document conversion All 234 tests passing. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Move version handling LOGIC to ObjectRepositoryWriter: - needsVersionInit() - Check if entity needs version initialization - getNextVersionValue() - Calculate next version value Update Operations to use Writer's version methods: - NitriteEntityOperations uses needsVersionInit() for inserts - NitriteEntitiesOperations uses needsVersionInit() for batch inserts Trade-offs: - Version MUTATION (helper.updateEntityId) remains in Operations because it must handle immutable entities (records) - Writer provides centralized version LOGIC - Operations handle entity mutation mechanics This separation allows: - Writer to be the source of truth for version rules - Operations to handle entity mutation (mutable vs immutable) - Future refactoring to move mutation to Writer if needed All 234 tests passing. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
The CollectionAggregator now handles both camelCase and snake_case field names when retrieving values from Nitrite documents. Issue: Micronaut Data's default naming strategy stores fields in snake_case (e.g., date_of_birth) but the extractFieldName() method returns camelCase (e.g., dateOfBirth). Fix: When d.get(fieldName) returns null, try the snake_case conversion before giving up. This allows aggregation methods like findMaxDateOfBirthByNameRegex() to work correctly. Test: test date project on single property now passes (235/235 tests) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
## Problem
Method-name queries on ONE_TO_MANY associations failed because the query builder
was losing the property path information at compile time.
Example: Author.findByBooksTitle("The Stand")
- Generated: {"book_author": {"": ":0"}} ← .title lost
- Needed: {"book_author.title": {"": ":0"}}
## Root Cause
NitritePredicateVisitor.getFieldNameInternal() stopped at the association field
name for all non-embedded associations, discarding the target property name.
## Solution
### Query Builder (NitritePredicateVisitor.java)
- For ONE_TO_MANY/MANY_TO_MANY: Continue building property path (e.g., "book_author.title")
- For MANY_TO_ONE: Stop at FK field (e.g., "author_id") - unchanged behavior
### Runtime (NitriteFilterBuilder.java)
- Added buildAssociationFilter() to detect and handle association queries
- Added SubQueryExecutor interface for runtime sub-query execution
- Implemented auto-join for FK mismatches (e.g., author_id compared to "Stephen King")
### DefaultNitriteRepositoryOperations.java
- Added createFilterBuilderWithSubQueryExecutor() to configure sub-query support
- Pass entity metadata to SQL parsers for association detection
## Test Results
- Re-enabled: test project on single ended association (35/35 tests passing)
- Still excluded: test join on many ended association (@Join not implemented)
## Files Modified
- data-nitrite/build.gradle: Re-enabled test exclusion
- NitritePredicateVisitor.java: Preserve property paths for ONE_TO_MANY
- NitriteFilterBuilder.java: Auto-join implementation (~300 lines added)
- DefaultNitriteRepositoryOperations.java: Sub-query executor setup
## Impact
- Criteria queries with ONE_TO_MANY reverse lookups now work correctly
- MANY_TO_ONE auto-join works for non-UUID ID comparisons
- No breaking changes to existing functionality
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…ciations
## Problem
The test 'test join on many ended association' was excluded because @Join
annotation was not supported for ONE_TO_MANY and MANY_TO_MANY associations.
When using @Join('books') on a repository method, the books association
was not being fetched, resulting in empty collections.
## Root Cause
NitriteEntityMapper.fromDocumentInternal() was auto-hydrating all mappedBy
associations unconditionally, which:
1. Made associations eager by default (incorrect behavior)
2. Didn't respect @Join annotation semantics
3. Caused the TCK test to fail because findAll() (without @Join) was
returning entities with populated associations
## Solution
### NitriteEntityMapper.java
- Removed auto-hydration of mappedBy associations from fromDocumentInternal()
- Associations are now lazy by default (correct behavior)
- @Join annotation explicitly controls when associations are fetched
### NitriteQueryExecutor.java
- Added fetchJoins() method to fetch joined associations after loading entities
- Added fetchSingleLevelJoin() to handle single-level @Join paths
- Modified findOne() and findAll() to call fetchJoins() when @Join is specified
- For ONE_TO_MANY/MANY_TO_MANY: queries associated collection using back-reference
- Groups results by parent ID for efficient batch fetching
## Implementation Details
- Only handles single-level joins (e.g., @Join('books'))
- Nested joins (e.g., @Join('books.pages') require future enhancement
- Uses Filter.or() for batch fetching multiple parents efficiently
- Falls back to single equality filter when only one parent exists
## Test Results
- Re-enabled: test join on many ended association (36/36 tests passing)
- All existing tests continue to pass
- Associations are now lazy by default, fetched only with @Join
## Files Modified
- NitriteEntityMapper.java: Removed auto-hydration (~30 lines removed)
- NitriteQueryExecutor.java: Added @Join support (~130 lines added)
- build.gradle: Removed test exclusion
## Impact
- @Join annotation now works for ONE_TO_MANY and MANY_TO_MANY associations
- Associations are lazy by default (TCK compliant)
- No breaking changes to existing functionality
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…d new sections Add comprehensive Nitrite documentation covering optimistic locking, association mapping, and criteria API with full multi-language examples (Java/Kotlin/Groovy). New Documentation: - nitriteOptimisticLocking.adoc: @Version with Long and Instant types, partial updates - nitriteAssociationMapping.adoc: @relation kinds, cascade types, bidirectional associations - nitriteAssociationFetching.adoc: @Join usage, batch fetching, limitations - nitriteSpatial.adoc: Geometry types, spatial filters ($near, $within, $intersects) - nitriteTransactions.adoc: @transactional usage with Nitrite repositories New Example Entities (Java/Kotlin/Groovy): - Author: ONE_TO_MANY association with cascade=PERSIST - Address: Embedded entity example - VersionedBook: Optimistic locking with Long version - VersionedBookTemporal: Optimistic locking with Instant version - VersionedBookRepository, VersionedBookTemporalRepository: Repository examples - AuthorRepository: @Join fetching, reverse lookups - AuthorRepositoryExample, GeometryTypesExample: Usage examples Modified Entities with AsciiDoc Tags: - Book: Added tag::book[] with MANY_TO_ONE association callout - BookRepository: Added repository methods with tags (update, projections, case-insensitive, IN queries) - PersonRepository: Added criteria API tags (find, count, update, delete, sorting-pagination, specifications) - PersonRepositoryExample: Added usage example tags Documentation Standards: - All examples use multi-language tabs with proper callout markers - Section heading levels fixed (=== for subsections within == sections) - Cross-references added between related sections - Follows MongoDB/Cosmos DB documentation patterns Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Fix NitriteEntityOperations.execute() to use upsert when entity ID is present - ID null → INSERT with generated ID - ID present → UPSERT (update or insert if absent) - Applies to both IN_MEMORY and MVSTORE storage modes - Works for UUID, String, and Long ID types - Fix NitriteEntitiesOperations for batch save operations with same upsert logic - Handles mixed batches of new and existing entities correctly - Fix NitriteFilterBuilder.buildFieldFilter() for @query JSON filters - Convert camelCase property names to snake_case persisted names - Resolves bug where @query filters with camelCase fields failed to match - Example: {"firstName": "John"} → {"first_name": "John"} - Add comprehensive regression tests - NitriteUpsertSpec: 26 tests covering upsert behavior across ID types and storage modes - NitriteQueryFilterBugSpec: Test camelCase→snake_case conversion in @query filters - NitriteDuplicateResultsBugSpec: Verify no duplicates in MVSTORE mode - Update NitriteLoggingSpec assertion for upsert behavior - Update documentation - nitriteQuickStart.adoc: Document save() upsert semantics - nitriteRepositories.adoc: Add 'Save Method Upsert Semantics' section - nitriteAssociationFetching.adoc: Document association name matching for irregular plurals - nitriteNaming.adoc: Document @query JSON filter field naming convention - Add Javadoc comments to internal operations classes - NitriteEntityOperations: Document save and update operation behavior - NitriteEntitiesOperations: Document saveAll and updateAll operation behavior - NitriteFilterBuilder: Document association name matching Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…ead of sub-query
- Fix NitriteFilterBuilder.buildNestedFilter() for ONE_TO_MANY and MANY_TO_MANY associations
- Previous implementation used elemMatch() which assumes embedded arrays (MongoDB-style)
- Nitrite stores ONE_TO_MANY as foreign keys in child documents, not embedded arrays
- Example: State {id, name} and City {id, name, state_id} - cities NOT embedded in State
- Implement sub-query approach for collection associations (matching buildAssociationFilter)
- Extract mappedBy property (the FK in child entity pointing back to parent)
- Build sub-query filter on associated entity (e.g., City where name = 'Austin')
- Execute sub-query to get matching back-reference values (state_ids)
- Filter main entity by ID IN (matchingValues)
- Handle parameter resolution in sub-query filter construction
- Resolve positional and named parameters before building sub-query filter
- Ensure field name is included in sub-query filter map
- Fallback to elemMatch when SubQueryExecutor is unavailable
- Maintains API compatibility for non-Nitrite use cases
- Test coverage
- Add 'test multiple cities with same state' to PluralAssociationSpec
- Verifies findByCitiesName() works correctly with sub-query approach
- All 4 PluralAssociationSpec tests now pass
Fixes: Query produced no result when filtering State by cities.name using findByCitiesName()
Root cause: elemMatch(city_state: name == Austin) fails because cities are not embedded in State documents
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…sted filters - Fix lifecycle events on save(existing) Overrode persist() in NitriteEntitiesOperations to correctly split batches into new and existing entities. Now triggers PrePersist/PostPersist for new entities and PreUpdate/PostUpdate for existing ones. Ensures @DateUpdated and @Version are correctly updated during upserts. Added NitriteUpsertLifecycleSpec to verify timestamp and version updates. - Fix MANY_TO_MANY join and filter support Updated NitriteQueryExecutor to use a custom Filter that correctly handles collection fields in Nitrite 4.x (using 'contains' logic). Fixed result grouping in fetchSingleLevelJoin to handle back-reference collections instead of assuming scalar IDs. Updated DefaultNitriteRepositoryOperations to flatten results in subQueryExecutor, supporting MANY_TO_MANY result mapping. Updated NitriteFilterBuilder to support owner-side MANY_TO_MANY filtering (mappedBy == null). - Fix nested MANY_TO_ONE association filtering Modified NitriteFilterBuilder to correctly structure sub-filter maps for nested associations (e.g. author.name), preventing the property name from being lost. - Fix Criteria API IllegalArgumentException Updated AbstractRuntimePersistentEntityJoinSupport to return correct RuntimePersistentAssociationPath instances for associations instead of generic property paths. - Test Coverage Added NitriteUpsertLifecycleSpec, TimestampedRecord, and TimestampedRecordRepository to cover lifecycle regressions. Existing NitriteAssociationRegressionSpec now passes. Reported-by: Codex Fixed-by: Gemini CLI
…tering - Added NitriteManyToManyFilterSpec to verify filtering on both sides of MANY_TO_MANY relationships. - Added NitriteOneToManyFilterSpec to verify filtering on ONE_TO_MANY and MANY_TO_ONE relationships. - Added supporting models (OneToManyParent, OneToManyChild) and repository methods. - Verified that recent fixes in NitriteFilterBuilder correctly handle all association cardinalities.
This test verifies that criteria API association navigation works correctly
using root.get("association").get("property") pattern.
The test demonstrates the fix in AbstractRuntimePersistentEntityJoinSupport
which returns proper RuntimePersistentAssociationPath instances for
associations instead of generic property paths.
Without this fix, navigating associations throws:
IllegalArgumentException: Property path doesn't support get operation
This is required for Nitrite (and other NoSQL stores) to support
PredicateSpecification with association paths like:
cb.equal(root.get("author").get("name"), "Stephen King")
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…d optimistic locking
This commit documents the fixes introduced in the preceding runtime commits
(upsert lifecycle, MANY_TO_MANY join/filter, ONE_TO_MANY reverse lookup,
MANY_TO_ONE criteria navigation) and cleans up incorrect documentation
added by a prior automated pass.
## Example files (Java / Groovy / Kotlin)
### AsciiDoc callout markers
All `// <1>` callout markers that were placed on annotation lines (e.g.
`@Version // <1>`, `@Relation(...) // <1>`) have been moved to the
corresponding field declaration line. The snippet macro only recognises
callout markers on non-tag lines, so placing them on annotation lines
caused "no callout found for <1>" build warnings.
Affected entities: Author, Book, Student, VersionedBook, VersionedBookTemporal
(all three language variants each).
### Removed incorrect entities
Deleted Department / DepartmentRepository files (Java, Groovy, Kotlin) that
were introduced to demonstrate MANY_TO_ONE criteria navigation but used the
wrong domain model. The existing Book → Author relationship already covers
this use case.
### Reverted Person / PersonRepository
Removed the `@Relation(Kind.MANY_TO_ONE) Department department` field and
the `departmentNameEquals` association-navigation tag that were added to
Person / PersonRepository as part of the incorrect approach above.
### AuthorRepository — eager loading
Added `@Join("books")` overloads for `findById`, `searchByName`, and
`findAll` so that the association-fetching doc examples have concrete,
tested methods to reference. Also added the `reverse-lookup` tagged snippet
(`findByBooksTitle`) for the ONE_TO_MANY reverse-lookup section.
### BookRepository — criteria navigation + JpaSpecificationExecutor
Extended BookRepository to implement `JpaSpecificationExecutor<Book>` and
added an inner `Specifications` class with an `authorNameEquals` spec that
navigates `root.get("author").get("name")`. This is the canonical example
for MANY_TO_ONE criteria association queries and replaces the deleted
Department approach.
Also added `@Join("students")` on `findAll` (MANY_TO_MANY owning-side eager
fetch) tagged as `join-students`.
### BookRepositoryExample — many-to-many-fetch snippet
Added a `many-to-many-fetch` tagged method showing how to iterate over
eagerly loaded students after a `@Join` fetch. This replaces the hardcoded
Java-only source block that was in nitriteAssociationFetching.adoc.
### VersionedBookExample
Replaced test-style `assert` callouts (e.g. `assert created.getVersion() == 0 // <1>`)
with clean, self-contained example methods. The `save-versioned-new` and
`save-versioned-existing` tags are now separate methods so that callout
`<1>` and `<2>` each target a single meaningful statement.
### VersionedBookRepository — split tag
Split the single `versioned-repository` tag into:
- `versioned-repository-declaration` — just the interface header (used in
the "Using Optimistic Locking" intro section)
- `versioned-repository` — the partial update/delete methods (used in the
"Partial Updates" section)
This prevents the same snippet from appearing twice with different framing.
## AsciiDoc files
### nitriteAssociationMapping.adoc
Removed a duplicate `snippet::example.Book[tags="book-many-to-one"]` from
the "Example: Many-To-One" section. The same snippet was already shown in
the immediately preceding "One-To-Many with Cascade Persist" section;
duplicating it there was redundant. Replaced with a prose sentence
cross-referencing the already-visible entity.
### nitriteAssociationFetching.adoc
- Replaced "Supported Association Types" bullet list that used JPA-style
annotation names (`@OneToOne`, `@ManyToOne`, etc.) with the correct
Micronaut Data `Kind.X` enum names.
- Replaced hardcoded `[source,java]` MANY_TO_MANY fetch block with the
proper multi-language `snippet::example.BookRepositoryExample[tags="many-to-many-fetch"]`.
- Corrected the MANY_TO_MANY batch-fetch description: Nitrite does not use
a separate "join collection"; the owning side stores associated IDs
inline, so the fetch reads those IDs and issues an `IN` query on the
associated collection.
- Fixed `@OneToMany` → `ONE_TO_MANY` in the Association Name Matching
section.
### nitriteOptimisticLocking.adoc
Changed the "Using Optimistic Locking in Repositories" snippet reference
from `tags="versioned-repository"` to `tags="versioned-repository-declaration"`
so that the intro section shows only the interface declaration rather than
the partial-update methods (which belong to the "Partial Updates" section).
### nitriteCriteriaExecuteQuery.adoc
- Changed the criteria query example from `PersonRepository` to
`BookRepository` for the association-navigation snippet, matching the
corrected example files.
- Updated surrounding prose to describe the Book → Author MANY_TO_ONE
navigation (`root.get("author").get("name")`).
- Removed a misleading "Query Format" bullet that claimed association
queries use a `book_author.title` dot-notation path — this was
inaccurate and not relevant to user-facing API usage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ughput This commit resolves severe performance regressions introduced in recent updates by addressing several hotpaths identified during benchmarking: - Removed the custom `NumericFilter` class and restored `FluentFilter` with precise type coercion in `NitriteEntityMapper`. The custom filter was bypassing Nitrite's internal query optimizer, causing full-table scans for all numeric equality checks. - Bypassed expensive `runtimeEntityRegistry.getEntity()` lookups for standard JDK classes (java.*) in `toFilterValue`. - Added fast paths for `String`, `Number`, `Boolean`, and `Character` in `toFilterValue` to completely bypass the `instanceof` cascade and `Class.getClass()` invocation, reducing overhead for primitive mappings. - Pre-compiled multiple recurring regex `Pattern` instances in `CollectionAggregator` and `NitriteQueryExecutor` to prevent continuous recompilation during query generation. - Optimized `NitriteQueryExecutor.findOne()` to execute the database cursor fetch exactly once per method call, instead of redundantly across projection checks. - Avoided invoking `collection.getName()` inside the index auto-creation loop by passing down the known string name. These bottlenecks were successfully identified using the jvm-hotpath agent (https://github.com/sfkamath/jvm-hotpath) and resolved by Gemini CLI.
…ilter parsing overhead This commit introduces an AST-like pre-compilation model for Nitrite JSON filters, resolving a major performance bottleneck where query structures were being re-parsed from Map/JSON on every execution. Key changes: - Introduced CompiledNitriteFilter and CompiledValue functional interfaces for binding pre-compiled structures to runtime parameters. - Refactored NitriteFilterBuilder.compile() to transform JSON-like Maps into highly optimized bind lambdas. - Pre-calculated field metadata (persisted names, identity status, and association potential) during the one-time compilation phase, moving expensive RuntimePersistentEntity lookups out of the execution path. - Optimized the common equality path to use FluentFilter.where().eq() directly in the bind lambda. - Updated DefaultNitriteRepositoryOperations and NitriteQueryExecutor to store and utilize the CompiledNitriteFilter for recurring stored queries. - Disabled benchmark console logging to ensure clean JMH throughput measurements. Benchmarks show a 2x to 4x throughput increase for standard repository queries after these optimizations.
…tion This commit evolves the filter pre-compilation model from an opaque lambda-based approach to a structured Abstract Syntax Tree (AST). This architectural shift ensures that filter structures are parsed and validated once at compilation time, while still allowing dynamic parameter resolution and execution strategy decisions at runtime. Key changes: - Introduced NitriteFilterAST with AndNode, OrNode, FieldNode, and AllNode implementations. - Refactored NitriteFilterBuilder.compile() to transform JSON-like Maps into a tree of AST nodes. - Updated CompiledValue with typed records for Literal, Parameter, and NamedParameter to ensure stable binding regardless of runtime parameter reordering. - Preserved the recursive buildFieldFilter logic within FieldNode.toFilter() to maintain 100% correctness for complex associations and nested paths. This structured model eliminates runtime Map traversal and string parsing overhead while passing all 281 tests in the Nitrite module.
This commit optimizes the structured filter AST by introducing specialized nodes for the most common query patterns. These nodes pre-calculate metadata and selection logic, removing almost all runtime overhead from standard repository operations. Key changes: - Introduced SimpleEqualityNode and SimpleOperatorNode to directly emit FluentFilter objects for standard property queries, bypassing heavy recursive logic and metadata lookups. - Implemented DynamicFieldNode as a stable fallback for complex association paths, dot-notation, and reverse lookups, ensuring 100% behavioral compatibility. - Improved association detection in NitriteFilterBuilder.compileFieldFilter to correctly identify foreign key fields (e.g., author_id) during the one-time compilation phase. - Exposed prepareFilterValue and buildOperatorFilter in NitriteFilterBuilder to allow AST nodes to reuse optimized type coercion and behavioral logic. These optimizations restore the 4x performance gain observed in benchmarks while maintaining 100% stability across all 281 tests in the Nitrite module.
…ed specs Test Coverage Improvements: - Add NitriteConfigurationSpec (16 tests): Complete coverage of NitriteConfiguration - All setter methods (setDbPath, setUsername, setPassword, setStorageMode, setFieldSeparator, setCreateIndexes) - All StorageMode enum values (MVSTORE, IN_MEMORY, ROCKSDB) - Configuration binding from properties - Add NitriteEventSpec (28 tests): Event-based predicate and temporal type tests - IS TRUE / IS FALSE predicates via Criteria API - IS NULL / IS NOT NULL predicates - IS EMPTY / IS NOT EMPTY predicates - Temporal type handling (Instant, LocalDate, LocalDateTime) - Comparison operators (greaterThan, lessThanEquals) - Pattern matching (Containing, StartingWith, EndingWith) - Logical operators (AND, OR, NOT) via Criteria API - IN / NOT IN with empty collections - Add NitriteProjectionSpec (10 tests): Single-property projection tests - findAllNames projection via ProjectionPersonRepository - Count projection tests - Enhance CriteriaPersonRepositorySpec (+12 tests): Criteria API coverage - IS NULL / IS NOT NULL predicates - BETWEEN predicate with inclusive bounds - IN / NOT IN predicates - OR / NOT logical operators - LIKE predicates (startsWith, endsWith, contains) - Enhance PersonRepositorySpec (+8 tests): Derived query and Criteria API tests - IN / NOT IN operators via Criteria API - Empty collection handling for IN / NOT IN - OR / AND operators via Criteria API - IS TRUE / IS FALSE via Criteria API - Enhance NitriteUpsertSpec (+4 tests): Optimistic locking tests - saveAll with versioned records (batch optimistic locking) - Multiple updates incrementing version Bug Fixes: - NitritePredicateVisitor: Fix NPE when handling empty IN/NOT IN collections - Handle nested Collection<?> from Criteria API - Use Collections.singletonMap() instead of Map.of() to allow null values Repository Changes: - PersonRepository: Extend JpaSpecificationExecutor<Person> for Criteria API support - EventRepository: Already extends JpaSpecificationExecutor<Event> - Add ProjectionPersonRepository for projection tests Result: 331 tests passing with comprehensive coverage across all nitrite module classes Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…ConversionService
Implements the intended layering for the Nitrite module:
Repository Interface (Entity In, Entity Out)
↓
Micronaut Serde (ObjectMapper) ← this commit
(Entity ↔ Map<String,Object>)
↓
NitriteEntityMapper (Map ↔ Document)
↓
Jackson (internal to Nitrite only)
Changes
───────
NitriteEntityMapper
- Inject io.micronaut.serde.ObjectMapper (replaces SerdeRegistry which was
wired but never used — dead code)
- convertFromDocumentValue: uses Serde for Map → POJO conversions (embedded
document hydration), ConversionService for scalar coercions (String → Instant,
Integer → Long, etc.) which are cheaper without a JSON round-trip
- serializeForDocument (new): delegates to toFilterValue for all types it
handles cheaply (Instant, UUID, Enum, temporal, etc.); only falls through to
Serde for custom non-java.* POJOs with Jackson/Serde annotations — keeps
JMH throughput neutral on the hot path
NitriteOperationsFactory
- Factory @bean method now injects io.micronaut.serde.ObjectMapper and
passes it through to DefaultNitriteRepositoryOperations
- Renamed MicronautJacksonMapperModule → NitriteJacksonMapperModule: the
internal Jackson ObjectMapper is a fresh locally-constructed instance, not
Micronaut's ObjectMapper bean; the old name was misleading
- Javadoc on createJacksonMapperModule corrected to reflect actual architecture
build.gradle
- micronaut-serde-jackson, micronaut-serde-support: api → implementation
(consuming classes are all @internal)
- nitrite-mvstore-adapter, nitrite-jackson-mapper, jackson-datatype-jsr310:
api → implementation (only used inside @internal factory/mapper classes;
not part of the public compile surface)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e queries Replaces ISO string storage of temporal fields with epoch-based numeric values. Nitrite performs lexicographic string comparison on String fields, making gt/lt/ between queries on dates unreliable. Numeric fields use Nitrite's native numeric comparator, which is both correct and faster. Storage format ────────────── Instant → epoch nanoseconds (Long) — full nanosecond precision LocalDateTime → epoch nanoseconds via UTC (Long) LocalDate → epoch day (Long) — no timezone assumption LocalTime → nanosecond of day (Long) Helper methods epochNanos(Instant) / fromEpochNanos(long) are added to NitriteEntityMapper as the single source of truth used by all converters. Read path ───────── convertFromDocumentValue (NitriteEntityMapper): Number → temporal reverse mapping added ahead of the ConversionService fallback. Accepts Integer, Long, or Double since Nitrite's Jackson may return any of these depending on value magnitude. convertValue (DefaultNitriteRepositoryOperations): same reverse mapping for projected scalar results (e.g. findMaxDateOfBirth returns LocalDate directly). The old ISO-string parse branches are removed. findOne(PreparedQuery) (DefaultNitriteRepositoryOperations): projected Number results are now converted via convertValue before reaching the framework interceptor, which would otherwise call ConversionService and fail. Filter/query path ───────────────── Five separate toFilterValue / convertValue / preConvertForFilter methods each had their own temporal conversion that had drifted out of sync with storage: NitriteEntityMapper.toFilterValue — primary, updated to epoch numbers NitriteCriteriaExecutor.toFilterValue — updated to match NitritePredicateVisitor.convertValue — updated to match (JPA criteria) NitriteFilterBuilder.preConvertForFilter — updated (was using double epoch seconds, now epoch nanos) NitriteQueryExecutor.toFilterValue — redundant handlers removed; delegates to entityMapper.toFilterValue Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ot-path overhead Hotpath analysis showed NitriteEntityMapper.convertToDocumentInternal() accounting for ~43.5% of all execution ticks, with the dominant cost being a 15-step instanceof chain, ClassUtils.forName() geometry class lookups, and Relation annotation reads on every property of every persisted entity. Introduce a `ConcurrentHashMap<Class<?>, NitriteEntityMeta<?>>` on the mapper instance. On first access per entity type, `buildEntityMeta()` iterates the persistent properties once and produces two immutable lists: - `writableProps` — non-readOnly, non-@transient properties with a pre-classified `PropertyStrategy` enum constant and pre-fetched `associatedIdProp`/`backRefProperty` BeanProperty references. - `mappedByAssocs` — RuntimeAssociation properties that have a `mappedBy` value; used exclusively by `persistOne()` for back-reference wiring. `buildEntityMeta()` is side-effect-free, so concurrent first-access builds on the same type produce identical immutable results; `computeIfAbsent` stores one winner. `classifyValueStrategy(Class<?>)` maps a declared property type to one of: JAVA_PASSTHROUGH, INSTANT, UUID, ENUM, LOCAL_DATE, LOCAL_DATETIME, LOCAL_TIME, URL, URI, CHARSET, OPTIONAL, ENTITY_ID_REF, GEOMETRY, SERDE, ASSOCIATION_EMBEDDED, ASSOCIATION_ID_REF, ASSOCIATION_IDS_REF, ASSOCIATION_MAPPED_BY. The hot loop in `convertToDocumentInternal()` is now a plain switch over the pre-computed constant — zero instanceof checks, zero annotation reads, zero classloader lookups per iteration. The JTS geometry class is resolved once in the constructor via ClassUtils.forName() and cached in `geometryClass`; GEOMETRY strategy is only assigned when that field is non-null, so the GEOMETRY switch arm is unreachable when JTS is absent. Replace the per-call loop that read `Relation.mappedBy` from annotation metadata with a loop over the pre-computed `meta.mappedByAssocs()` list. The annotation read now happens once at meta-build time rather than on every persist. Add NitriteEntityMetaDispatchSpec with four targeted tests: - @transient field excluded from document and null on reload - SERDE strategy round-trips a custom @introspected POJO (CustomTag) - Non-@relation entity property stored as ASSOCIATION_ID_REF (book_ref_id) by the annotation processor's auto-classification of @MappedEntity-typed fields - Empty mappedByAssocs list safely iterated (no NPE) for entities with no bi-directional associations Add GEOMETRY strategy assertion to NitriteIndexSpec: verifies that Geometry objects are preserved as-is in the reloaded entity rather than serialized to a Map or String. Supporting test fixtures: CustomTag (@introspected POJO), MappingTestEntity (@MappedEntity with @transient, CustomTag, and Book fields), MappingTestRepository. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes a regression where single-field projections (e.g., findSessionIdByLevel) failed for properties with explicit @MappedProperty naming or camelCase-to- snake_case conventions. - Enhance NitriteEntityMapper.normalizeFieldName to use RuntimePersistentEntity for property-to-persisted-name mapping (e.g., sessionId -> session_id). - Update CollectionFieldMapper regex to correctly capture full camelCase property names from method names (e.g., 'SessionId' instead of 'session'). - Pass RuntimePersistentEntity to centralized projection strategy classes (CollectionFieldMapper, CollectionProjectionMapper) in NitriteQueryExecutor. - Restore projection result extraction logic that correctly uses normalized field names when reading from Nitrite Documents. Add NitriteSnakeCaseRegressionSpec to verify native projections for properties with explicit snake_case mapping.
…ough call chains Root cause: static entity metadata was re-derived on every operation via chained runtime lookups (getIdentity(), getProperty(), getEntity()). Fixed by computing once in buildEntityMeta() and propagating NitriteEntityMeta through call chains. NitriteEntityMeta record extended with: - cascadeProps: pre-filtered list of cascade-capable associations - hasBackReferences: boolean flag for early exit in persistOne - idAccessor: cached BeanProperty for ID access - persistentEntity: cached RuntimePersistentEntity reference buildEntityMeta(): compute all above fields once per entity type getOrBuildMeta(): use get-first pattern before computeIfAbsent NitriteEntitiesOperations: cache meta at batch start, reuse for all entities NitriteEntityOperations: hoist meta to method top in persist() and execute() triggerPrePersist(): iterate cascadeProps directly, remove ArrayList copy, pass Iterable directly to persistBatch persistOne(): check hasBackReferences before iterating mappedBy associations, hoist backRefProperty null check outside inner loop generateIdIfNecessary(): use meta.idAccessor() instead of registry lookup + getProperty() chain toDocumentInternal(): call getEntity() once, reuse for EmbeddedId and idProperty NitriteEntityMapper: add idEqualsFilter(NitriteEntityMeta, Object) overload, delegate Class-based method to it DefaultNitriteRepositoryOperations: cache non-transactional collections by name in collectionCache; transaction path remains dynamic Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> Reviewed-by: Claude Sonnet 4.6 Extended <claude@anthropic.com>
This commit addresses several issues with Nitrite index usage and query optimization: ## Changes ### 1. Use Nitrite's native $between operator (NitritePredicateVisitor.java) - Replace decomposed $gte + $lte with native $between operator - Reduces filter evaluation overhead from 2 filters to 1 - Enables more efficient index range scans ### 2. Fix compound index column name translation (DefaultNitriteRepositoryOperations.java) - Translate property names to persisted names before creating indexes - Fixes issue where compound indexes were created with wrong column names - Ensures compound indexes are properly recognized by Nitrite's FindOptimizer ### 3. Optimize numeric coercion for compound indexes (NitriteEntityMapper.java) - Add property lookup by persisted name as fallback - Infer target type from property metadata for precise numeric coercion - Reduce unnecessary OR filters that prevent compound index usage - Preserve index optimization opportunities by being conservative with type fallbacks ### 4. Add compound index test coverage - DirectNitriteCompoundIndexTest.groovy: Validates FindOptimizer selects compound indexes - NitriteIndexSpec.groovy: Extended with compound index query tests - IndexedBookRepository: Added findByTitleAndPages() method for testing ## Testing - Compound index creation now uses correct persisted column names - Queries with AND conditions on compound-indexed fields use index scan - Between queries generate single $between filter instead of $gte+$lte - Numeric equality checks preserve type information for index optimization Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
`NitriteIdModule` was not being registered with Jackson's `ObjectMapper`, causing `NitriteId` values to fall back to default serialization behavior instead of Nitrite's custom serializers. **Changes** - Register `NitriteIdModule` via reflection in `NitriteOperationsFactory.java` when creating the `ObjectMapper`, avoiding a hard dependency on `nitrite-jackson-mapper` - Log a warning if module registration fails **Why it matters** Without `NitriteIdModule`, ID format consistency can vary across Jackson versions, cross-version database compatibility may break, and ID deserialization edge cases can fail silently. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
…panded filter types Replace the bedatadriven JtsModule with Nitrite's own GeometryModule for consistent JTS Geometry serialization — ensuring spatial data is written and read in the same format that nitrite-spatial expects internally. Expand $near filter handling to support GeoPoint, JTS Point, JTS Geometry (coordinate extraction), and JTS Coordinate — previously only Geometry was handled, requiring callers to always pass a full Geometry object. The $within and $intersects filters now also handle GeoPoint in addition to JTS Geometry, unwrapping to Point via reflection. Replace silent Filter.ALL fallback on failure with DataAccessException so that spatial filter errors (unrecognised type, API mismatch with nitrite-spatial version) surface immediately rather than returning all documents unexpectedly. All spatial class loading remains reflection-based so nitrite-spatial stays an optional dependency with no compile-time coupling from the main module. Add NitriteSpatialSpec covering $near, $within, and $intersects operators against a @SpatialIndex field, null geometry handling, and spatial index creation verification.
Contributor
|
@sfkamath please retarget the 5.1.x branch if you want to take this forward |
Author
|
Done — retargeted onto 5.1.x in #3960 (feature-complete, TCK passing). Closing this one in favour of it. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds micronaut-data-nitrite, an embedded document store module for Micronaut Data, filling the gap that SQLite fills for relational use cases. Targets embedded apps, CLI tools, desktop apps, and integration tests where MongoDB is too heavy.
New module: data-nitrite
Non-nitrite changes
Notes
Test plan