Skip to content

Enhanced Queries (Joins & Aggregations) in AWS SDK Java v2#6813

Draft
anasatirbasa wants to merge 1 commit intoaws:masterfrom
anasatirbasa:feature/ddb-enhanced-queries
Draft

Enhanced Queries (Joins & Aggregations) in AWS SDK Java v2#6813
anasatirbasa wants to merge 1 commit intoaws:masterfrom
anasatirbasa:feature/ddb-enhanced-queries

Conversation

@anasatirbasa
Copy link
Contributor

@anasatirbasa anasatirbasa commented Mar 24, 2026

Motivation and Context

Implements the Enhanced Query API for the DynamoDB Enhanced Client, enabling SQL-like join, aggregation, filtering, ordering, and projection capabilities on top of DynamoDB.
Initial ticket: #491

Modifications

New query API surface (query/ package)

  • QueryExpressionBuilder -- fluent builder for constructing enhanced query specifications (single-table, join, aggregation, filtering, ordering, projection, execution mode, limit).
  • QueryExpressionSpec -- immutable query specification produced by the builder.
  • Condition / ConditionEvaluator -- in-memory condition tree (eq, gt, gte, lt, lte, between, contains, beginsWith, and, or, not, group) with recursive evaluation, dot-path nested attribute support, and combined base+joined map view for post-join filters.
  • EnhancedQueryResult / DefaultEnhancedQueryResult -- sync iterable result set.
  • EnhancedQueryRow -- single result row with getItem(alias) and getAggregate(outputName).
  • EnhancedQueryLatencyReport -- timing breakdown (base query, joined lookups, in-memory processing, total).
  • AggregateSpec / OrderBySpec -- aggregation and sort specifications.
  • Enums: AggregationFunction (COUNT, SUM, AVG, MIN, MAX), JoinType (INNER, LEFT, RIGHT, FULL), ExecutionMode (STRICT_KEY_ONLY, ALLOW_SCAN), SortDirection (ASC, DESC).

Query engine (query/engine/ package)

  • QueryExpressionEngine / QueryExpressionAsyncEngine -- sync and async execution interfaces.
  • DefaultQueryExpressionEngine / DefaultQueryExpressionAsyncEngine -- full sync and async implementations handling base-only, join, and join+aggregation paths with parallel joined-table lookups (200-thread executor).
  • QueryEngineSupport -- shared helpers for aggregation bucketing, sorting, and group key construction.
  • JoinedTableObjectMapSyncFetcher / JoinedTableObjectMapAsyncFetcher -- joined-table lookup strategy (PK query, GSI query, or scan fallback based on ExecutionMode).
  • JoinRowAliases -- alias map helpers for base/joined item assembly.
  • AttributeValueConversion -- DynamoDB AttributeValue to Java object conversion.

Client integration

  • DynamoDbEnhancedClient / DynamoDbEnhancedAsyncClient -- added enhancedQuery(spec) and enhancedQuery(spec, latencyConsumer) default methods.
  • DefaultDynamoDbEnhancedClient / DefaultDynamoDbEnhancedAsyncClient -- wired engine construction and enhancedQuery delegation.

Documentation

  • package-info.java -- comprehensive Javadoc for the query package (concepts, filter phases, execution modes, tooling scripts).
  • ENHANCED_QUERY_GUIDE.md -- 23-section playbook covering setup, class reference, query examples with expected output, ExecutionMode deep-dive, condition system deep-dive (leaf/composite types, operator semantics, tree structure, dot-path resolution, 10 input/output examples), execution flow diagrams, build validation, performance tips, complete example, and benchmark guide.
  • EC2_BENCHMARK_INSTRUCTIONS.md -- local and EC2 benchmark setup guide.

Tooling scripts

  • run-enhanced-query-benchmark-local.sh -- local benchmark runner (DynamoDB Local, 1000x1000 dataset, CSV output).
  • run-enhanced-query-tests-and-print-timing.sh -- functional test runner with timing output.

Testing

Unit tests:

  • ConditionTest -- all condition types (eq, gt, gte, lt, lte, between, contains, beginsWith), combinators (and, or, not, group), dot-path nested attribute resolution, null handling, resolveAttribute edge cases.
  • QueryExpressionBuilderTest -- build validation (groupBy without aggregate, filterBase/filterJoined without join, incomplete join config), all builder methods, deprecated method aliases.
  • EnhancedQueryRowTest -- builder, getItem, getAggregate, empty defaults.
  • EnhancedQueryResultTest -- iteration over wrapped rows.
  • EnhancedQueryLatencyReportTest -- constructor and accessor verification.
  • BuildValidationTest -- builder shape validation error messages.
    Functional tests (DynamoDB Local, 1000 customers x 1000 orders):
  • EnhancedQueryJoinSyncTest -- base-only query, INNER/LEFT/RIGHT/FULL joins, filterBase, filterJoined, tree conditions, ExecutionMode (STRICT_KEY_ONLY vs ALLOW_SCAN), scan limit, latency report assertions.
  • EnhancedQueryJoinAsyncTest -- async equivalents of all sync join tests.
  • EnhancedQueryAggregationSyncTest -- GROUP BY + COUNT/SUM/MIN/MAX/AVG, joined aggregation with filters, orderByAggregate, limit, execution mode behavior.
  • EnhancedQueryAggregationAsyncTest -- async equivalents of all sync aggregation tests.
  • NestedAttributeFilteringTest -- dot-path filters (address.city, address.state, address.zip), between, contains, beginsWith, AND/OR combinations, missing path handling.
    Benchmark runner:
  • EnhancedQueryBenchmarkRunner -- 5 scenarios (baseOnly_keyCondition, joinInner_c1, aggregation_groupByCount_c1, aggregation_groupBySum_c1, joinLeft_c1_limit50), warmup + measured iterations, console table and CSV output.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@anasatirbasa anasatirbasa force-pushed the feature/ddb-enhanced-queries branch from c3d0f3b to 373439e Compare March 24, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant