feat: add Query.Builder, aggregates, joins, and nested ON

[abnegate]

Why

Appwrite query-lib added aggregates, grouping, joins, extra spatial operators, and a POST /query body so large queries arrays are not capped by URL length. SDKs that already expose Query helpers need those methods, a fluent collector, and a way to nest JOIN ON conditions without leaking SQL.

What changed

• New Query helpers on every SDK that already had Query: aggregates (count, sum, avg, …), groupBy/having/distinct, joins (join/leftJoin/rightJoin/fullOuterJoin/crossJoin), extra spatial (covers, …).
• Fluent collector nested as Query.Builder where the language allows it (C#/Swift/Kotlin/Ruby/Python/TS namespace). PHP is Appwrite\Query\Builder imported as QueryBuilder. Go/Rust use package Builder with NewBuilder() / query::Builder. Dart cannot nest classes and must not export a top-level Builder (Flutter widget clash), so the type is QueryBuilder in lib/query/builder.dart.
• Nested JOIN ON: Query.on(...) plus join overloads that take an alias, an ON query, and extra filters. The wire shape is [alias?, Query.on, filters…].
• Client flatten is internal. There is no public Query.normalize. Flattening is structural (builder / nested query-string lists) and does not special-case the queries parameter name.
• Closed live-spec string enums encoded as type: string plus oneOf of single-value schemas are treated as wire enums. Untitled oneOf members take their name from the parent property.
• PHP from() and generated mocks wrap a scalar example for an array-of-enum field into a one-element list (OAuth prompt in the live spec is a string example on an array field).
• E2E QUERY_HELPER_RESPONSES covers each new helper in every SDK language test.

Raw/union/json/naturalJoin stay unexposed — those are not on the Documents validator allow-list.

Vector helpers were already on main; duplicate copies introduced during the rebase were removed so each language has a single definition.

Why this approach

A client-side JSON collector (not SQL) matches the existing Query string format {method, attribute?, values?}. Nesting the builder under Query keeps the public surface one type. Flattening inside the Client, without a queries key check, means any param that already carries query strings (join ON lists, having, or/and) is handled the same way.

POST /query itself lives in Appwrite (appwrite/appwrite#13300, stacked on #11649). SDK methods will be generated as createQuery once that spec is published.

Verified

• composer lint (phpcs PSR-12)
• composer lint-twig (djLint, 0 errors)
• composer refactor:check (Rector dry-run)
• Twig line length ≤ 1200
• composer test tests/e2e/PHP83Test.php — OK, 1511 assertions (includes array-enum scalar hydration)
• Generated Dart lib/ + tests/tests.dart analyze clean for the Builder/part-of/MockKind failures
• Generated Go query.NewBuilder().Limit(1).Build() compiles and prints the expected limit query

Not verified

• Full sdk-generator language e2e matrix on this HEAD. CI is the authority for that (previous HEAD was red on Dart/Flutter/Go Tests and Dart/Flutter/PHP/CLI Validation).
• Live Appwrite POST /query round-trip from a generated SDK (depends on appwrite#13300 + spec publish).
• CLI compiles in this repo against ../go via a replace in go.mod.twig (generated from the same spec as the CLI). Compiling the published CLI package against the last published Go module is not verified here.

[greptile-apps]

Confidence Score: 4/5

The implementation appears functionally sound, but the new catch-all generation tests violate an explicit repository testing requirement and should be corrected before merging.

Summary

• Adds language-specific Query builder generation and transport normalization.
• Adds aggregate, grouping, join, spatial, vector, and pagination helpers.
• Upgrades OpenAPI parsing to support constrained compound response unions.
• Extends generated-SDK and mock-service coverage across the language matrix.
• Updates CLI-to-Go SDK generation alignment.

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[Copilot]

Pull request overview

Adds cross-language query aggregation, grouping, joins, spatial operators, fluent builders, request flattening, and enum-handling improvements to generated SDKs.

Changes:

• Expands Query APIs and adds fluent builders across supported SDK languages.
• Normalizes nested query inputs in clients and adds broad E2E/unit coverage.
• Improves closed oneOf enum detection and scalar array-enum hydration.

Reviewed changes

Copilot reviewed 110 out of 110 changed files in this pull request and generated 29 comments.

Show a summary per file

File(s)	Description
tests/resources/spec-openapi3.json	Adds closed oneOf enum fixtures.
tests/e2e/Base.php	Adds helper-output and enum hydration assertions.
tests/e2e/languages/{web,node,react-native,deno}/*	Exercises TypeScript query helpers and builders.
tests/e2e/languages/{swift,apple}/Tests.swift	Exercises Swift query helpers.
tests/e2e/languages/{kotlin,android}/Tests.kt	Exercises Kotlin query helpers.
tests/e2e/languages/{go,go-v2}/tests.go	Exercises Go query helpers.
tests/e2e/languages/{dart,flutter}/tests.dart	Exercises Dart query helpers.
tests/e2e/languages/{dotnet,unity}/Tests.cs	Exercises C# query helpers.
tests/e2e/languages/{ruby,python,php}/*	Exercises dynamic-language query helpers.
templates/web/src/{query,client,index}.ts.twig	Adds Query builder/input normalization and exports.
templates/web/src/services/realtime.ts.twig	Normalizes realtime query inputs.
templates/node/src/{client,index}.ts.twig	Adds Node request flattening and exports.
templates/node/test/query.test.js.twig	Tests Node query behavior.
templates/react-native/src/{query,client,index}.ts.twig	Adds React Native query building and flattening.
templates/deno/{mod.ts,src/query.ts,src/client.ts}.twig	Adds Deno query building and flattening.
templates/deno/test/query.test.ts.twig	Tests Deno query behavior.
templates/swift/Sources/Query*.twig	Adds Swift helpers and nested builder.
templates/swift/Sources/Client.swift.twig	Flattens Swift request parameters.
templates/swift/{Tests/Tests.swift,Package.swift}.twig	Adds tests and conditional package targets.
templates/apple/{Sources/Client.swift,Package.swift}.twig	Applies inherited Swift client/package updates.
templates/kotlin/src/main/kotlin/io/appwrite/{Query,Client}.kt.twig	Adds Kotlin query APIs and flattening.
templates/kotlin/src/test/kotlin/io/appwrite/QueryTest.kt.twig	Adds Kotlin Query tests.
templates/android/library/src/main/java/io/package/{Query,Client}.kt.twig	Adds Android query APIs and flattening.
templates/go/query*.twig	Adds Go helpers, builder, and tests.
templates/go/client.go.twig	Flattens Go request values.
templates/rust/src/query/{mod,builder}.rs.twig	Adds Rust Query APIs and builder.
templates/rust/src/client.rs.twig	Normalizes Rust request parameters.
templates/rust/tests/tests.rs	Exercises Rust query helpers.
templates/dotnet/Package/Query*.twig	Adds .NET helpers and nested builder.
templates/dotnet/Package/Client.cs.twig	Flattens .NET request parameters.
templates/php/src/{Query,Client}.php.twig	Adds PHP helpers, builder integration, and flattening.
templates/php/src/Query/Builder.php.twig	Implements the PHP fluent builder.
templates/php/src/Models/{Model,RequestModel}.php.twig	Wraps scalar array-enum values during hydration.
templates/php/tests/QueryTest.php.twig	Tests PHP Query behavior.
templates/python/package/{query,client}.py.twig	Adds Python builder APIs and flattening.
templates/python/test/test_query.py.twig	Tests Python Query behavior.
templates/ruby/lib/container/{query,client}.rb.twig	Adds Ruby helpers and request flattening.
templates/ruby/lib/container/query/builder.rb.twig	Implements the Ruby fluent builder.
templates/dart/lib/{query,package}.dart.twig	Adds Dart Query APIs and builder registration.
templates/dart/lib/query/builder.dart.twig	Implements QueryBuilder.
templates/dart/lib/src/client_mixin.dart.twig	Flattens Dart request parameters.
templates/dart/lib/src/models/{model,request_model}.dart.twig	Handles scalar array-enums and enum serialization.
templates/dart/test/{query_test,src/models/model_test}.dart.twig	Extends Dart Query/model tests.
templates/flutter/lib/{package,src/client_mixin}.dart.twig	Registers builders and flattens Flutter parameters.
templates/skills/*.md.twig	Documents query aggregation, joins, and builders.
templates/cli/internal/typegen/templates/{types,databases}.ts.hbs	Extends typed CLI Query helpers.
templates/cli/internal/generator/typescript.go	Reuses the generated QueryBuilder type.
templates/cli/internal/generator/testdata/*	Updates expected TypeScript output.
templates/cli/go.mod.twig	Redirects CLI builds to a local generated Go SDK.
src/SDK/SDK.php	Improves enum-name fallback resolution.
src/SDK/Language.php	Resolves untitled enum names from parent properties.
src/SDK/Language/{Swift,Apple}.php	Registers Swift builder output.
src/SDK/Language/{Kotlin,Android}.php	Registers Query tests.
src/SDK/Language/{Dart,Flutter}.php	Registers Dart builder output.
src/SDK/Language/{Go,Rust,PHP,Ruby,DotNet}.php	Registers new builder/module templates.
example.php	Generates Go alongside CLI and updates the Go SDK version.

Suppressed comments (2)

templates/cli/internal/typegen/templates/types.ts.hbs:76

• These fixed scalar signatures cannot express the new nested JOIN ON form, and the facade exposes no on helper. Add on plus overloads/unions for (table, on) and (table, alias, on) so the typed callback can construct the nested wire shape supported by Query.

  join: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
  leftJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
  rightJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
  fullOuterJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;

templates/cli/internal/typegen/templates/databases.ts.hbs:44

• The facade delegates only the scalar join form and does not expose Query.on, so typed CLI callbacks cannot use the nested JOIN ON feature added by this PR. Mirror the nested overloads and on helper exposed by Query.

  join: (table, left, right, operator = '=', alias = '') => Query.join(table, left, right, operator, alias),
  leftJoin: (table, left, right, operator = '=', alias = '') => Query.leftJoin(table, left, right, operator, alias),
  rightJoin: (table, left, right, operator = '=', alias = '') => Query.rightJoin(table, left, right, operator, alias),
  fullOuterJoin: (table, left, right, operator = '=', alias = '') => Query.fullOuterJoin(table, left, right, operator, alias),

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[abnegate]

Went through Copilot's 29 threads.

Fixed

• Rust flatten no longer treats numeric nested arrays as query input
• Go flatten uses reflect.DeepEqual (no panic on [][]float64)
• Dart/Flutter flatten spreads [Query.builder()...] into a flat string[]
• Regenerating Rust deletes leftover src/query.rs
• Typed CLI QueryBuilder includes stddev/variance/bitwise aggregates
• Go skill examples match GroupBy([]interface{}) and 5-arg Join
• Skills drop createQuery until the spec has it, and only show build() into the generated string[] APIs

Left as-is

• CLI replace => ../go is for this generator (examples/cli + examples/go). Test-mode CLI omits it. The published CLI repo is a different tree.
• Generated service methods keep wire type string[]. Flattening is internal. Pass builder.build(). QueryInput stays on realtime, where a union already existed. Widening every query param is a separate API change and would still have to avoid a queries-key special case.

[abnegate]

Pushed two commits on top of the flatten-bugfix head:

• Language e2e tests now assert flatten: wrapped builders expand, nested numeric arrays stay nested, already-flat query lists stay lists. That is the coverage Copilot's flatten bugs needed.
• CLI generator testdata includes stddev/variance/bitwise so TestGenerateMatchesBaseline matches the typed QueryBuilder templates.

CLIGo126 / Validation cli (console) were red on the previous head because those baselines still expected groupBy at the old offset.

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

[abnegate]

@greptileai review

Addressed the Tests Mirror Query Internals finding (tests/e2e/Base.php:209-240).

What the assertions pinned before

• 32 lines of the form console.log(Query.sum('price', 'total')), each compared against a hard-coded JSON string. Nothing left the process, so the client, its parameter encoding and the wire were all outside the assertion.
• Three flatten-*:ok labels the harness computed for itself, matched by name in QUERY_FLATTEN_RESPONSES.
• Query.page(2, 10)[0] / [1] and Query.builder().limit(1).build()[0], read by index.
• The .NET and Unity harnesses required Query.Flatten to hand back the same collection instance (object.ReferenceEquals).

What they assert now

Every language harness builds those helpers into the queries parameter of a real general.listRows call and prints the list the mock server echoes back. Base::QUERY_TRANSPORT_RESPONSE marks that line, and Base::assertQueriesReachedTheServer decodes the echo and compares it element by element against QUERY_TRANSPORT_QUERIES. What is pinned is the request the server received, decoded, so key order and whitespace are free to change. Query.page and Query.builder feed the request list instead of being indexed, and the identity check is gone.

Flatten keeps its coverage, on the path a caller can reach

QUERY_FLATTEN_RESPONSES and the labels are gone. Every request now carries its queries array through the client's flatten before transport in all 20 SDKs, so a flatten that mangles a query list turns the suite red everywhere instead of being self-reported. The languages whose service signature accepts a builder (JS/TS, PHP, Python, Ruby) pass an unbuilt Query.builder().limit(1) and let the server prove it was spread. The generated SDKs' own unit tests (node, deno, php, python, dart, go, swift, rust) still assert the helper directly.

Seen red, then green

vendor/bin/phpunit tests/e2e/Node20Test.php:

• Renaming the method Query.sum emits in templates/web/src/query.ts.twig:
  Query 6 did not reach the server as the helper described it, -'method' => 'sum' / +'method' => 'summ'.
• Collapsing the array index for queries in Client.flatten (templates/node/src/client.ts.twig), a pure transport regression the previous in-process shape could not have seen: The SDK did not print the query list the mock server echoes back, it printed: AppwriteException: Invalid queries param ....
• Reordering the keys Query.toString emits, a representation change the server does not care about: still OK (1 test, 193 assertions).
• Both breaks reverted: OK (1 test, 193 assertions).

One real defect fell out of this

templates/deno/src/query.ts.twig called an undefined JSONbig in nestedJoin, so Query.leftJoin(table, alias, [on...]) threw ReferenceError: JSONbig is not defined at runtime. The previous shape could not see it because it never made a request, and the Deno suites are absent from the CI matrix. Fixed here; adding those suites to the matrix is filed separately.

Run locally: Node18, Node20, Node26, WebNode, WebChromium, Deno1303, PHP85, Python313, Ruby27, Ruby31, Go113, Go118, DotNet80, DotNet90, DartStable, FlutterStable, KotlinJava17, Swift61, Rust183, plus --testsuite Generation, vendor/bin/phpcs and vendor/bin/rector process --dry-run.

[abnegate]

@greptileai review

Addressed the Tests Mirror Serialization Internals finding.

The three generated-SDK test files that compared a helper against a literal JSON string now decode the query and assert its fields, so key order, whitespace and number formatting are free to change while method, attribute and values stay pinned:

• templates/node/test/query.test.js.twig — expectQuery(query) parses before asserting; the parameterised fixtures hold real arrays instead of JSON text.
• templates/deno/test/query.test.ts.twig — same, via a parsed() helper.
• templates/python/test/test_query.py.twig — same, via parsed(); expectations are dicts.

The finding also named PHP, Go, Dart, Kotlin, Swift and Rust. Those six already decode the query and assert its fields (json_decode in QueryTest.php.twig, json.Unmarshal in query_test.go.twig, jsonDecode in query_test.dart.twig, JsonParser in QueryTest.kt.twig, JSONSerialization in Tests.swift.twig, to_value() in query/mod.rs.twig), so there was nothing to change there. The Operator tests still compare strings, but this PR does not touch them.

Converting Deno surfaced five expectations that had drifted from the SDK: isNull and isNotNull carried an empty values array, select put its attributes under attribute, and both cursor helpers put their id there too. The Deno unit tests run in no job, so nothing had said so. Corrected to the shape the SDK emits, which is the shape Base::QUERY_HELPER_RESPONSES already pins.

Verified

• node: npx jest test/query.test.js → Tests: 86 passed, 86 total; npx prettier --check test/query.test.js → All matched files use Prettier code style!
• deno: deno test --allow-net --allow-read test/query.test.ts → ok | 1 passed (112 steps) | 0 failed
• python: python -m unittest test.test_query → Ran 54 tests ... OK; python -m black --check test/test_query.py → 1 file would be left unchanged
• Seen red: renaming the method Query.sum emits gives - "method": "sum" / + "method": "summ" on the node sum test, and green once reverted.
• Re-ran Node20, Deno1303, Python313 E2E (193 / 172 / 175 assertions), --testsuite Generation (85 tests, 1130 assertions), vendor/bin/phpcs, vendor/bin/rector process --dry-run, djlint templates/ --lint (0 errors) and the twig line-length script.