perf: bound repeated work in analytic queries - #28752
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
5222ef2 to
231ec9a
Compare
|
Addressed the P1 on exact head I removed decimal aggregate-prefix reuse entirely, including its separate 32 GiB admission path. The remaining grouping-set rewrite aggregates each set from raw input, so it does not introduce Added two complementary regressions: a planner-shape test that requires decimal |
XuPeng-SH
left a comment
There was a problem hiding this comment.
GPT-6 medium deep review at exact head 58ec680c6de96fc8db0e07a22f0716973a65bb95:
Source review found no verified correctness blocker in the current planner/executor changes. The prior Decimal SUM reassociation issue is closed: finalized-prefix reuse is gone, grouping sets consume raw input, and the added plan-shape plus SQL differential cases cover cancellation, NULL, empty input, and nested prefixes. I also reviewed the CTE/full-drain and scalar-filter proofs, DNF/semi-containment changes, unique hash-join projection, peer-aware RANK Top-N, and v67 compatibility/fallback paths.
P2 — required implementation-acceptance evidence is incomplete. RFC v9 makes these measurements approval gates and requires exact-revision raw artifacts. The PR body reports only three TPC-DS elapsed times; I could not find the fixed TPC-H control results, base-versus-candidate planner p50/p95 time and allocation data, or per-target rows/bytes scanned, peak query memory, and spill bytes. Please attach the exact-revision report (or mark unavailable scale runs as open under the RFC's exception, with its required plan/result/resource artifacts). The exact-head run 34749464335 also still has UT, SCA, coverage, and BVT checks pending, so the mandatory validation set is not terminal yet.
The RFC states that one exact-head APPROVE accepts the design, implementation, and evidence together. I am therefore requesting changes for the missing acceptance evidence, not claiming an unresolved code defect.
What type of PR is this?
Which issue(s) this PR fixes:
Related to #26768.
What this PR does / why we need it:
Large analytic plans can repeat the same CTE/grouping work, miss safe early pruning, project unique hash matches row by row, and retain a full partition for a small
RANKbound. This PR addresses those mechanisms generically:RANK <= Nsemantics by retaining boundary ties, with chunked output, accounted state, and an MORPC v67 compatibility gate.No query ID is encoded in these rules. Statistics participate only in conservative cost/admission decisions, never in semantic legality. Unsupported, uncertain, mixed-version, or over-budget shapes keep the previous plan. The design and counterexample matrix are recorded in the stats-independent analytic rewrite RFC v10 (exact revision).
Measured on the isolated 1T TPC-DS lab with 128 MiB operator spill limits, merge disabled, a 20 GiB no-swap cgroup, and memory/OOM monitoring:
The latest revision removes decimal
SUM(SUM(v))prefix reuse. Checked fixed-width addition is not associative in error semantics: a finer partial sequence can overflow even when the raw coarser input succeeds through cancellation. Dynamic grouping-set expansion remains available because each grouping set consumes raw rows in their original order.Exact-head validation includes the complete
pkg/sql/plantest package,go vet ./pkg/sql/plan, a clean build, and an executed public-SQL differential between shared and legacy grouping-set plans. The regression covers Decimal(38,0) near-bound cancellation in the failing row order, NULL aggregate input, runtime-empty input, and four nested rollup prefixes. CI on the updated head is pending.