Skip to content

sql: preserve integer digits in decimal division - #174719

Open
Alignyx wants to merge 1 commit into
cockroachdb:masterfrom
Alignyx:fix-168373-variance-expression-precision
Open

sql: preserve integer digits in decimal division#174719
Alignyx wants to merge 1 commit into
cockroachdb:masterfrom
Alignyx:fix-168373-variance-expression-precision

Conversation

@Alignyx

@Alignyx Alignyx commented Sep 5, 2026

Copy link
Copy Markdown

Fixes #168373.

Root cause

SQL decimal division always used the fixed 20-significant-digit DecimalCtx. In the reported expression, 5845710769898530048144 / 44 requires 21 integer digits, so the exact quotient 132857062952239319276 was rounded to 1.3285706295223931928E+20. Subtracting that rounded quotient from the exact sum left -4, and the final division produced -1/11 even though every input in the group was constant and the variance-equivalent expression must be zero.

How I tracked it down

I reproduced the result for both constants and all three reported groups, then compared the expression with VAR_POP, a translation to zero, and an algebraically equivalent form that performs the division last; all three metamorphic oracles returned zero. Changing only the row count showed the precision boundary: counts 4 and 11 returned zero, while 44 and 101 returned -1/n. Optimizer and distributed plans had the same aggregation levels, and row and vectorized execution failed identically. Inspecting the final scalar render then exposed the rounded quotient and exact -4 residual, locating the problem in the shared decimal division policy rather than aggregation.

Fix

Added tree.DecimalQuo, which first divides with the existing 20-digit context and retries only when an inexact finite result has a positive exponent, adding just enough precision to retain the quotient's integer portion. The helper preserves aliased operands across a retry. Row evaluation, numeric constant resolution, and generated vectorized decimal/integer division now use the same helper. Ordinary fractional results retain the established 20-significant-digit behavior, so this does not raise the global decimal context or change unrelated decimal operations.

Test coverage

  • Added evaluator coverage for the exact large quotient that triggered the bug and for 10/3 retaining the existing 20-digit result.
  • Added the reported aggregate expression for both large constants and all three groups, with VAR_POP as the zero oracle.
  • Verified the regression across five local/fake-distributed row, vectorized, and disk-spilling configurations; ran the full evaluator, tree, and vectorized projection tests and the complete aggregate and decimal logic suites; and confirmed the generated vectorized output is reproducible.

Decimal division previously used a fixed 20-significant-digit context. When
a quotient had more than 20 integer digits, this rounded digits to the left of
the decimal point. A variance-equivalent expression could then subtract that
rounded quotient from an exact sum and return a false non-zero residual.

Select additional division precision only when an inexact result has a
positive exponent, preserving the existing 20-digit behavior for ordinary
quotients. Use the same helper for row evaluation, constant resolution, and
generated vectorized projections.

Add evaluator and SQL regressions for cockroachdb#168373.

Release note (bug fix): Fixed decimal division rounding integer digits in
large quotients, which could produce incorrect results in expressions that
combine exact aggregates with division.
@Alignyx
Alignyx requested a review from a team as a code owner September 5, 2026 01:23
@Alignyx
Alignyx requested review from ZhouXing19 and removed request for a team September 5, 2026 01:23
@blathers-crl

blathers-crl Bot commented Sep 5, 2026

Copy link
Copy Markdown

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl blathers-crl Bot added the O-community Originated from the community label Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

O-community Originated from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql: Incorrect aggregation result for variance-equivalent expression on constant input (returns -1/n instead of 0)

1 participant