[fix](function) Reject inconsistent array_sort lambda comparators instead of crashing - #67628
Draft
mrhhsg wants to merge 1 commit into
Draft
[fix](function) Reject inconsistent array_sort lambda comparators instead of crashing#67628mrhhsg wants to merge 1 commit into
mrhhsg wants to merge 1 commit into
Conversation
Member
Author
|
/review |
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
|
Codex automated review failed and did not complete. Error: Your access token could not be refreshed because your refresh token was already used. Please log out and sign in again. Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
…tead of crashing
### What problem does this PR solve?
Issue Number: None
Problem Summary:
`array_sort` hands the user's lambda comparator straight to `std::sort`.
libstdc++'s introsort relies on the comparator being a deterministic strict
weak ordering: its unguarded partition and unguarded insertion loops walk
past the range as soon as that contract is broken. A comparator such as
```sql
SELECT array_sort(
(x, y) -> IF(x > 100 AND y > 100, -1, IF(x < y, -1, IF(x = y, 0, 1))),
[1, ..., 10, 101, ..., 160]);
```
therefore crashes BE with SIGSEGV in `ArraySortFunction::execute` /
`std::__introsort_loop`, and the same happens for a non-deterministic
comparator like `(x, y) -> IF(random() < 0.5, -1, 1)`.
A pre-check cannot fix this: detecting every violation before sorting costs
O(n^2) to O(n^3) lambda evaluations, and any sampled check lets some
comparator through to the unguarded sort. So this PR does two things:
1. Adds `bounded_stable_sort`, a bottom-up merge sort in which every element
access is clamped to `[first, last)` regardless of what the comparator
answers, and uses it in `array_sort`. BE can no longer be taken down by a
comparator. Comparison count is unchanged (O(n log n)), which is what
dominates because every comparison evaluates the lambda.
2. After sorting each array, verifies that no adjacent pair of *different*
elements satisfies `less(next, prev)` (n - 1 extra lambda evaluations). If
the check fails the comparator is not a strict weak ordering and the query
returns `InvalidArgument` with a message explaining the contract, instead
of an unspecified order. A comparator that is only wrong about identical
elements (the common `CASE WHEN x IS NULL THEN -1 WHEN y IS NULL THEN 1 ...`
idiom reports NULL < NULL, and `<=` instead of `<`) is tolerated: the
relative order of identical elements cannot change the sorted output, and
such comparators already appear in existing tests and queries.
As a side effect the sort is now stable: elements the comparator reports as
equal keep their input order.
### Release note
None
### Check List (For Author)
- Test:
- Unit Test: `be/test/util/bounded_stable_sort_test.cpp` covers agreement
with `std::stable_sort` (including stability), the reported inconsistent
comparator, always-true / always-false comparators and a random
comparator, with every index range-checked.
- Regression test: `test_array_sort_lambda_comparator` expects the
strict-weak-ordering error for the reported comparator on literal and
table input, an always-less comparator and a random comparator; checks
that `<=` and NULL < NULL comparators are still accepted; and checks
large / nullable arrays and stability with consistent comparators.
- Behavior changed: Yes. An `array_sort` lambda comparator that is not a
strict weak ordering now fails the query with `InvalidArgument` instead of
crashing BE, and equal elements keep their input order.
- Does this need documentation: No
Claude-Session: https://claude.ai/code/session_016A7UJu7EA7j4NkGz3yjkt6
mrhhsg
force-pushed
the
fix/array-sort-inconsistent-comparator
branch
from
September 7, 2026 16:05
09280e6 to
ca7958c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Problem Summary:
array_sorthands the user's lambda comparator straight tostd::sort.libstdc++'s introsort relies on the comparator being a deterministic strict
weak ordering: its unguarded partition and unguarded insertion loops walk
past the range as soon as that contract is broken. A comparator such as
therefore crashes BE with SIGSEGV in
ArraySortFunction::execute/std::__introsort_loop, and the same happens for a non-deterministiccomparator like
(x, y) -> IF(random() < 0.5, -1, 1).A pre-check cannot fix this: detecting every violation before sorting costs
O(n^2) to O(n^3) lambda evaluations, and any sampled check lets some
comparator through to the unguarded sort. So this PR does two things:
bounded_stable_sort, a bottom-up merge sort in which every elementaccess is clamped to
[first, last)regardless of what the comparatoranswers, and uses it in
array_sort. BE can no longer be taken down by acomparator. Comparison count is unchanged (O(n log n)), which is what
dominates because every comparison evaluates the lambda.
elements satisfies
less(next, prev)(n - 1 extra lambda evaluations). Ifthe check fails the comparator is not a strict weak ordering and the query
returns
InvalidArgumentwith a message explaining the contract, insteadof an unspecified order. A comparator that is only wrong about identical
elements (the common
CASE WHEN x IS NULL THEN -1 WHEN y IS NULL THEN 1 ...idiom reports NULL < NULL, and
<=instead of<) is tolerated: therelative order of identical elements cannot change the sorted output, and
such comparators already appear in existing tests and queries.
As a side effect the sort is now stable: elements the comparator reports as
equal keep their input order.
Release note
None
Check List (For Author)
be/test/util/bounded_stable_sort_test.cppcovers agreementwith
std::stable_sort(including stability), the reported inconsistentcomparator, always-true / always-false comparators and a random
comparator, with every index range-checked.
test_array_sort_lambda_comparatorexpects thestrict-weak-ordering error for the reported comparator on literal and
table input, an always-less comparator and a random comparator; checks
that
<=and NULL < NULL comparators are still accepted; and checkslarge / nullable arrays and stability with consistent comparators.
array_sortlambda comparator that is not astrict weak ordering now fails the query with
InvalidArgumentinstead ofcrashing BE, and equal elements keep their input order.
https://claude.ai/code/session_016A7UJu7EA7j4NkGz3yjkt6