sql/jsonpath: preserve boolean results for empty comparisons

[Alignyx]

Problem

The query reported in #145399 returns the JSON string "null" instead of "boolean":

SELECT jsonb_path_query('[1, 2, 3]', '($[*].a > 3).type()');

In the default lax mode, $[*].a selects no items from this input. The comparison has existential semantics: it is true if some selected item satisfies > 3. With no items and no operand-evaluation error, it must be false. Its type is therefore "boolean".

The defect occurs before .type(): the comparison incorrectly produces unknown, which is converted to JSON null. The type method correctly reports the type of that wrong input. JSON null here is not SQL NULL.

Metamorphic relations used to establish the correct result

The investigation compared the original query W with two semantics-preserving transformations, C_materialized and C_guard, on the same database state. It observed the predicate value, its .type(), and is unknown together, rather than checking the displayed type alone.

Both relations below are scoped to lax evaluation, an error-free path operand producing a sequence of scalar terminal values, and the fixed, error-free scalar right operand 3. Incompatible scalar comparisons are retained as genuine-unknown controls. These are not unconditional rewrite rules for strict paths, suppressed evaluation errors, arbitrary nested arrays, or context-dependent paths.

The input matrix was:

CREATE TABLE inputs (id INT PRIMARY KEY, doc JSONB);
INSERT INTO inputs VALUES
  (1, '[1,2,3]'),
  (2, '[{},{}]'),
  (3, '[]'),
  (4, '[{"a":2}]'),
  (5, '[{"a":5}]'),
  (6, '[{"a":"x"}]'),
  (7, '[{"a":null}]'),
  (8, '[{"a":"x"},{"a":5}]');

-- W: original predicate, observed in three ways.
SELECT id,
  jsonb_path_query(doc, '$[*].a > 3') AS predicate,
  jsonb_path_query(doc, '($[*].a > 3).type()') AS kind,
  jsonb_path_query(doc, '($[*].a > 3) is unknown') AS unknown
FROM inputs ORDER BY id;

MR1: materialize and unwrap the same selected sequence

Let S be the scalar sequence selected by $[*].a. Materializing S as a JSON array and then automatically unwrapping that root array in a lax comparison preserves the items and their JSON types. Thus S > 3 must agree with unwrap(materialize(S)) > 3:

WITH seq AS MATERIALIZED (
  SELECT id, jsonb_path_query_array(doc, '$[*].a') AS items
  FROM inputs
)
SELECT id,
  jsonb_path_query(items, '$ > 3') AS predicate,
  jsonb_path_query(items, '($ > 3).type()') AS kind,
  jsonb_path_query(items, '($ > 3) is unknown') AS unknown
FROM seq ORDER BY id;

The direct root $ is important to the diagnosis. An initial probe using $[*] after materialization still reached the faulty nil-empty representation and was rejected as a correct-result oracle. Comparing the root array instead uses existing automatic unwrapping, producing a non-nil empty slice for []. It preserves the intended sequence while exposing the representation-dependent result.

MR2: guard an existential comparison with existence of its operand

Under the conditions above, P(S) = (S > 3) must equal exists(S) && P(S): when S is empty, both are false; when S is nonempty, the guard is true and preserves P, including a genuine unknown result.

SELECT id,
  jsonb_path_query(doc, 'exists($[*].a) && ($[*].a > 3)') AS predicate,
  jsonb_path_query(doc, '(exists($[*].a) && ($[*].a > 3)).type()') AS kind,
  jsonb_path_query(doc, '(exists($[*].a) && ($[*].a > 3)) is unknown') AS unknown
FROM inputs ORDER BY id;

This guard is not an oracle for arbitrary error-producing operands: its short circuit can hide an error. Such cases were tested separately as preservation controls, not counted as equivalent votes.

Observed results

Three complete baseline runs agreed. Both transformed queries agreed on all eight inputs; W disagreed only on the three empty-sequence inputs. Three candidate runs restored agreement. The tuples below are (predicate, type, is unknown); null denotes JSON null.

Inputs	Baseline W	Baseline C_materialized and C_guard	Candidate W and both C queries
IDs 1–3: empty selected sequence	(null, "null", true)	(false, "boolean", false)	(false, "boolean", false)
ID 4: numeric non-match	(false, "boolean", false)	same	same
ID 5: numeric match	(true, "boolean", false)	same	same
ID 6: incompatible string	(null, "null", true)	same	same
ID 7: actual JSON null item	(false, "boolean", false)	same	same
ID 8: incompatible item plus a matching number	(true, "boolean", false)	same	same

The expectation is justified by the existential semantics and the two different equivalent evaluation paths, not merely by choosing the majority output. The issue's PostgreSQL result also agrees with the corrected exact reproducer.

How the root cause was localized

For W and C_guard, EXPLAIN (OPT, VERBOSE) retained the same relational structure; the relevant difference was the JSONPath scalar argument. EXPLAIN ANALYZE showed the same local scan -> project set -> sort pipeline, with eight decoded input rows and eight emitted rows at each operator. The separate SELECT results above establish the value discrepancy; EXPLAIN ANALYZE itself is not a value oracle.

Row-engine and forced-DistSQL-setting runs reproduced the same result matrix. The captured W/C analyzed plans ran on n1, so this evidence does not claim a cross-node execution trace. It localizes the discrepancy to scalar JSONPath evaluation rather than row loss or a different relational operator pipeline. The following branch-level explanation comes from source inspection, not from instrumented branch tracing.

In pkg/util/jsonpath/eval/operation.go, evalPredicate previously treated left == nil or right == nil as unknown, even when the returned error was nil. However, a successful lax path with no matches can legitimately return a nil slice. The original query therefore followed this path:

lax missing member -> valid empty sequence (nil, no error)
                   -> evalPredicate's nil guard returns unknown
                   -> convertFromBool emits JSON null
                   -> type() correctly reports "null"

The materialized-root arm represents the same empty sequence as a non-nil empty slice and reaches the existing false result. The existence-guard arm yields false for the empty selection. This explains both the observed metamorphic violation and its precise source-level divergence.

There is a second constraint: with silent=true, a real evaluation error can also be suppressed into (nil, nil). Simply removing the nil checks would therefore conflate genuine errors with successful empty results and incorrectly change some unknown predicates to false.

Repair and rationale

evalPredicate now copies the evaluation context and disables silent suppression only while evaluating its operands:

operandCtx := *ctx
operandCtx.silent = false

Both operands use this copy. The existing error classification is retained: non-ignorable errors still propagate, and ignorable operand errors produce unknown. When evaluation succeeds, a nil slice is accepted as a valid empty sequence and reaches the existing no-matching-pair result, false. Non-nil empty sequences follow the same semantics.

This is preferable to special-casing .type() because it repairs the predicate value itself and preserves the type of genuinely unknown predicates. It is preferable to merely removing the nil guards because it preserves real-error behavior. Copying the context keeps the enclosing expression's silent mode and existing evaluation state intact; it does not change global silent-mode policy. The eval return-value comment is updated to document the actual ambiguity instead of treating nilness as a reliable failure indicator.

The repair also respects the relevant implementation history:

• 30ce5204 unified comparison and regex predicate evaluation, explaining why the invariant belongs in the shared evaluator.
• PR #144188 separated structural strictness from silent error suppression. The fix preserves that distinction instead of forcing lax mode.
• d374f596 addressed non-nil empty-array results, but a valid nil-empty result can still hit the earlier unknown guard. This repair removes that remaining dependence on slice representation while retaining error information.

Regression coverage and completed local validation

The checked-in regression_145399 section adds the exact reported query, the eight-input value/type/unknown matrix, a right-side-empty comparison, and strict-missing-member, silent arithmetic-error, and actual JSON-null type controls. The new regression was red on the baseline and green on the candidate with unchanged test bytes. Existing expectations in jsonb_path_exists_index_acceleration are corrected for the same shared predicate behavior. No .type() implementation is modified.

The complete MR transformations above were investigation SQL; the committed native regression asserts their established outcomes and preservation controls rather than adding the full MR harness to the repository.

Completed local validation for commit 48ebc1147e82e6174ca522022fcf1dfe53a296b6, based on 8812064a015d2faf99d3fc7e15880f94042954b0, includes:

• Three baseline/candidate matrices, the W/C plan comparison, and row-engine/forced-DistSQL-setting controls.
• Fifteen held-out empty predicates: all six comparisons with empty operands on either side, plus prefix, regex, and two-empty-operand cases; query-first/query-array/match API checks.
• Strict/lax crossed with silent true/false arithmetic-error controls, base scalar types, genuine-unknown method chaining, and unchanged enclosing non-predicate silent suppression.
• Shared-invariant validation with a 26-input degeneration matrix: existence-guard and independent SQL-fold mismatch counts both fell from six to zero. Strict/lax silent-mode mismatch counts remained zero; nested context and valid strict-empty controls also passed.
• All seven JSONPath/jsonb_path_* logic files in local, local-vec-off, fakedist, fakedist-vec-off, and fakedist-disk, including actual RUN/PASS records for this regression in every configuration.
• The complete builtins, pkg/sql/sem/eval, JSONPath parser, and SQL package test targets; SQL ran in 16 shards.

These are completed local results, not a claim that upstream PR CI has passed.

Scope and related issue

This fixes #145399 at its predicate root cause and covers the related empty-operand behavior handled by the same evaluator. It deliberately preserves genuine unknown comparisons, strict structural errors, ignorable-error conversion to unknown, non-ignorable error propagation, and outer silent behavior. It does not redesign .type(), JSON null semantics, or all JSONPath compatibility behavior; the MR assumptions above are not broadened into general optimizer rewrites.

#154589 has the same underlying nil-empty predicate defect, exposed through regex. The two production-file changes and existing acceleration-fixture corrections are identical to that issue's prepared fork repair. This PR carries the shared fix with the dedicated #145399 regression; it is not a second independent implementation, and it does not include #154589's separate new regression section. There is no separate upstream PR for that fork repair at submission time. The shared production patch should be landed once, not as two unrelated fixes.

Resolves: #145399
See also: #154589
Epic: none

Release note (bug fix): Fixed JSONPath comparisons over missing lax members returning JSON null rather than false, which could cause a following type() method to return "null" rather than "boolean".

[blathers-crl]

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}