jsonpath: interpret regex backspace escapes consistently

[Alignyx]

Summary and compatibility decision

This PR fixes the two reported like_regex results in #145270 by interpreting an unescaped JSONPath regex \b as backspace, as PostgreSQL does, instead of RE2's word-boundary assertion. The change is confined to JSONPath regex compilation; ordinary SQL regular expressions retain their existing behavior.

This is intentionally backward-incompatible for existing JSONPath queries that relied on the old word-boundary interpretation. It is a narrow compatibility proposal, not a complete PostgreSQL ARE implementation or a claim that the issue's compatibility concerns are resolved. There is no opt-in, session setting, or version gate in this patch. The questions below ask whether this scoped behavior change is acceptable and how any remaining work should proceed.

Problem

The original queries are:

-- Baseline: three rows. Candidate and the reported PostgreSQL result: one row.
SELECT jsonb_path_query(
  '[null, 1, "a\b", "a\\b", "^a\\b$"]',
  'lax $[*] ? (@ like_regex "a\\b")');

-- Baseline: zero rows. Candidate and the reported PostgreSQL result: one row.
SELECT jsonb_path_query(
  '[null, 1, "a\b", "a\\b", "^a\\b$"]',
  'lax $[*] ? (@ like_regex "^a\\b$")');

Both corrected queries return only the JSON string whose two characters are a and U+0008 (backspace). They do not match the strings containing a literal backslash followed by b. Backspace escapes in character classes also fail regex validation on the baseline, although the corresponding explicit hex form works.

Metamorphic relations and correct-result evidence

The target contract comes from PostgreSQL's character-entry escape definitions. For a nonliteral backspace atom, outside escaped-backslash literals and RE2 quoted spans, these spellings should select the same strings under that contract:

regex \b  ==  regex \x08  ==  a U+0008 character decoded from the JSONPath string

This supplies two transformations of the wrong query W: C_hex replaces the backspace atom with an explicit hex escape; C_unicode encodes the actual character at the JSONPath string layer. It is not an equivalence under CRDB's previous RE2 word-boundary contract, so passing this metamorphic check does not establish backward compatibility. Literal flag q, escaped backslashes, and \Q...\E regions are preservation controls, not interchangeable backspace spellings.

The frozen investigation used this five-row matrix:

CREATE TABLE inputs (id INT PRIMARY KEY, j JSONB);
INSERT INTO inputs VALUES
  (1, 'null'), (2, '1'), (3, '"a\b"'),
  (4, '"a\\b"'), (5, '"^a\\b$"');

-- W: regex backspace spelling.
SELECT id FROM inputs
WHERE jsonb_path_exists(j, '$ ? (@ like_regex "a\\b")')
ORDER BY id;

-- C_hex: explicitly spell the same backspace atom in regex syntax.
SELECT id FROM inputs
WHERE jsonb_path_exists(j, '$ ? (@ like_regex "a\\x08")')
ORDER BY id;

-- C_unicode: let JSONPath string decoding produce the backspace character.
-- chr(92) supplies the backslash of the Unicode escape in the path source.
SELECT id FROM inputs
WHERE jsonb_path_exists(j,
  ('$ ? (@ like_regex "a' || chr(92) || 'u0008")')::JSONPATH)
ORDER BY id;

The C_unicode concatenation above constructs the same Unicode-escaped path text that was written literally in the archived investigation SQL. The same matrix was run with anchors: "^a\\b$", "^a\\x08$", and the Unicode spelling with the decoded U+0008 character between a and $. Three complete baseline runs agreed, and three complete candidate runs agreed:

Query	Baseline selected IDs	Candidate selected IDs
W, unanchored	3, 4, 5	3
C_hex and C_unicode, unanchored	3	3
W, anchored	no rows	3
C_hex and C_unicode, anchored	3	3

Additional held-out checks used strpos, equality, and chr(8) rather than another regex to establish expected matches. Across 12 sample strings, five mismatch counts for unanchored, anchored, repeated-backspace, and quote-boundary cases changed from 2/2/2/2/4 to all zero. Literal, quoted, and explicit-hex control mismatch counts remained zero. This avoids relying solely on agreement between two uses of the same regex engine.

Root cause and how it was localized

EXPLAIN (OPT, VERBOSE) for W and C_hex retained the same relational structure, differing in regex contents. EXPLAIN ANALYZE showed the same five input rows being scanned, with the baseline W filter emitting three rows and C_hex emitting one. After the patch both emitted one. Exact selected values and IDs came from the separate SELECT results above, not from EXPLAIN ANALYZE.

Additional row-engine and forced-DistSQL-setting controls reproduced the unanchored W result for each revision. The captured analyzed plans executed locally on n1: these are not cross-node or mixed-version execution traces. Source inspection supplies the compilation-path explanation; no instrumented branch trace is claimed.

The input JSON strings were already decoded correctly through AsText(). The wrong transition occurs when the decoded regex pattern is compiled using Go regexp semantics:

JSONPath string decoding -> regex pattern containing \b
                        -> Regex.Pattern() returns that pattern unchanged
                        -> RE2 compiles \b as a word boundary
                        -> extra unanchored matches / missing anchored match

Changing JSON decoding would target the wrong layer. The JSONPath parser's regexBinaryOp validates patterns using ReCache.GetRegexpWithFlags; runtime evalRegexFunc uses the same cache/key interface. Both consult jsonpath.Regex.Pattern() when compilation is needed, making it the JSONPath-specific boundary shared by validation and execution.

The relevant history informed this placement: 7cff4e8f introduced like_regex; 26024c41 shared parse-time validation and runtime cache use; and c96c5a45 incorporated flags into the JSONPath cache key and compilation path. This patch preserves that shared boundary and flag separation.

Repair and rationale

Regex.Pattern() now produces a temporary compilation string in which the applicable \b tokens become \x08:

• Return the original pattern immediately in literal q mode or when no \b substring is present.
• Scan complete escape pairs so escaped literal backslashes are not accidentally translated.
• Preserve RE2 \Q...\E spans, including unclosed quoted tails. A quoted span ends at its first literal \E, consistent with the existing engine's handling even when another backslash precedes it.
• Leave other escapes, original pattern text, AST formatting, and the pattern-plus-flags cache key unchanged.

This is preferable to replacing the shared SQL regex engine or changing generic SQL regex keys because it limits the behavior change to JSONPath. It is preferable to a blind string replacement because literal backslashes and quoted patterns must not acquire backspace semantics. Putting the translation in the shared JSONPath key keeps parse-time validation and runtime matching consistent, including character classes that previously failed validation.

Existing regex compilation errors still propagate through the existing paths; parser errors retain the invalid-regex classification. Non-string JSON values are not coerced to strings by this patch. The changes do not add shared mutable state or alter cache locking.

Supported and preserved scenarios

Patterns in this table are regex-level spellings after JSONPath string decoding, unless a JSONPath regex flag is explicitly named.

Scenario	Current behavior and evidence
Ordinary unescaped \b, including a\b and ^a\b$	Fixed: backspace matching; both reported queries and the W/C matrix agree.
Character class [\b] and tested range [\b-\n]	Fixed: compile and match backspace instead of the baseline 2201B errors. This is not a claim that every possible class combination was exhaustively tested.
Repeated backspaces and escapes after quoted regions	Covered by held-out regex-independent checks and lexical boundary tests.
Backspace with case-insensitive flag i	Covered by the native regression; both uppercase and lowercase backspace strings match.
Literal flag q	Preserved: backslash-b remains literal text; it is not translated. A regression uses the same textual pattern with and without q to exercise flag isolation.
Escaped literal backslashes	Preserved by consuming escape pairs; unit tests cover odd/even backslash counts, and SQL tests cover literal matching.
Existing RE2 \Q...\E quoting	Preserved, not presented as PostgreSQL ARE compatibility. Closed/unclosed spans and terminator boundaries have unit/held-out coverage.
Original JSONPath formatting and cache-key identity	Preserved; unit tests verify Pattern() does not mutate formatted AST text.
Ordinary SQL regex operators/functions	Unchanged by this JSONPath-only key. Native ~ controls and held-out regexp_replace controls retain their results.
JSONPath API paths exercised	jsonb_path_query, jsonb_path_exists, and jsonb_path_match are covered by native or investigation SQL using the shared implementation.

Existing JSONPath regex flags i, s, m, and q remain supported; the existing rejection of flag x is unchanged. The targeted new backspace/flag regressions cover i and q; passing the broader flag suites is not a claim of exhaustive new backspace tests for every flag combination.

Not supported, not changed, or not yet established

• Old JSONPath word-boundary semantics for \b: not preserved. Existing expressions can select different rows. There is no compatibility mode, opt-in, or version-gating mechanism in this commit.
• Complete PostgreSQL ARE compatibility: not implemented. Other differences, including \B, remain unchanged. PostgreSQL's official definition of \B is a backslash synonym, not the letter B; this patch does not translate it. Existing RE2 extensions remain available.
• PostgreSQL behavior for ordinary SQL regexes: not added. Their existing RE2 semantics intentionally remain unchanged. This is isolation, not a fix for all regex operations mentioned in the issue discussion.
• Mixed-version/rolling-upgrade compatibility: not established. Old and new binaries interpret affected patterns differently, even though stored text is unchanged. No mixed-binary upgrade test was run. The local/fakedist tests do not prove upgrade safety, and no observed distributed inconsistency or data corruption is claimed.
• User-documentation changes and a completed migration mechanism: not included. The patch contains implementation, build metadata, and tests only. The breaking release note discloses the change but does not itself fulfill the discussion's documentation proposal.
• Release target, backport policy, and approval of the semantic change: not decided by this PR. The local validation does not resolve those compatibility decisions.

Relation to the issue discussion and migration considerations

The issue explicitly raises existing users' reliance on CRDB behavior, discusses major-release changes with clear release notes, and favors documenting the engine difference. Restricting the patch to JSONPath reduces the affected surface but does not eliminate that objection. This PR is not evidence that the proposed exception has already been accepted.

For callers that intend to match an actual backspace, the explicit hex form shown in C_hex above works on both the baseline and candidate. This is also a usable workaround if the existing engine semantics are retained. Literal backslash-b matching continues to use literal q mode or appropriately escaped backslashes.

Callers that intend a word boundary must audit affected JSONPath expressions before adopting this change. Replacing \b with \x08 is not a migration for that intent: it changes the meaning. This PR supplies no general word-boundary migration rewrite or legacy mode. Such a migration/compatibility design, associated user docs, and any release-note expansion remain follow-up work subject to the decision below.

Completed local validation

Validation applies to commit 0fc8460c63ec9a9dcfb74fea881d420076613ec0, directly based on 8812064a015d2faf99d3fc7e15880f94042954b0:

• A 17-case TestRegexBackspacePattern unit test and eight native SQL regression queries cover the reported behavior and preservation boundaries above.
• With unchanged test bytes, the final baseline gate failed on nine unit-pattern expectations and the first original SQL query's three-versus-one result. The SQL harness stopped at that first failure; the anchored zero-row and class/range failures were independently captured by the behavior and held-out runs. The candidate native gate passed the complete new test section and unit target.
• Three baseline and three candidate W/C runs, exact result/plan/row-count checks, independent scalar held-outs, and preservation controls passed their expected gates. The archived verifier and its eight corrupted-evidence rejection self-tests also passed.
• All seven JSONPath/jsonb_path_* logic files passed in local, local-vec-off, fakedist, fakedist-vec-off, and fakedist-disk.
• Complete root JSONPath, JSONPath parser, tree, builtins, pkg/sql/sem/eval, and SQL package test targets passed; SQL ran in 16 shards.
• Repository formatting and whitespace checks passed.

The full metamorphic/held-out investigation harness is not added to the product repository; the committed unit and native SQL regressions capture the selected behavior and preservation boundaries. These are completed local results, not a claim that upstream PR CI has passed or that all ARE compatibility cases have been tested.

Questions for maintainers

1. Is this JSONPath-only \b compatibility change acceptable, despite the remaining backward-compatibility cost, or should jsonpath: incorrect handling of like_regex with escape characters in pattern #145270 instead retain RE2 behavior and be addressed through documentation and explicit-backspace workarounds?
2. If the change is acceptable, should follow-up work add a compatibility/versioning mechanism, and which major-release and migration policy should govern it?
3. Should the next step focus on documentation, migration guidance, and upgrade tests, while deliberately leaving other ARE/RE2 differences out of scope? Or is a broader compatibility design desired before proceeding?
4. If the reported \b mismatch is accepted as resolved, should the broader documentation/compatibility work be tracked separately? The existing commit uses Resolves: #145270 for the reported mismatch; it does not claim that those broader concerns are implemented.

No general ARE translator or compatibility mechanism is proposed as an already completed solution. The explicit-backspace workaround above is validated; the broader alternatives require a design decision and additional implementation/verification.

Resolves: #145270
Epic: none

Release note (backward-incompatible change): JSONPath like_regex now treats an unescaped backslash-b as a backspace character instead of a word boundary, matching PostgreSQL's JSONPath behavior. Existing JSONPath expressions that relied on the previous word-boundary interpretation can return different results. This does not change ordinary SQL regular expression operators/functions, literal q patterns, or escaped backslashes.

[blathers-crl]

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}