Skip to content

Improve accuracy of escape sequences in string and character literals#252

Open
narpfel wants to merge 2 commits into
tree-sitter:masterfrom
narpfel:escape-sequences
Open

Improve accuracy of escape sequences in string and character literals#252
narpfel wants to merge 2 commits into
tree-sitter:masterfrom
narpfel:escape-sequences

Conversation

@narpfel

@narpfel narpfel commented Mar 15, 2025

Copy link
Copy Markdown

There are a few cases where the current escape sequence parser is inaccurate, which causes misleading syntax highlighting.

  • hexadecimal-escape-sequence does not have a maximum length, so '\x00000000a' is the same as '\n'. Godbolt link: https://godbolt.org/z/fq18PbPc4
  • octal-escape-sequence can only contain octal digits [0-7], so "\749" is "<9" and the 9 should not be highlighted as part of the escape sequence. Godbolt link: https://godbolt.org/z/rEcM88had
  • Any other character except the valid escape sequence characters are not allowed in escape sequences, so they should not be highlighted as escape sequences. This PR introduces invalid_escape_sequence so that invalid escape sequences can be highlighted.

narpfel added 2 commits May 16, 2026 15:22
There are a few cases where the current escape sequence parser is inaccurate,
which causes misleading syntax highlighting.

* `hexadecimal-escape-sequence` does not have a maximum length, so `'\x00000000a'`
  is the same as `'\n'`. Godbolt link: https://godbolt.org/z/fq18PbPc4
* `octal-escape-sequence` can only contain octal digits `[0-7]`, so `"\749"`
  is `"<9"` and the `9` should not be highlighted as part of the escape sequence.
  Godbolt link: https://godbolt.org/z/rEcM88had
* Any other character except the valid escape sequence characters are not allowed
  in escape sequences, so they should not be highlighted as escape sequences. This
  PR introduces `invalid_escape_sequence` so that invalid escape sequences can be
  highlighted.
@narpfel narpfel force-pushed the escape-sequences branch from 7482419 to 49fd6a8 Compare May 16, 2026 13:24
@narpfel

narpfel commented May 16, 2026

Copy link
Copy Markdown
Author

Rebased onto current master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant