Incorrect Unicode Handling in String Diff Functions

[olegmingaleev]

When comparing strings containing complex Unicode characters like emojis with ZWJ sequences, the diff functions produce incorrect results due to improper character segmentation.

🌍 Affected Unicode Characters

Emojis with ZWJ: 👨‍🍳, 👩‍💻, 👨‍🎨, etc.
Multi-byte characters: accented characters, CJK characters
Surrogate pairs: any emoji or character outside BMP
Combining characters: characters with diacritics

Related

Unicode Standard: https://unicode.org/
Intl.Segmenter MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/S

[olegmingaleev]

I've already fixed this issue in #962

@maintainers Could you please review the changes and provide feedback? Thanks! 🙏

[sanshan]

We’re also affected by this issue in production.
In our project (which relies heavily on json-joy diff/patch for collaborative data editing), incorrect Unicode handling causes visible data corruption and inconsistent patch results — especially with emojis, CJK characters, and accented text.

This bug has become a major problem for us, and a proper fix would be extremely valuable.

If there’s any ongoing work or branch addressing this, we’d be glad to help test it.

[streamich]

Fixed in #964