docs(samples): keep the manual test fixtures where the samples are - #550
Merged
Conversation
Five documents that existed only in scratch directories while #207, #372 and #393 were being worked on, and got regenerated by hand every time somebody needed to check something. They belong beside `stress-test.md`, which is the same kind of thing. - **`link-test.md`** — the three link cases the editor and the preview treat differently: an `https://` URL, a `file://` URL, and both again as Markdown links. Monaco's link detector recognises exactly `http://`, `https://` and `file://`, and the `file:` branch of its opener is `mainWindow.location.href = href`, which in Tauri means the app's own window. This is the document that showed the app's CSP refuses that navigation (#393). - **`encoding-gbk.md`** and **`encoding-gbk-large.md`** — the same GBK content either side of the 50 KB preview threshold, so the full-read path and the truncated-preview path can both be exercised. The large one is also a reproduction for #547. - **`encoding-shift-jis.md`** — Japanese, so a detector cannot pass by guessing a Chinese encoding. - **`encoding-cp1252.md`** — 359 bytes of French with no CJK in it at all. The point of this one is that #372 is not a Chinese problem: a single `é` is enough to make a file invalid UTF-8, and every "my document opened as ���" report from a European user has the same cause. Each carries a header saying what it is and what should happen, in its own encoding, so opening one in Markpad is itself the instructions. Writing them was its own demonstration: Shift_JIS refused the simplified 样 and an em dash, and CP-1252 refused Chinese outright — which is exactly the `unmappable` refusal `encode_text` raises when someone pastes an emoji into a GBK document. `.gitattributes` gives them `-text -diff`. `-text` opts them out of `* text=auto eol=lf`; line-ending normalisation would not in fact corrupt them — no GBK or Shift_JIS trail byte can be 0x0A or 0x0D — but it is not worth resting on. `-diff` is the separate half and the one that shows: without it git diffs them as text and prints their bytes through a UTF-8 lens, so every review that touches one reads a screen of mojibake.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Five documents that existed only in scratch directories while #207, #372 and
#393 were being worked on, and got regenerated by hand every time somebody
needed to check something. They belong beside
stress-test.md, which is thesame kind of thing — and two open issues now point at them as reproductions.
link-test.mdencoding-gbk.mdencoding-gbk-large.mdencoding-shift-jis.mdencoding-cp1252.mdEach carries a header saying what it is and what should happen, written in
its own encoding, so opening one in Markpad is itself the instructions.
Why the CP-1252 one is worth 359 bytes
#372 reads as a Chinese-and-Japanese problem and is not. A single
éis enoughto make a file invalid UTF-8, and every "my document opened as ���" report from
a European user has the same cause. This fixture is a French note with no CJK
character in it, and it reproduces the bug in full.
Why the large GBK one is twice the threshold
The preview renders the first 50,000 bytes and a background read fetches the
rest. At 101 KB the missing half is most of the document, so a full load that
fails to land is obvious rather than subtle — which is what #547 needs, and it
matches the 121 KB document the symptom was first seen on.
What writing them demonstrated
Shift_JIS refused the simplified 样 and an em dash; CP-1252 refused Chinese
outright. Each had to be written in the language its encoding was built for.
That is the same refusal
encode_textraises when someone pastes an emoji intoa GBK document (#372) — pleasant to have run into it from the other side while
producing the fixtures for it.
.gitattributessamples/encoding-*.mdget-text -diff.-textopts them out of* text=auto eol=lf. Normalisation would not actuallycorrupt them — no GBK or Shift_JIS trail byte can be 0x0A or 0x0D — but it is
not worth resting on.
-diffis the separate half and the one that shows: without it git diffs themas text and prints their bytes through a UTF-8 lens, so every review that
touches one reads a screen of mojibake. With it, "Binary files differ".
Every fixture is verified to decode as the encoding it claims and to be invalid
UTF-8; 926 tests and
npm run checkare unaffected.