Fix spurious interword spaces in CJK/non-space-delimited text output

[XHXIAIEIN]

Problem

When using GetUTF8Text() on CJK (Chinese, Japanese, Korean) or Thai content, Tesseract inserts a space between every recognized word. Since the LSTM engine segments non-space-delimited text into individual characters or short runs as separate WERD_RES objects, the output contains a spurious space after each character:

Input image: 床前明月光          (Li Bai, 静夜思)
Old output:  床 前 明 月 光
New output:  床前明月光

Root cause: IterateAndAppendUTF8TextlineText() in resultiterator.cpp unconditionally inserted one space between every pair of words (numSpaces = (words_appended > 0)), regardless of whether the surrounding scripts use spaces as word delimiters.

Fix

Add a static helper IsWordSpaceDelimited() that returns false when a word's first character:

• Belongs to a non-space-delimited script (Han, Hangul, Hiragana, Katakana, Thai) as reported by unicharset::IsSpaceDelimited(), or
• Falls in a Unicode block where some characters have Script=Common but contextually belong to a non-space-delimited writing system:
  • U+3000–U+303F CJK Symbols and Punctuation
  • U+3040–U+309F Hiragana block (covers U+3099/U+309A Inherited, U+309B/U+309C Common)
  • U+30A0–U+30FF Katakana block (covers U+30FC PROLONGED SOUND MARK, Script=Common)
  • U+FE30–U+FE4F CJK Compatibility Forms
  • U+FF00–U+FF60 Halfwidth and Fullwidth Forms

In IterateAndAppendUTF8TextlineText(), a space is suppressed only when both the previous and the current word are non-space-delimited. Spaces at script boundaries (CJK↔Latin, CJK↔digits) are intentionally preserved.

The preserve_interword_spaces code path is unaffected.

Performance

IsWordSpaceDelimited() is called once per word in the post-processing / output formatting phase (GetUTF8Text), which runs after recognition is complete. It has no effect on the recognition pipeline.

The function is designed with two fast exits:

1. Script check (unicharset::IsSpaceDelimited): O(1) array lookup — returns immediately for Han/Hiragana/Katakana/Thai/Hangul without further work.
2. ASCII fast-path: if the UTF-8 first byte is < 0x80, the character is ASCII (U+0000–U+007F), which is below all checked ranges (≥ U+3000). The UNICHAR decode is skipped entirely, keeping Latin/English text at near-zero overhead.

Only characters that pass both of the above checks (i.e., multi-byte UTF-8, Script=Common) reach the full Unicode range comparison.

Scope

This fix targets GetUTF8Text() plain-text output only. hOCR, TSV, and PDF output are handled by separate code paths and are left for follow-up.

Testing

Both the original and patched binaries were built from the same source tree (v5.5.2) and run against identical input images.

Script	Input	Before (original)	After (patched)	Result
Chinese	床前明月光 (Li Bai, 静夜思)	床 前 明 月 光	床前明月光	✓ spaces removed
Japanese hiragana	はるはあけぼの (Sei Shōnagon, 枕草子)	は る は あ け ぼ の	はるはあけぼの	✓ spaces removed
Japanese katakana	アイスクリーム (two ー U+30FC, Script=Common)	ア イ ス ク リ ー ム	アイスクリーム	✓ spaces removed
Thai	สวัสดีครับ (common greeting)	ส ว ั ส ด ี ค ร ั บ	สวัสดีครับ	✓ spaces removed
Chinese + Latin (script boundary)	开源OCR引擎	开 源 OCR 引 擎	开源 OCR 引擎	✓ boundary space preserved
English phrase	optical character recognition	optical character recognition	optical character recognition	✓ unchanged
English sentence	The quick brown fox jumps over the lazy dog	The quick brown fox jumps over the lazy dog	The quick brown fox jumps over the lazy dog	✓ unchanged