Certain Unicode characters not properly matched with fontconfig backend

[ofluffydev]

On Arch Linux with Hyprland, and having installed all of the following fonts:

    ttf-noto
    ttf-linux-libertine
    ttf-unifont
    ttf-arimo-nerd
    ttf-unifont-bin
    noto-fonts
    ttf-dejavu
    ttf-liberation
    gnu-unifont

The characters used in Xilem's transforms example: ✓⨯↶↑↷←→↓ don't have a suitable font selected, and so show a tofu.

See also #parley > Fontconfig backend for fontique - why not FFI?

[DJMcNab]

On my machine, I can see the four directional arrows (↑←→↓), but not the other symbols. Which is quite bizarre, because they're all in the unicode "arrows" block.

[dhardy]

These all now work in Kas (now uses Fontique and mostly the same look-up), though the font used for directional arrows depends on the preceeding character (a quirk of kas-text: it continues to use the previous char's font if possible):

Char '✓'=0x2713 has:
  category OtherSymbol
  block Dingbats
  script Common
  bidi_class ON
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::resolver] select: Script::Zzzz, GenericFamily::Serif, FontWeight(400), FontWidth(256), Normal
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Noto Serif
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Nimbus Roman
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: URW Bookman
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: C059
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Nimbus Roman
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: P052
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Standard Symbols PS
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Caladea
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Symbola
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: STIX
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Lohit Bengali
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Lohit Gujarati
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Lohit Marathi
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Lohit Tamil
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Lohit Kannada
[2025-06-03T08:13:39Z DEBUG kas_text::fonts::library] match: Lohit Telugu
Char '⨯'=0x2a2f has:
  category MathSymbol
  block SupplementalMathematicalOperators
  script Common
  bidi_class ON
Char '↶'=0x21b6 has:
  category OtherSymbol
  block Arrows
  script Common
  bidi_class ON
Char '↑'=0x2191 has:
  category MathSymbol
  block Arrows
  script Common
  bidi_class ON
Char '↷'=0x21b7 has:
  category OtherSymbol
  block Arrows
  script Common
  bidi_class ON
Char '←'=0x2190 has:
  category MathSymbol
  block Arrows
  script Common
  bidi_class ON
Char '→'=0x2192 has:
  category MathSymbol
  block Arrows
  script Common
  bidi_class ON
Char '↓'=0x2193 has:
  category MathSymbol
  block Arrows
  script Common
  bidi_class ON

(The above lists the fonts matching a fontique::Query. All are checked for the required char in order.)

Fontique supports font look-up by script (inferred from text) and language (optionally specified, but usually not and not inferred at the text level). I think here we would need to specify the block and/or category?

On a related note, fullwidth latin characters are not resolved:

Char 'Ａ'=0xff21 has:
  category UppercaseLetter
  block HalfwidthandFullwidthForms
  script Latin
  bidi_class L

This is Latin but not in standard fonts supporting Latin. (Using kas-text's quirk, this does resolve if the prior character is Hiragana like 'あ'=0x3042, possibly using the font Noto Sans CJK JP.)

[dhardy]

Another observation: font matching currently (if I understand correctly) matches all fonts with the specified family and attributes, followed by all fonts with the matching fallback script. I'm not sure if that's what should happen here.

Fonts should have a unicode range property. If the matcher were to specify a block, then the matching fonts could be limited to those which contain this block, thus reducing the number of irrelevant matches.

As for the latter problem of fullwidth latin characters not getting any suitable font matches, perhaps a second level of fallback is needed: if no fonts support the requested block, then check for all fonts which do? But that would likely cause too many matches. (Or perhaps not, since we already know at this point that both family and script specifiers failed to find any fonts covering this block.)

I don't see a Unicode standard on font selection, though there is a mention of "OpenType font selection".

[nicoburns]

Fonts should have a unicode range property. If the matcher were to specify a block, then the matching fonts could be limited to those which contain this block, thus reducing the number of irrelevant matches.

Unicode range matching is also important for Blitz. In Blitz it's a correctness issue as the unicode range can be user-specified (as part of an @font-face directive) and may not match the range of the font (it may for example be smaller than the range actually supported by a font because a different font is preferred for some of the unicode range).

[dhardy]

I was looking at implementing this, but hit a couple of blockers.

First, my suggestion above is essentially adding an (optional) swash::text::Block to a Query. This requires adding a swash dependency to fontique though, so may not be desirable?

A simpler alternative would be to perform this lookup by char like ttf_parser supports.

---

Both approaches require some kind of lookup table, and that leads to another problem. For example, Block::Adlam, corresponds to the Unicode range 1E900–1E95F but this is not listed the Unicode Ranges specification.

Thus, I think our question "does this font support this char" could have the following answers (when using only the Unicode Ranges table):

• Yes (at least, the font advertises that it is functional for this range)
• No (not functional for the range, though it might still cover the char)
• Unknown (not covered by the Unicode Ranges property)
• No OS/2 table (we could probably just return "No" in this case)

---

This leads back to another question: why not directly test whether the char is covered by the font instead?

To allow the shaper to run on the largest inputs possible, we must avoid changing the font face unnecessarily (hence the kas-text quirk mentioned above).

Supporting (enabling) the above, a Query does not return a single match, but rather a set of font faces. For performance, this query result should be cached. Caching this efficiently requires that the query be less specific than the char; currently I use the script, font family and attributes; I'd also like to use the swash::text::Block (or similar) here.