[p5.js 2.0 Bug Report]: That one font test is flaky

[davepagurek]

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

p5.js version

2.x

Web browser and version

Chrome

Operating system

Github action runner

Steps to reproduce this

If you've submitted a PR before, you may have seen this visual test fail:

p5.js/test/unit/visual/cases/typography.js

Lines 79 to 94 in 60ef55e

	visualTest('can control non-variable fonts', async function (p5, screenshot) {
	p5.createCanvas(100, 100);
	const font = await p5.loadFont(
	'https://fonts.googleapis.com/css2?family=Sniglet:wght@400;800&display=swap'
	);
	for (const weight of [400, 800]) {
	p5.background(255);
	p5.textFont(font);
	p5.textAlign(p5.LEFT, p5.TOP);
	p5.textSize(35);
	p5.textWeight(weight);
	p5.text('p5*js', 0, 10, p5.width);
	screenshot();
	}
	});

When it fails, we see it just using the default serif font rather than the Google Font.

Often just rerunning the CI jobs causes it to work. It'd be great to figure out exactly why this is flaky and see what we can do about it.

My current theory is this: when we load a font via a CSS file (as we are when we load a Google Font), there are potentially a lot of different font files referenced in it, and we pick what we think the most relevant one is to load. However, we also attach the whole CSS file to the page, so the other font options are still accessible to the sketch. But potentially the browser isn't adding those other fonts to document.fonts reliably until they are actually requested for use by something in the DOM or on the canvas, so when we immediately try to render using it, it's not present, but might be present if we waited for it to load and then rerendered. The main problem with this theory: why does rerunning the tests work, then, when it should be running in a fresh container with no history of the last run? Does it just happen to load fast enough sometimes?

There are two potential things we can do about this:

• Identify and fix the underlying cause of the flakiness and make it awaitable
• Figure out another way to test font weight usage in CSS fonts without this flakiness

[Harsh16gupta]

Hi, I'd like to take this on. Please assign it to me.

[davepagurek]

Thanks @Harsh16gupta! Before assigning this one to someone, we need to do a bit of research to figure out specifics on how to proceed -- do feel free to start looking into it though! Keep us updated in the comments here about what you find and what you would do to address the issue, and once we figure out a path forward we can then move to implementation.

[Prince-ES]

HI @davepagurek , I looked into the issue and found that:

When loading a Google Font via a CSS file, multiple font-face rules are included.
P5.loadFont() loads only the first matching font-face block, but the browser may not fetch all the weights (400, 800) immediately.
Even with await, the browser may still load the CSS before the actual font files are ready.

How we could approach fixing it:-

Wait for document.fonts.ready or document.fonts.load() to confirm speciic weights are available before running test.
Replace google fonts with a local test font so CI doesn't depend on external network requests.
Or maybe render twice - first load the font, then force a rerender after a delay to ensure weight availability.

If this direction seems right, I'd like to work on this issue and propose a PR.
Let me know if you would like me to take ownership of the issue.

[Ankit-raj-11]

Hi @davepagurek ,

I’ve looked into the flaky font test and found it’s likely caused by the font not being fully loaded before rendering in CI, making tests unreliable. To fix this, we should wait for document.fonts.ready before taking screenshots, and consider preloading the font or hosting it locally to avoid network issues. If you’re open to it, I’d love to work on a PR to address this!

[davepagurek]

We actually already await document.fonts.ready here:

p5.js/src/type/p5.Font.js

Line 1065 in 60ef55e

await document.fonts.ready;

I also tried waiting for document.fonts.load in my WIP WebGPU mode PR while working on font support there, but that doesn't help this test's case -- again possibly because it isn't loading all the weights. We could try an experiment and see if loading every weight here works? That may not be what we want long term though because that's a lot of upfront loading for fonts that may not get used. Lazy loading is likely the best default behaviour here. But possibly we could add a method to p5.Font like font.waitFor(style) so we could do await that manually in the test and pass in a CSS string with the weight we want? (I don't actually know 100% that this is the problem though, it's worth testing this out to see if it makes a difference.)

[Homaid]

Hii @davepagurek I tested the hypothesis by adding explicit document.fonts.load() calls but this caused the test to timeout (1000ms), which suggests the fonts aren't available in document.fonts at all after the CSS is loaded.This seems to confirm the theory the browser doesn't add all font weights to document.fonts just from loading the CSS file. They only become available
when actually used by the DOM/canvas.So the issue is: we can't await fonts that the browser hasn't registered yet.

[davepagurek]

Thanks @Homaid, what did you pass in to load()? My initial test didn't specify a given weight so it made sense that that wouldn't necessarily work. Did you try requesting the weights that the test was about to use?

[Homaid]

Yes, I specified the exact weights and size that the test uses

await document.fonts.load('400 35px Sniglet');
await document.fonts.load('800 35px Sniglet');

Both calls timed out at 1000ms that means even though the the CSS is loaded, the specific font weights (400 and 800) aren't being registered in document.fonts,so there's nothing for fonts.load() to load.It seems the browser only registers these fonts when they're actually rendered what do you think @davepagurek

[davepagurek]

If you manually add something to the DOM first, like a <span style="font-family: Sniglet; font-weight: 400">Test</span>, does that work? If that works, going on the idea that we can make a p5.font.waitFor(...) function, that could first add something like that to the DOM (maybe with display: none or something to hide it), then await, then remove it

[Homaid]

So adding DOM elements with the font styles first worked! The fonts.load() calls completed successfully after the spans were added to the DOM. so i think we can implement your idea!!! @davepagurek

[davepagurek]

Thanks for testing that out @Homaid! Are you interested in making that function and updating the test?

[dhowe]

nice find - have we tested this on other browsers?