test: schedule WPT variants individually - #65984
Open
panva wants to merge 2 commits into
Open
Conversation
Discover WPT tasks through their existing JavaScript drivers so the Python runner can schedule, report, and rerun generated paths. Assisted-by: Codex Signed-off-by: Filip Skokan <panva.ip@gmail.com>
Collaborator
|
Review requested:
|
avivkeller
approved these changes
Sep 11, 2026
This comment has been minimized.
This comment has been minimized.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #65984 +/- ##
==========================================
+ Coverage 90.17% 90.38% +0.21%
==========================================
Files 771 771
Lines 265451 265487 +36
Branches 50459 50671 +212
==========================================
+ Hits 239361 239951 +590
+ Misses 17059 16487 -572
- Partials 9031 9049 +18 🚀 New features to boost your workflow:
|
jasnell
approved these changes
Sep 11, 2026
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Schedule generated WPT paths through the Python test runner, including query variants that can be validated independently.
Currently, WPT wrappers run serially in the Python runner, but each wrapper can independently launch up to
availableParallelism() - 1specs. Lowering-jtherefore does not limit that internal concurrency.With this change, WPT tasks share the existing Python work queue and each task runs one variant at a time. On our overloaded CI workers, lowering
-jnow also limits WPT concurrency, giving us control over its CPU and memory pressure.This also provides:
Labeled as flaky-testIssues and PRs involving tests that fail intermittently in CI.
given this will either eliminate the flakes or allow us to identify individual specs that flake in Jenkins which so far we weren't able to figure out.
Note that this is not intended to lower the wall-time spent in WPTs, it's meant to have individual WPT specs behave as if they were our individual tests rather than one massive unit (like is the case with e.g. webcrypto)
Assisted-by: Codex
Closes: #51854