Run test262 tests in CI #408
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nickva
force-pushed
the
run-test262-in-ci
branch
12 times, most recently
from
8ee7286 to
24e9e37
Compare
nickva
force-pushed
the
run-test262-in-ci
branch
4 times, most recently
from
9c0da0d to
6960dd5
Compare
nickva
force-pushed
the
run-test262-in-ci
branch
2 times, most recently
from
ab51af9 to
7ca7dc7
Compare
nickva
force-pushed
the
run-test262-in-ci
branch
2 times, most recently
from
4a07bd7 to
44d52ea
Compare
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 15, 2025
Test262 are 70k+ Javascript conformance tests [1]. QuickJS has its own test runner but unfortunately there is not automatic bootstraping and they run by hand. There is an upstream PR [2] to add these tests to run there but until that happens we can patch our own QuickJS with. This particular commit doesn't run the tests just patches the necessary updates for bootstrapping and some timeout tweaks to fix some flakes. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 15, 2025
Test262 are 70k+ Javascript conformance tests [1]. There is an upstream PR [2] to run these in CI there but until then can patch our own QuickJS and run them in our CI. One of the patches we have `04-test262-errors.patch` patches one of the exception we expect to get since modified the parser to accept the `function` expression syntax. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 15, 2025
Test262 are 70k+ Javascript conformance tests [1]. There is an upstream PR [2] to run these in CI there but until then can patch our own QuickJS and run them in our CI. One of the patches we have `04-test262-errors.patch` patches one of the exception we expect to get since modified the parser to accept the `function` expression syntax. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 15, 2025
Test262 are 70k+ Javascript conformance tests [1]. There is an upstream PR [2] to run these in CI there but until then can patch our own QuickJS and run them in our CI. One of the patches we have `04-test262-errors.patch` patches one of the exception we expect to get since modified the parser to accept the `function` expression syntax. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
force-pushed
the
run-test262-in-ci
branch
2 times, most recently
from
0ced3d2 to
17475ee
Compare
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 20, 2025
Test262 are 70k+ Javascript conformance tests [1]. There is an upstream PR [2] to run these in CI there but until then we can patch our own QuickJS and run them in our CI. One of the patches we have `04-test262-errors.patch` patches one of the exception we expect to get since modified the parser to accept the `function` expression syntax. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 20, 2025
Test262 are 70k+ Javascript conformance tests [1]. There is an upstream PR [2] to run these in CI there but until then we can patch our own QuickJS and run them in our CI. One of the patches we have `04-test262-errors.patch` patches one of the exception we expect to get since modified the parser to accept the `function` expression syntax. For now run the test in the full CI only for Linux and skip s390x. Some architectures/os combinations show various discrepancies around TypedArrays or rounding errors. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 21, 2025
Test262 are 70k+ Javascript conformance tests [1]. There is an upstream PR [2] to run these in CI there but until then we can patch our own QuickJS and run them in our CI. One of the patches we have `04-test262-errors.patch` patches one of the exception we expect to get since modified the parser to accept the `function` expression syntax. For now run the test in the full CI only for Linux and skip s390x. Some architectures/os combinations show various discrepancies around TypedArrays or rounding errors. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
added a commit
to apache/couchdb
that referenced
this pull request
Jun 21, 2025
Test262 are 70k+ Javascript conformance tests [1]. There is an upstream PR [2] to run these in CI there but until then we can patch our own QuickJS and run them in our CI. One of the patches we have `04-test262-errors.patch` patches one of the exception we expect to get since modified the parser to accept the `function` expression syntax. For now run the test in the full CI only for Linux and skip s390x. Some architectures/os combinations show various discrepancies around TypedArrays or rounding errors. [1] https://github.com/tc39/test262 [2] bellard/quickjs#408
nickva
force-pushed
the
run-test262-in-ci
branch
from
17475ee to
2b07a0b
Compare
Run test262 tests for Linux, macOS and Cosmopolitan Add a Makefile `test2-bootstrap` helper to clone and patch test262 tests at a particular commit. Running with the latest commit noticed a few tests were failing and added them to the errors list. Noticed a few flaky staging math tests (acosh and cbrt) and made a PR to test262 to fix those. The CI environment, especially for macOS, was a bit more unpredictable and was failing in atomics tests due to lower timeouts in the `atomicsHelper` so double the timeouts. The time didn't worsen too much. It still takes about 2-3 min.
nickva
force-pushed
the
run-test262-in-ci
branch
from
2b07a0b to
11325a5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Run test262 tests for Linux, macOS and Cosmopolitan
Add a Makefile
test2-bootstraphelper to clone and patch test262 tests at a particular commit. Running with the latest commit noticed a few tests were failing and added them to the errors list.Noticed a few flaky staging math tests (acosh and cbrt) and made a PR to test262 to fix those.
The CI environment, especially for macOS, was a bit more unpredictable and was failing in atomics tests due to lower timeouts in the
atomicsHelperso double the timeouts. The time didn't worsen too much. It still takes about 2-3 min.