Skip to content

Run browser tests in safari #684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 31, 2025

Conversation

mcbarton
Copy link
Collaborator

@mcbarton mcbarton commented Jul 22, 2025

Description

Please include a summary of changes, motivation and context for this PR.

Fixes # (issue)

Type of change

Please tick all options which are relevant.

  • Bug fix
  • New feature
  • Requires documentation updates

Testing

Please describe the test(s) that you added and ran to verify your changes.

Checklist

  • I have read the contribution guide recently

Copy link

codecov bot commented Jul 22, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.64%. Comparing base (7902b82) to head (1e839a8).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #684   +/-   ##
=======================================
  Coverage   79.64%   79.64%           
=======================================
  Files           9        9           
  Lines        3930     3930           
=======================================
  Hits         3130     3130           
  Misses        800      800           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mcbarton mcbarton force-pushed the enable-safari-testing branch from b7e3db9 to 27adc2d Compare July 23, 2025 08:37
@mcbarton mcbarton changed the title [wip] Run browser tests in safari Run browser tests in safari Jul 23, 2025
@mcbarton mcbarton marked this pull request as ready for review July 23, 2025 08:40
@mcbarton mcbarton requested a review from vgvassilev July 23, 2025 08:46
@mcbarton
Copy link
Collaborator Author

This PR is ready for review. Safari has no headless mode unlike other browsers, so doing automated testing with it is done a little differently to other browsers. I run emrun with the no_browser option, and then use safari driver with selenium to run the webpage we want.

@mcbarton
Copy link
Collaborator Author

I have also enabled running the browser tests in Safari in xeus-cpp here compiler-research/xeus-cpp#368 . This PR is also ready for review.

@mcbarton mcbarton linked an issue Jul 23, 2025 that may be closed by this pull request
@mcbarton mcbarton force-pushed the enable-safari-testing branch 2 times, most recently from 8b0a90f to 2b980f9 Compare July 28, 2025 15:12
@mcbarton
Copy link
Collaborator Author

@anutosh491 @vgvassilev pinging for review. Also please the review the PR adding Safari browser testing in xeus-cpp. We can detect failures just like for other browsers. The only difference between this and other browsers, is we use safari driver and selenium to open the needed webpage, since Safari doesn't have a headless browser mode.

@mcbarton mcbarton requested a review from anutosh491 July 31, 2025 09:44
@mcbarton mcbarton force-pushed the enable-safari-testing branch from 2b980f9 to b836470 Compare July 31, 2025 09:50
@@ -1,3 +1,4 @@
file(COPY ${CMAKE_SOURCE_DIR}/scripts/browser_tests_safari.py DESTINATION ${CMAKE_CURRENT_BINARY_DIR})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to copy this in the build folder?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just so that after I have run the emrun command I don't give a relative path to the python script. Technically you don't need it, but it makes the python command look tidier.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to not copy it since it can cause invalidation issues in development areas if one gets edited.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR has now been updated to use the relative path instead of copying the file. I think I got the relative path correct, but will know for certain once the ci finishes running.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably have some github action magic to give the full path to the source dir.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could make it nicer in the ci using the environment variable for the github workspace, but feels like its best to use relative path there too, given that is what a user will have to do when following the documentation

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got the relative path correct, but forgot to modify the static library Emscripten tests to use thee relative path in the ci. This has been fixed now, and the ci should pass this time.

@mcbarton mcbarton force-pushed the enable-safari-testing branch from b836470 to 608489e Compare July 31, 2025 12:18
@mcbarton mcbarton requested a review from vgvassilev July 31, 2025 18:06
Copy link
Contributor

@vgvassilev vgvassilev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mcbarton mcbarton merged commit b7ab5ae into compiler-research:main Jul 31, 2025
61 of 62 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: Run browser tests in Safari as part of ci
2 participants