Skip to content

Conversation

@kulcsaradam
Copy link
Contributor

This is just a demonstration of expanding the testing system of Walrus.
Add tests:

  • regression: random tests
  • malformed: incorrect syntax tests
  • standalone: feature tests (with heavy wasi utilization)
  • proposal_features: features that are only just proposed in some way (for example anyref and array type)

This will be changed to fit the right licenses, it is just a demonstration.

@zherczeg
Copy link
Collaborator

zherczeg commented Nov 8, 2024

Thank you! We can discuss what kind of tests we want to run. For example, malformed tests are testing wabt, do we need them?

@kulcsaradam kulcsaradam force-pushed the more_testing branch 3 times, most recently from e6a4980 to d024da4 Compare November 13, 2024 11:27
@clover2123
Copy link
Collaborator

Thanks for your great contribution!
Please consider the followings:

  • What about merging tools/run-tests-other.py into tools/run-tests.py file?
  • IMO we don't need to run tests for proposed-features

@kulcsaradam
Copy link
Contributor Author

Initially we just wanted the files to be seperated so it's easier to keep track of "other" tests but I will merge it if that is prefered.
Also if proposed-feature tests are not really needed then we can remove them, it was just in case if we implemented any of them then they could be tested instantly.

@kulcsaradam
Copy link
Contributor Author

@clover2123 We have been trying out things in this patch and started to wonder how should the tests be added.

Just to clarify all of these tests have been collected from wamr but most of the complicated programs are open source and we can compile them if we would like to.

But the important thing is:
Should we add tests separately or together as one big patch? (like this)
Which tests should we add? Currently, we are thinking about:

  • Coremark
  • Dhrystone
  • c-ray
    • the problem is that it's more of a demo, only benchmark we can do is measuring the render time
  • regression tests from wamr
    • these come handily because walrus still fails on some of them and they are actual errors most of the time and not Insufficient impport errors
  • unit tests from wamr
    • some of these test optional features that we have not implemented yet

We would not add tests that are still failing though.

Lastly, there are still some tests that we are looking at. Should we contiue looking into them and search for more or should we rather focus on merging these into walrus?

Also, should we maybe introduce a new CI run for more complicated tests that take more time?

@kulcsaradam kulcsaradam force-pushed the more_testing branch 3 times, most recently from 87ba7c0 to 7fff43e Compare December 16, 2025 13:32
@kulcsaradam kulcsaradam marked this pull request as ready for review December 16, 2025 13:50
@kulcsaradam
Copy link
Contributor Author

For license concers, as far as I know dhrystone does not have a definitive license, it is just free to use, but please correct me if I am wrong.

Add a script to build benchmark programs to test walrus
@kulcsaradam kulcsaradam changed the title WIP: Add more tests Add more tests Jan 7, 2026
Copy link
Collaborator

@zherczeg zherczeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ksh8281 ksh8281 merged commit fa7d1ff into Samsung:main Jan 7, 2026
89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants