Merging regression suites from engines
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- wasm
- Domain
- testing-qa
Research direction
Start by reviewing the existing .wast regression suites in Wizard and other engines, along with how tests are currently represented in this repository. Check whether candidate mirrored tests accurately represent the Wasm 3.0 specification. Done means the project has an agreed approach for importing and maintaining the combined suites.
Written by the indexing model from the issue text.
Description
In some backroom discussions, I've raised the possibility of putting together the union of all the regression suites that are accumulating in various engines. For example, Wizard has thousands of .wast files that are either new tests, regression tests for old bugs, or expansions of spec tests. Due to the magic of .wast, all such tests pass on the Wasm 3.0 spec interpreter (yay!), so I think these could also be useful to other engines. Similarly, other engines' accumulated regression tests have been useful for Wizard as well.
I wonder if there is appetite for duplicating all of the .wast tests out there into this repo? We could do this manually at first, letting engine maintainers push new tests here via PRs. In the future we could also consider scripts to pull in said tests.
There are maybe some issues to discuss, such as making sure that tests mirrored or duplicated here actually do accurately represent the spec, but I think this could be good for the ecosystem.
Thoughts?
@fitzgen @eqrion @kmiller68 @tlively @cfallin
- Dominant language
- WebAssembly
- Stars
- 249
- Forks
- 65
- Avg merge
- 49m
- Merged PRs (30d)
- 3
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from WebAssembly/testsuite
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
WebAssembly/testsuite#119 · 10 comments ·
All issues in WebAssembly/testsuite
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
openwatersio/maritime-zones#33 ·
Maintainers usually reply within 1 day
-
json_params_matcher fails on falsy top-level JSON primitives (0, False, "")Possibly taken @mayureshsonawane17 claimed this today. OpenWaiting for: Product Owner
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 5 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
[quality] useFocusTrap's Shift+Tab wrap and non-Tab/non-Escape key arms are never driven end to endPossibly taken @hivecommons-hive claimed this today. Openagent/quality hive/covered-by-pr hive/hosted-available-lke648397-260827-5n31 quality testing
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
edrobertsrayne/planner#352 ·
Maintainers usually reply within 1 day