Add mutation testing in the CI

Open
#506 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
cpp
Domain
ci-cd, testing

Research direction

Inspect the existing CI configuration and the fuzzing, coverage, and Valgrind jobs first. Choose and document a mutation-testing approach, then verify that a parallel job runs on one OS and compiler and reports how many tests need improvement.

Written by the indexing model from the issue text.

Description

❗️ priority/low 👷 ci 🧪 tests
Is your feature request related to a problem?

I’d like to improve ArkScript test suites. We already have coverage, but that only tells us the code was executed, not that it’s correct.

Describe the solution you would like

To improve our tests, I’d like to add mutation testing (along side the existing fuzzing and coverage).

The idea would be to add a job running in parallel with the valgrind job. The idea is to run mutation testing on a single OS with a single compiler, to report how many tests should be improved.

https://www.youtube.com/watch?v=lhcXAnNgzlo

Is it a big modification to the language? Leave it if you don't know
  • Breaking change
  • Compiler/syntax modifications
  • Runtime modifications (standard library, modules, packages...)
Dominant language
C++
Stars
729
Forks
54
Avg merge
4h 8m
Merged PRs (30d)
2

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ArkScript-lang/Ark

All issues in ArkScript-lang/Ark

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.