Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[quality] Mutation testing discards the coverage of every script tests/helpers.mjs spawns

Closed
  • Is something wrong?
#1,817 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@hivecommons-hive is already working on this.

Since Oct 23, 2026.

  • #1818 by @hivecommons-hive — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
javascript, node.js
Domain
testing, tooling

Research direction

Start with tests/helpers.mjs and the existing runScriptWithFixtures spawn, then inspect how the tap runner collects Stryker coverage. The issue describes a preload that transfers child coverage and hit counts back to the test process; verify it only runs when globalThis.__stryker__ exists and preserves normal spawns. Confirm with the reported scripts/validate-awards.mjs mutation run: all four mutants should be covered and killed.

Written by the indexing model from the issue text.

Description

agent/quality stage/040-covered-by-pr testing

Finding

runScriptWithFixtures in tests/helpers.mjs mirrors the repo into a temp
sandbox and spawns the script: spawnSync('node', [join(work, 'scripts', scriptName), ...args]).

Stryker's instrumented source records which mutants ran on
globalThis.__stryker__.mutantCoverage, and the tap runner reads that object
out of the test process only (@stryker-mutator/tap-runner/dist/src/setup/hook.cjs
dumps global[namespace] at exit). A spawned script gets a fresh global, so
its counters die with the child. Stryker never sees a mutant in a top-level
scripts/*.mjs file run, so it classifies every one of them as "no coverage"
and never schedules it.

Mutant activation already crosses the boundary — the instrumented prelude
reads __STRYKER_ACTIVE_MUTANT__ from the inherited environment. Only the
counters are lost.

Evidence

Measured on 8abcb3f, npx stryker run --mutate 'scripts/*.mjs' (29 files,
4135 mutants, 29m06s, StrykerJS 10, coverageAnalysis: 'perTest', tap runner):

File                              |  total | covered | # killed | # timeout | # survived | # no cov |
All files                         |  12.34 |   89.38 |      480 |         8 |         58 |     3408 |

3408 of 4135 mutants (82%) were never tested. The split is exactly the
spawn boundary — the five scripts whose tests import them scored normally,
the 24 whose tests go through runScriptWithFixtures scored 0.00 with zero
covered mutants:

imported by its test score spawned via the helper mutants, all "no coverage"
check-duplication.mjs 91.19 validate-members.mjs 406
audit-gate.mjs 40.26 validate-enduser-members.mjs 401
netlify-build.mjs 74.36 generate-members.mjs 348
pr-queue-hygiene.mjs 57.55 collect-metrics.mjs 287
fetch-community-people.mjs 222
…20 more

Unit line coverage is 100% on every one of these files
(npm run test:unit:coverage:check, same revision), so the unit coverprofile
cannot surface this: the suites genuinely exercise the scripts, Stryker just
cannot observe that they did.

Recommendation

Bridge the counters across the spawn, in the harness that owns the spawn:

  1. Add a node -r preload that writes the child's
    globalThis.__stryker__.mutantCoverage (and hitCount) to a file at exit,
    and arms hitLimit/hitCount from STRYKER_HIT_LIMIT first so an
    infinite-loop mutant kills the one child instead of spinning until Stryker
    times out the whole test file.
  2. In runScriptWithFixtures, attach that preload only when
    globalThis.__stryker__ exists — outside a mutation run the spawn must stay
    byte-for-byte as it is — and merge the dumped counters back afterwards. The
    tap runner never sets currentTestId, so the child's static bucket and
    the test process's describe the same test file and add together directly.

Verified against this change: scripts/validate-awards.mjs goes from
0 covered / 4 no coverage / 0.00 to 4 killed / 100.00.

This issue covers the harness bridge only. It does not add scripts/*.mjs
to the mutate set in stryker.config.mjs — that is a separate call with its
own backlog and threshold consequences, and it is filed separately.

Priority

  • Impact: high — it silently voids mutation testing for every data validator
    and collector the CI gate runs, and no existing gate can detect it.
  • Effort: medium

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 8abcb3f

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Dominant language
JavaScript
Stars
0
Forks
3
Avg merge
7h 35m
Merged PRs (30d)
579

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cncf/endusers

All issues in cncf/endusers

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.