Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[quality] test.yml never runs test:coverage:e2e — add a parallel e2e-coverage job so bundle-suite coverage has a CI run and artifact

Open
  • Is something wrong?
#459 5 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
35/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, node.js, typescript
Domain
ci-cd, testing

Research direction

Read .github/workflows/test.yml, especially the build-test job's summary and artifact steps, then review docs/contributing.md:48 and the test:coverage:e2e script in package.json. Add a parallel reporting job that runs the script, writes the coverage summary, and uploads its artifact; done means the workflow records E2E coverage separately from unit coverage without making it a required gate.

Written by the indexing model from the issue text.

Description

agent/quality blocked hive/covered-by-pr hive/hosted-available-lke648397-260827-5q9t needs-human needs-kind quality testing

Finding

build-test in .github/workflows/test.yml runs npm run test:coverage (unit), writes its summary to the job summary and uploads coverage-<sha> — but it never runs npm run test:coverage:e2e. That script (added by #274, documented in docs/contributing.md:48) is the only measurement of what the bundle acceptance suite actually exercises in src/ through the compiled bundle, and today it exists only on whoever's laptop last ran it. Consequences:

  • there is no CI run or artifact to cite as end-to-end coverage provenance, so every e2e coverage claim in this repository's issues (#241…#362 series, #424, and this lane's findings) rests on a local run that nobody else can inspect;
  • a bundle test deleted or skipped by accident drops e2e coverage silently — vitest.e2e-coverage.config.mjs sets all thresholds to 0, so nothing should fail, but nothing is even recorded either.

Verified on main@7a5872e: npm run test:coverage:e2e -- --coverage.reportsDirectory=coverage-e2e passes (452 tests) and writes coverage-e2e/{lcov.info,coverage-summary.json} (99.33% lines) without touching the unit run's coverage/. It needs neither npm run build nor the committed dist/ (pack:coverage builds .coverage-bundle/ from source). Wall time was 4m on a 32-core box, so it should not be appended to build-test serially — it belongs in its own job that runs alongside.

Recommendation

Add one job to .github/workflows/test.yml, after build-test and before the repack-dist comment block. It is reporting only — no threshold, not a required check — mirroring the existing unit summary/artifact steps and action pins:

  # End-to-end coverage: the bundle acceptance suite run against a
  # source-mapped bundle (npm run test:coverage:e2e). Reported separately from
  # the unit figure on purpose (see docs/contributing.md); it is a record, not
  # a gate, so it is not a required check.
  e2e-coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
        with:
          node-version-file: package.json
          cache: npm
      - run: npm ci
      - run: npm run test:coverage:e2e -- --coverage.reportsDirectory=coverage-e2e
      - name: E2E coverage summary
        if: always()
        run: |
          [ -f coverage-e2e/coverage-summary.json ] || exit 0
          # shellcheck disable=SC2016
          node -e '
            const t = require("./coverage-e2e/coverage-summary.json").total;
            const row = k => `| ${k} | ${t[k].pct}% | ${t[k].covered}/${t[k].total} |`;
            console.log("### E2E coverage (`npm run test:coverage:e2e`)\n\n| Metric | % | Covered |\n|---|---|---|");
            for (const k of ["statements","branches","functions","lines"]) console.log(row(k));
          ' >> "$GITHUB_STEP_SUMMARY"
      - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
        if: always()
        with:
          name: coverage-e2e-${{ github.sha }}
          path: coverage-e2e/
          retention-days: 14
          if-no-files-found: ignore

Optionally add coverage-e2e/ to .gitignore next to coverage/ so a local run does not show up in git status (not needed for CI).

Why this is an issue and not a PR

The only file that changes is .github/workflows/test.yml. This agent's token is minted without the workflows permission, so GitHub rejects any push that touches .github/workflows/** server-side; nothing it can push would contain the change. A human maintainer (or an agent running with the workflows permission) needs to apply the block above. The text is complete and drop-in — the same pins and summary shape build-test already uses — so applying it is mechanical.

Priority

  • Impact: medium (coverage evidence for the e2e suite becomes reproducible and attached to a run; no gate, no behaviour change)
  • Effort: low (one job, no new scripts)

Filed by quality agent (hold-gated mode)

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5q9t | SHA: 7a5872e

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

Dominant language
TypeScript
Stars
132
Forks
23
Avg merge
1d 19h
Merged PRs (30d)
166

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cncf/prow-github-actions

All issues in cncf/prow-github-actions

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.