Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[quality] e2e coverage failure reports say 'run null; status unknown' for every error raised after the manifest is read

Chiusa Adatta ai principianti
  • Qualcosa non va?
#1,069 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@hivecommons-hive ci sta già lavorando.

Dal 5/10/2026.

  • #1070 di @hivecommons-hive — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
78/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
github-actions, javascript, node.js
Ambito
ci-cd, testing

Direzione di ricerca

Inizia con il blocco catch in tests/tools/e2e-coverage-report.mjs e i casi CLI in tests/e2e-coverage-report.test.mjs; esegui npm run test:unit:coverage per vedere i test di coverage esistenti. Aggiungi casi per manifest leggibili con esito positivo e negativo che attivano un Error semplice, oltre al caso di un manifest assente o non leggibile. Il lavoro è completato quando i report di errore mantengono runId/runStatus del manifest quando disponibili, conservano lo stato failed/cancelled e continuano a usare campi null quando non è possibile leggere alcun manifest.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

agent/quality hive/covered-by-pr hive/hosted-available-lke648397-260827-5n31 quality testing

Finding

When tests/tools/e2e-coverage-report.mjs fails for any reason after the run
manifest has been read and validated, the failure report it publishes claims it
does not know which run it was rendering — even though it does.

main() builds its fallback report from fields that only CoverageReportError
carries (tests/tools/e2e-coverage-report.mjs:688-696):

    status:
      error.runStatus === 'failed' || error.runStatus === 'cancelled'
        ? error.runStatus
        : 'tooling-error',
    runId: error.runId ?? null,
    runStatus: error.runStatus ?? null,

Only four throw sites construct a CoverageReportError, and all four are the
run-status/capture guards near the top of collectE2ECoverage
(lines 461, 467, 484). Every later failure is a plain Error:

  • coverage run contains incomplete temporary artifacts (line 474)
  • unexpected coverage artifact kind in <entry> (line 490)
  • coverage artifact <entry> belongs to another run (line 493)
  • No src/** coverage was attributable (line 619)
  • every per-script conversion failure — source map escapes build directory
    (162, 166), generated source has no sourceMappingURL comment (178),
    generated source hash mismatch (369), and the map-resolution throws at
    211, 222, 229, 260, 344, 351, 361.

For all of those, error.runId and error.runStatus are undefined, so
renderE2ECoverageReport (line 626) emits:

E2E coverage: tooling-error (run null; status unknown)
Why it matters

.github/workflows/ci.yml:236-249 cats coverage/e2e/report.txt straight into
$GITHUB_STEP_SUMMARY, and that header line is the whole diagnostic. A reader
of a red End-to-end coverage job cannot tell from it whether

  • the Playwright run itself failed and the reporter is faithfully reporting a
    sealed-failed run, or
  • the Playwright run passed and only the reporting step broke.

Those two have opposite remedies, and the artifact erases the distinction at
exactly the moment it is needed. The same null lands in report.json, so
anything consuming the JSON inherits it.

There is a second, smaller consequence: a run sealed failed or cancelled
that then trips one of the plain-Error sites is reported with
status: 'tooling-error' rather than failed/cancelled, because the ternary
above also reads error.runStatus. The report blames the tooling for a test
failure.

Reproduction

At 900592b, render any sealed-passed run whose scripts cannot be attributed
to the local build/ — e.g. a published e2e-coverage artifact unpacked next
to a different build:

$ node tests/tools/e2e-coverage-run.mjs seal --dir "$RUN_DIR" --status passed
{"schemaVersion":1,...,"runId":"local1","status":"passed",...}

$ npm run report:e2e:coverage -- --input "$RUN_DIR" --build build --text out.txt
$ head -1 out.txt
E2E coverage: tooling-error (run null; status unknown)

The manifest on disk says "runId": "local1", "status": "passed". The report
says null / unknown.

Evidence and provenance
  • Unit: npm run test:unit:coverage (TZ=UTC node tests/tools/coverage-report.mjs,
    node v26.10.0), local, revision 900592b — tests/tools/e2e-coverage-report.mjs
    100.00% lines / 90.68% regions. The main() catch block is executed by the
    existing CLI tests, but no test asserts the runId/runStatus it writes for
    a non-CoverageReportError failure, which is why the defect is invisible to
    the suite.
  • Reproduced locally twice at 900592b as shown above, once against a freshly
    sealed local run and once against the e2e-coverage artifact of CI run
    37164361552.
  • This is a reporting-fidelity defect, not a coverage gap: no claim is made
    here about any source path's end-to-end coverage.

Recommendation

Fall back to the run manifest in main()'s catch, so the published failure
report names the run whenever the manifest is readable:

  } catch (error) {
    // Only the run-status guards raise CoverageReportError; every later
    // failure is a plain Error, so re-read the manifest rather than publish a
    // report that cannot say whether the run passed or the tooling broke.
    const run = await readCoverageRun(resolve(options.input)).catch(() => null);
    const runStatus = error.runStatus ?? run?.status ?? null;
    const failure = {
      schemaVersion: 2,
      kind: ERROR_KIND,
      status:
        runStatus === 'failed' || runStatus === 'cancelled'
          ? runStatus
          : 'tooling-error',
      runId: error.runId ?? run?.runId ?? null,
      runStatus,
      ...

readCoverageRun is already imported at line 14, and the .catch(() => null)
keeps an unreadable or absent manifest behaving exactly as it does today.

Cover it in tests/e2e-coverage-report.test.mjs with two CLI cases:

  1. a sealed-passed run that trips a plain-Error site, asserting the written
    report carries that runId and runStatus: 'passed' with
    status: 'tooling-error';
  2. a sealed-failed run that trips the same site, asserting status is
    'failed' rather than 'tooling-error'.

Coordination

The fix lands only in main() (tests/tools/e2e-coverage-report.mjs:674-713).

  • #1040 edits the same file at
    convertScript and inside collectE2ECoverage (script-root resolution); it
    does not touch main() or any throw site.
  • #1051 edits getRegionCoverage
    and the union; it does not touch main().

No open PR claims this function, so the change is disjoint from both.

Completion criteria

  • A failure report for a readable manifest carries the run's real runId and runStatus
  • A run sealed failed/cancelled is reported with that status, not tooling-error
  • An unreadable or absent manifest still yields runId: null / runStatus: null
  • tests/e2e-coverage-report.test.mjs covers all three

Priority

  • Impact: medium (CI's only published e2e diagnostic misattributes test failures to the tooling)
  • Effort: low

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 900592b

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Lingua principale
JavaScript
Stelle
0
Fork
2
Merge medio
22h 21m
PR unite (30g)
404

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di cncf/endusers

Tutte le issue di cncf/endusers

Issue simili

Altre issue su JavaScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.