Define merge semantics for `results.errors`

Offen
  • Stimmt etwas nicht?
#93 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
25/100
Issue-Typ
Feature
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
typescript
Bereich
tooling

Rechercherichtung

Start with the spec text that #88 adds for results.errors, then read src/merge.ts in ctrf-cli and in ctrf-js to see how the first shard's fields are kept. The issue first asks for a spec rule (concatenate every input's errors, in input order, with no de-duplication), which needs maintainer agreement before any code. Done means the spec wording is agreed and both merge functions keep every shard's errors, with tests for multi-shard input.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Follow-up to #65, from this comment. Builds on #88, which adds results.errors: [{message, trace, extra}].

Problem

Sharded runs are routinely merged into one CTRF document. Merging is where run-level errors get lost. #88 says nothing about merging. Current CTRF tooling would silently drop errors from every shard but the first:

  • ctrf merge in ctrf-cli keeps extra and environment from the first shard only.
  • ctrf-js merge() keeps the first value for each key and rebuilds the result without any results.errors handling.

So aAshard that failed in global teardown would produce a merged report with no error and, if all its tests passed, a green verdict.

Precedent

Tools that merge run-level errors concatenate them:

De-duplication is also risky: the same message from two shards (for example, "Database cleanup failed") might mean two separate failures.

Proposal

Add to the spec:

When merging CTRF documents (for example, shards of one run), results.errors SHOULD be the concatenation of every input document's results.errors, in input order, without de-duplication. A merging tool MAY record the source document or shard in each entry's extra.

Then update ctrf merge and ctrf-js merge() to follow it.

Related

If an optional run outcome is added (separate issue), its merge rule should follow Playwright's worst-first ordering.

Vorherrschende Sprache
Keine Sprachdaten
Sterne
104
Forks
4
Ø Merge
1 Std. 3 Min.
Gemergte PRs (30 T.)
2

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus ctrf-io/ctrf

Alle Issues in ctrf-io/ctrf

Ähnliche Issues

Weitere Issues zu DevTools

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.