Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Clarify spec intent: common xtra field names, labels vs xtra.traits, stdout/stderr line-splitting

Aperta
  • Qualcosa non va?
#53 7 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Documentazione
Chiarezza
Da chiarire
Stato di attività
Tranquilla
Ambito
documentation, testing-qa

Direzione di ricerca

Inizia esaminando le descrizioni dello schema CTRF per extra, labels, stdout e stderr, quindi confronta le interpretazioni collegate di xunit.v3 e Microsoft.Testing.Platform. Il lavoro è completato quando la specifica definisce chiaramente le chiavi extra comuni, labels rispetto a traits, la suddivisione in righe e se gli array di output vuoti sono presenti o omessi.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Context

We are adding a CTRF reporter to Microsoft.Testing.Platform (the engine behind MSTest, xunit.v3, NUnit, and various other .NET test frameworks). Brad Wilson (xunit) recently published a side-by-side comparison (microsoft/testfx#8903) of his xunit.v3 CTRF emitter vs ours, and several differences boiled down to "the spec doesn't say". Before each ecosystem settles on its own conventions, we would love your guidance on the following so that CTRF documents produced by different .NET reporters are interoperable.

1. Canonical key names for common extra fields

The spec marks environment and per-test as additionalProperties: false for everything except extra, which is the freeform escape hatch. That works, but means two producers reporting the same concept (machine name, exception type, etc.) can pick different keys inside extra, hurting tooling that wants to consume both.

Two concrete examples we hit:

  • Machine / hostname: we emit environment.extra.machine; xunit.v3 emits environment.extra.computer.
  • Exception type name: we emit tests[].extra.exceptionType (e.g. System.InvalidOperationException); xunit.v3 emits tests[].extra.exception.

Are there "recommended" key names for these (and other widely useful fields like build agent, runtime version, framework name)? It would be great either to (a) bless one of the two existing names per concept, or (b) publish a non-normative "common extra keys" appendix so future producers can converge.

2. What is labels for?

The spec says labels is "Structured key-value metadata for the test case (e.g. priority, severity, external identifiers)" typed as an object whose values are string | number | boolean.

We read the e.g. as non-exhaustive and use labels for the test framework's user-defined key/value traits (e.g. Category=Smoke, Owner=alice). xunit.v3 instead puts traits under tests[].extra.traits because Brad reads labels as restricted to the three listed examples (priority/severity/external IDs).

Could you clarify the intent?

  • Option A - labels is the right place for any structured key/value test metadata (our reading).
  • Option B - labels is reserved for a narrow administrative set; arbitrary trait-style metadata should go under extra.
  • Option C - something else (e.g. parameters for data-driven inputs, labels only for the three listed examples, extra.traits for everything else).

We are happy to follow whichever interpretation matches your intent - just want to avoid two .NET CTRF producers diverging on something this central.

3. Are stdout / stderr meant to be one entry per physical line?

The schema describes both as "Standard output lines from test execution", typed as an array of strings. We initially emitted a single multi-line string (one array element with embedded \n); we have since switched to one element per LF-terminated line (with CRLF normalization and the trailing newline not producing an extra empty entry).

Two follow-ups:

  • Is that one-element-per-physical-line interpretation what the spec author intended?
  • For tests that produce no captured output, should the array be present-and-empty ([]), or omitted entirely? We currently omit it; the spec doesn't say.

Thanks for the work on CTRF - it has made a real difference for us standardizing across .NET test frameworks.

Lingua principale
Nessun dato sulla lingua
Stelle
98
Fork
4
Merge medio
1h 21m
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ctrf-io/ctrf

Tutte le issue di ctrf-io/ctrf

Issue simili

Altre issue su Documentation

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.