Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

Clarify spec intent: common xtra field names, labels vs xtra.traits, stdout/stderr line-splitting

オープン
  • 何か問題がありますか？
#53 コメント 7 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
35/100
issue の種類
ドキュメント
明瞭さ
説明が足りない
活発さ
静か
領域
documentation, testing-qa

調査の方向性

まず、extra、labels、stdout、stderr の CTRF スキーマの説明を確認し、次にリンクされている xunit.v3 と Microsoft.Testing.Platform の解釈を比較します。仕様が共通の extra キー、traits に対する labels、行分割、および空の出力配列を含めるのか省略するのかを明確に定義していれば完了です。

索引モデルが issue の本文から書いたものです。

説明

Context

We are adding a CTRF reporter to Microsoft.Testing.Platform (the engine behind MSTest, xunit.v3, NUnit, and various other .NET test frameworks). Brad Wilson (xunit) recently published a side-by-side comparison (microsoft/testfx#8903) of his xunit.v3 CTRF emitter vs ours, and several differences boiled down to "the spec doesn't say". Before each ecosystem settles on its own conventions, we would love your guidance on the following so that CTRF documents produced by different .NET reporters are interoperable.

1. Canonical key names for common extra fields

The spec marks environment and per-test as additionalProperties: false for everything except extra, which is the freeform escape hatch. That works, but means two producers reporting the same concept (machine name, exception type, etc.) can pick different keys inside extra, hurting tooling that wants to consume both.

Two concrete examples we hit:

  • Machine / hostname: we emit environment.extra.machine; xunit.v3 emits environment.extra.computer.
  • Exception type name: we emit tests[].extra.exceptionType (e.g. System.InvalidOperationException); xunit.v3 emits tests[].extra.exception.

Are there "recommended" key names for these (and other widely useful fields like build agent, runtime version, framework name)? It would be great either to (a) bless one of the two existing names per concept, or (b) publish a non-normative "common extra keys" appendix so future producers can converge.

2. What is labels for?

The spec says labels is "Structured key-value metadata for the test case (e.g. priority, severity, external identifiers)" typed as an object whose values are string | number | boolean.

We read the e.g. as non-exhaustive and use labels for the test framework's user-defined key/value traits (e.g. Category=Smoke, Owner=alice). xunit.v3 instead puts traits under tests[].extra.traits because Brad reads labels as restricted to the three listed examples (priority/severity/external IDs).

Could you clarify the intent?

  • Option A - labels is the right place for any structured key/value test metadata (our reading).
  • Option B - labels is reserved for a narrow administrative set; arbitrary trait-style metadata should go under extra.
  • Option C - something else (e.g. parameters for data-driven inputs, labels only for the three listed examples, extra.traits for everything else).

We are happy to follow whichever interpretation matches your intent - just want to avoid two .NET CTRF producers diverging on something this central.

3. Are stdout / stderr meant to be one entry per physical line?

The schema describes both as "Standard output lines from test execution", typed as an array of strings. We initially emitted a single multi-line string (one array element with embedded \n); we have since switched to one element per LF-terminated line (with CRLF normalization and the trailing newline not producing an extra empty entry).

Two follow-ups:

  • Is that one-element-per-physical-line interpretation what the spec author intended?
  • For tests that produce no captured output, should the array be present-and-empty ([]), or omitted entirely? We currently omit it; the spec doesn't say.

Thanks for the work on CTRF - it has made a real difference for us standardizing across .NET test frameworks.

主要言語
言語のデータがありません
スター
99
フォーク
4
平均マージ
1時間 21分
マージ済み PR（30日）
1

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

ctrf-io/ctrf のほかの issue

ctrf-io/ctrf の issue をすべて見る

似ている issue

Documentation の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。