Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Clarify identity semantics for parameterized tests and merged retries

未关闭
  • 哪里不对？
#66 4 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
文档
描述清晰度
需要澄清
活跃度
活跃
领域
documentation, testing-qa

调研方向

Start with the identity model described in the issue, then read PRs #57 and #62 and the referenced MTP files: CtrfReportEngine.JsonTestWriter.cs and UnitTestElementExtensions.cs. The work is done when the specification documents a normative model for parameterized tests and merged retries, including one worked example and guidance MTP can implement.

由索引模型根据 Issue 内容生成。

描述

Producer context

Microsoft.Testing.Platform (MTP) produces CTRF for multiple test frameworks, so it cannot assume every framework assigns the same stability semantics to its native test UID.

Today the MTP CTRF reporter keeps the framework-provided UID under tests[].extra.uid rather than claiming it is a CTRF testId:

For MSTest specifically, that UID is a versioned hash of the assembly file name, fully-qualified test name, parameter types, and (for data-driven tests) the row index:

This is stable for many rebuilds, but a rename, assembly rename, or reordered dynamic data source can change it.

What the current spec resolves

The identity model is clear that:

  • testId identifies a logical test case and should be stable across runs within the producer's documented scope;
  • executionId is unique for one execution of that logical test within a run;
  • attemptId identifies an individual entry in retryAttempts;
  • identity fields are optional and opaque to consumers.

PR #57 and PR #62 provide useful general guidance, but we could not find guidance for parameterized cases or for reports transformed by retry merging.

Questions

  1. For a parameterized/data-driven method, is each argument row a distinct logical test case that should have its own testId, or should all rows share the method's testId and be distinguished by parameters plus executionId?
  2. If rows are distinct test cases, should a generated testId be based on canonical parameter values (stable across data-source reordering) rather than a row index? How should producers handle values that cannot be serialized deterministically or safely?
  3. Is a source rename/file move intentionally a new testId, or is preserving identity across renames an expected producer responsibility?
  4. For retries executed as separate processes and later collapsed into one CTRF test object, should the raw per-process documents and the merged test share one executionId, or should each raw document have its own execution identity and the merged document get another one?
  5. Is attemptId intentionally limited to previous attempts in retryAttempts? The final attempt is represented by the test object, which has no attemptId; that makes it difficult to correlate the final attempt between a raw per-process report and a merged report.

Why this matters for MTP

Mapping an arbitrary framework UID to testId may overstate its stability, while omitting standardized identities prevents CTRF consumers from correlating parameterized rows and retry artifacts. We would like to document and implement the intended model rather than establish a .NET-specific convention.

Possible interpretations include treating every parameter row as a logical test case, or treating the method as the logical test and the row as an execution. We are not prescribing either; a normative recommendation and one worked parameterized/retry example would unblock the mapping.

主要语言
没有语言数据
星标
100
派生
4
平均合并
1 小时 3 分钟
30 天内合并 PR
2

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

ctrf-io/ctrf 的其他 Issue

查看 ctrf-io/ctrf 的全部 Issue

相似的 Issue

更多 Documentation Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。