scripts: the `issue_workspace` exact-match test fails when a fixture commit's SHA contains 570 (flaky NVX CLI)
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 1/5
- 見積もり時間
- 1時間未満
- 初心者へのやさしさ
- 90/100
調査の方向性
scripts/test_issue_workspace.pyを読み、特にIssueWorkspaceTests.test_find_groups_exact_issue_matches_from_every_sourceと、283行目付近のアサーションを確認してください。アサーションを変更し、単独のissue番号ではなく、おとりのブランチ名を確認するようにしてから、そのunittestを実行してください。issueでは日付を固定した再現手順が説明されており、おとりのブランチとpull requestのチェックが引き続きカバーされることを想定しています。
索引モデルが issue の本文から書いたものです。
説明
Summary
scripts.test_issue_workspace.IssueWorkspaceTests.test_find_groups_exact_issue_matches_from_every_source fails whenever the 12-character SHA prefix of either commit that its fixture creates contains 570. Each time, it fails with AssertionError: True is not false at self.assertFalse(any("570" in name for name in names)), and nothing else in the job fails. In the CI runs created from 2026-11-06 to 2026-11-16, it failed 6 of the 1,554 NVX CLI jobs that completed, 4 on Linux and 2 on Windows. That is about 1 in 260, and 6 of the 21 NVX CLI jobs that failed.
The test checks that issue_workspace.find_issue, when it looks for issue 57, reports none of the fixture's decoys for issue 570. It does so by searching every candidate's name for 570. But the tool names a commit candidate after its SHA prefix, and the fixture's commit SHAs change with the time at which the test runs. A prefix contains 570 with a probability of about 0.24%, so with two commits the test fails about once in 205 runs. Pinning the fixture's commit dates reproduces the failure every time. The tool excludes the decoys correctly, so the defect is in the test.
Incident
- Workflow: CI run 43902030265, attempt 1,
pushtofeature/nvx-refactor(#1651) at263d9ee8dff. - Failed job:
NVX CLI / Windows, stepValidate NVX CLI. It failed in thepython -m unittestcommand of thevalidate-nvxaction's Windows step (action.yml#L25-L34). Of 2,213 tests, 87 were skipped, and this test was the only failure. The run'sNVX CLI / Linuxjob failed for an unrelated reason, which2b8b51581aafixed. - Runner: GitHub-hosted, image
windows-2025-vs202620261103.298.1, runner 2.338.2, Python 3.12.10. - The test and
scripts/issue_workspace.pyare byte-identical at all six failing heads and ondevat28a0c326f5e.
Decisive output:
======================================================================
FAIL: test_find_groups_exact_issue_matches_from_every_source (scripts.test_issue_workspace.IssueWorkspaceTests.test_find_groups_exact_issue_matches_from_every_source)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\a\nvx\nvx\scripts\test_issue_workspace.py", line 283, in test_find_groups_exact_issue_matches_from_every_source
self.assertFalse(any("570" in name for name in names))
AssertionError: True is not false
----------------------------------------------------------------------
Ran 2213 tests in 679.383s
FAILED (failures=1, skipped=87)
History
I classified the failed NVX CLI jobs of the 1,172 CI runs created from 2026-11-06 to 2026-11-16 18:43 UTC:
- 783
NVX CLI / Linuxjobs completed. 14 failed, 4 of them on this assertion. - 771
NVX CLI / Windowsjobs completed. 7 failed, 2 of them on this assertion. - In each of the 6, this assertion was the job's only failure.
- The window starts before #1337 added the assertion on 2026-11-07, so 6 in 1,554 slightly understates the rate.
| Created (UTC) | Run | Event | Head | Job | Outcome |
|---|---|---|---|---|---|
| 2026-11-10 01:25 | 42735627387 | push, dev |
f5ef2dceaee |
Linux | Run failed. |
| 2026-11-14 18:05 | 43624611183 | pull_request, #1591 |
3e4c9f59ebe |
Linux | Attempt 2 passed; #1591 merged. |
| 2026-11-15 08:26 | 43694553887 | push, dev |
45081e3e553 |
Linux | Run failed. |
| 2026-11-16 00:45 | 43782274623 | push, dev |
52da532f5e3 |
Linux | Run failed. |
| 2026-11-16 04:23 | 43803791787 | pull_request, #1646 |
181da97bea4 |
Windows | Attempt 2 passed; #1646 merged. |
| 2026-11-16 15:57 | 43902030265 | push, feature/nvx-refactor (#1651) |
263d9ee8dff |
Windows | This incident. |
Root cause
Permalinks point to dev at 28a0c326f5e.
- The tool names a commit candidate after its SHA.
_append_commit_candidatesnames each commit that mentions the issuef"{commit[:12]} {subject}"(issue_workspace.py#L272-L287). The test's fixture creates two such commits,Fix #57 on devandWork on issues/57(test_issue_workspace.py#L236-L239). - The fixture's SHAs change with the clock.
_commit(L39-L43) commits with a fixed identity but at the current time, so the SHAs of both commits differ from run to run. - The assertion searches every candidate's name for a bare
570.self.assertFalse(any("570" in name for name in names))(L283) is meant to catch the decoys: the branchissue-570-unrelated(L241) and pull request #11,Fix #570(L174-L181). But it also matches a SHA prefix that happens to contain570. Each of the 10 three-character windows of a 12-character prefix is570with a probability of 1 in 4,096, so a prefix contains570with a probability of about 0.24%, and either of two prefixes with about 0.49%: about 1 run in 205. The worktree candidate's name, a path under the fixture's random temporary directory (L243), adds about 0.01%. - The tool excludes the decoys.
_number_pattern(issue_workspace.py#L227-L228) requires the number to stand alone, so the branch scan (L396-L412) skipsissue-570-unrelated._reference_pattern(L223-L224) does the same for the pull requests' titles and bodies (L326-L338), as the commit searches do for their messages (L432-L438). The test checks pull request #11 separately, with"#11 "(test_issue_workspace.py#L282). The tool keeps a pull request's head ref, herefix-570, only in the candidate's details (issue_workspace.py#L256-L267).
Reproduction
Pinning the fixture's commit dates makes its SHAs deterministic. On Linux (Ubuntu under WSL, Python 3.14.4, Git 2.53.0), with the two files as they are on dev:
GIT_AUTHOR_DATE='1794787201 +0000' GIT_COMMITTER_DATE='1794787201 +0000' \
python3 -m unittest scripts.test_issue_workspace.IssueWorkspaceTests.test_find_groups_exact_issue_matches_from_every_source
This fails 3 times out of 3 with the CI's assertion at line 283, because these dates give the Fix #57 on dev commit the prefix 02e570e6ec14. With 1794787200 +0000, the test passes. Other platforms may produce other SHAs for the same dates.
Impact
- The failure fails
Required status check, the only required check, so a pull request that hits it needs a rerun: #1591 and #1646 passed on attempt 2. - A
devpush that hits it publishes no development release and records no performance results, becausePublish development releaseandRecord performance resultsneed bothNVX CLIjobs to succeed (ci.yml#L1598-L1610, ci.yml#L1847-L1857). Both jobs were skipped in the threedevpushes in the table.
Suggested fix
Match the decoy branch's name instead of the bare number:
- self.assertFalse(any("570" in name for name in names))
+ self.assertFalse(any("issue-570" in name for name in names))
issue-570cannot occur in a hexadecimal SHA prefix, and a temporary directory's random name holds only lowercase letters, digits, and_.- With this change, the test passes at both pinned dates above. It still fails at line 283 when
_number_patternloses its(?![0-9])guard, because the decoy branch then matches issue 57. The"#11 "check on line 282 still covers pull request #11. - The change modifies an existing test, so its pull request needs the
tests-modifiedlabel.
Pinning the commit dates in _commit would also stop the flake, but the assertion would still depend on whichever SHAs those dates produce.
Related work
- #1337 added the test and this assertion in
6de09c84e24. - #1591 and #1646 needed a rerun because of it, and #1651 hit it in a push run.
- 主要言語
- Rust
- スター
- 848
- フォーク
- 63
- 平均マージ
- 13時間 37分
- マージ済み PR(30日)
- 571
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドなし
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
microsoft/nvx のほかの issue
-
area: ci
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
メンテナーはふだん 1 日以内に返信
-
openvmm: virtiofs: aggregate synthetic root st_nlink counts single-file shares as subdirectoriesオープンarea: openvmm
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
microsoft/nvx#1632 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
area: openvmm
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
microsoft/nvx#1628 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
area: openvmm
難易度 2/5 1〜3時間 初心者へのやさしさ 80/100
microsoft/nvx#1629 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
area: openvmm
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
microsoft/nvx#1631 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
似ている issue
-
good first issue help wanted
難易度 1/5 1時間未満 初心者へのやさしさ 87/100
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
jolars/panache#627 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 84/100
swc-project/swc-next-issues#490 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
oxc-project/tsrs#1010 ·
メンテナーはふだん 1 日以内に返信
-
Link is deadオープン
難易度 1/5 1時間未満 初心者へのやさしさ 63/100
nicholas-ochoa/OpenSC2K#104 ·