Fuzz: a viewer known-5xx entry fires on one run and not the next at the same commit and seed

未关闭
#1,578 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
3/5
预计耗时
1-2 天
新手友好度
45/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
github-actions
领域
ci-cd, testing

调研方向

Start with the viewer pass of the fuzz workflow, where the per-entry summary is written, and with tests/fuzz/known-5xx.toml, which lists the 15 known entries. The first change is to print the names of the known-with-no-5xx, unlisted-5xx and listed-but-reached entries to the job log as well as the job summary. Done means a red run's log names the quiet entry; the second step, finding why that entry's 5xx varies, needs the run history.

由索引模型根据 Issue 内容生成。

描述

bug

The fuzz workflow's viewer pass gave two different verdicts for the same commit and the same seed.

  • Run 41328357368, commit 968d9b37, seed 20261006.
    • Attempt 1: viewer ... RED | ... | 5xx operations 14 (not listed 0, known entries 15, of them with no 5xx 1) | reached 72, unreached 172.
    • Attempt 2 (the failed job re-run, nothing else changed): viewer ... GREEN | ... | 5xx operations 15 (not listed 0, known entries 15, of them with no 5xx 0).

So one of the 15 viewer entries in tests/fuzz/known-5xx.toml does not answer 5xx on every run. A listed entry that does not fire turns the pass red, so the scheduled nightly run can go red with no change to the code.

The job log does not say which entry went quiet: the per-entry detail goes only to the job summary, and the run uploads no artifact. Two things would close this:

  1. Print the names of the entries in each category (known with no 5xx, unlisted 5xx, listed but reached) to the job log as well as the summary, so a red run can be diagnosed from its log.
  2. Once the quiet entry is known, find why its 5xx depends on the run (ordering, data left by earlier operations, a timeout) and either make it deterministic or mark it as one that may not fire.
主要语言
Python
星标
0
派生
4
平均合并
6 小时 5 分钟
30 天内合并 PR
453

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

Controlvar/controlvar 的其他 Issue

查看 Controlvar/controlvar 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。