selftest: count-unspec-canary is seed-dependent and goes red ~0.24% of selftests on a correct tree
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 22/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 领域
- testing-qa
调研方向
The flake comes from corpus/lua-tests/math.lua line 845, which calls math.randomseed() with no argument, so the count-unspec-canary plant in tools/selftest.sh (lines 79-82) grades lua-count with --repeat 20 against a floor set in tools/grade.sh line 417. Read the grade floor logic and the self-test row in tests/gut.sh first. Done means the plant's verdict is independent of the time-derived seed, with 1,000 consecutive clean runs or a stated false-red bound, and the row still goes red when the floor is gutted.
由索引模型根据 Issue 内容生成。
描述
Found while timing #71 (hq#58 Wave 2). The selftest plant count-unspec-canary can go red on a correct tree. Main has not changed since c650143.
Observed: run 41349374525 attempt 3, job 124982803346, step RUN selftest. The same head was green on attempts 1 and 2, and the failing step runs before any of #71's changes take effect:
PLANT count-unspec-canary expected=math.lua=PASS nextvar.lua=PASS rc=0 got=math.lua=FAIL nextvar.lua=PASS rc=1 WRONG
Mechanism:
corpus/lua-tests/math.lua:845callsmath.randomseed()with no argument, so the seed comes from the time and an address.- The statistics loops that follow then run a seed-dependent number of asserts.
- The plant (
tools/selftest.sh:79-82) gradeslua-countwith--repeat 20. tools/grade.sh:417setsfloor = lo * 0.60, wherelois the minimum of 20 reference runs. Any of the 20 candidate runs that falls below that floor turns the row FAIL.
Measured: 9,000 runs of .ref/bin/lua-count math.lua.
- Counts ran from a minimum of 25,699 to a maximum of 237,979, with a median of about 78.6k.
- rc was 0 every time, and the digit-free output had the same md5 every time.
- Under the plant's rule, that distribution gives an estimated ~0.24% false red per selftest, about 1 in 420.
- In the collector's 84 earlier suite logs it was never red, which fits that rate.
- Load does not change it. The distribution measured while a 4-lane fuzz ran was the same as without it (KS D=0.0147, 5% critical value 0.0304).
This is a gate that can go red without a bug. It was not a liar in the false-green direction, but a flaky plant trains people to rerun red.
Done when:
- The plant's verdict no longer depends on the time-derived seed. For example: fixed-seed or fixed-count inputs for the plant, or a floor taken from a reference sample with a stated false-red bound of 1e-6 or less per selftest. Do not edit the vendored test, and do not lower the 0.60 fraction as the fix.
- Evidence in the PR: at least 1,000 consecutive runs of the plant's grade command (
tools/grade.sh --candidate .ref/bin/lua-count --ref .ref/bin/lua --tier S --only math.lua,nextvar.lua --repeat 20) with 0 FAIL, or the computed bound. - The
count-unspec-canaryrow intests/gut.shstill bites: with the floor gutted, the self-test goes red.
Generated by Claude Code
- 主要语言
- Lua
- 星标
- 0
- 派生
- 0
- 平均合并
- 43 分钟
- 30 天内合并 PR
- 10
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
InauguralSystems/EigenLua 的其他 Issue
-
难度 5/5 一周以上 新手友好度 8/100
InauguralSystems/EigenLua#78 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 45/100
InauguralSystems/EigenLua#74 ·
维护者通常 1 天内回复
-
难度 3/5 1-2 天 新手友好度 35/100
InauguralSystems/EigenLua#63 ·
维护者通常 1 天内回复
-
bug eigenscript-gap
难度 3/5 1-2 天 新手友好度 42/100
InauguralSystems/EigenLua#50 ·
维护者通常 1 天内回复
-
bug eigenscript-gap
难度 5/5 一周以上 新手友好度 12/100
InauguralSystems/EigenLua#49 ·
维护者通常 1 天内回复
查看 InauguralSystems/EigenLua 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
silverbulletmd/silverbullet#2254 ·
维护者通常 1 天内回复
-
bug discord triage
难度 2/5 1-3 小时 新手友好度 77/100
维护者通常 1 天内回复
-
severity: medium
难度 2/5 1-3 小时 新手友好度 68/100
luainkernel/lunatik#2699 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
esmuellert/codediff.nvim#594 ·
-
难度 2/5 1-3 小时 新手友好度 78/100