Pin batch-size invariance of safety and judge generation in fp32 (`batch_size` 1 / 2 / 4 / 8)
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
Start with the batched generation helpers in forgelm/safety/_generate.py and forgelm/judge.py, then find the existing batch_size tests, which only check input validation. Add a tiny-model fp32 test that runs safety and judge generation at batch sizes 1, 2, 4 and 8 and asserts identical outputs. Done means that test passes, the row at docs/roadmap/risks-and-decisions.md:91 is removed, and the CHANGELOG names F-PR29-A8-10.
由索引模型根据 Issue 内容生成。
描述
Deferred work ·
F-PR29-A8-10· severity Low · cost ~1-2 h · recorded indocs/roadmap/risks-and-decisions.md:91
Summary
Both gates generate the answers they score with left-padded, batched greedy decoding (forgelm/safety/_generate.py, forgelm/judge.py), but every test stubs the batched helpers, so a change that made results depend on batch_size would ship without a failing test. The PR #29 review deferred the determinism test. #677 shows bf16 output legitimately varies with batch size, so the contract to pin is fp32.
Condition or budget
None recorded (the row still names v0.6.x).
State today (re-checked 2026-10-09)
No determinism test exists; the only batch_size tests check input validation.
Related
#676 asks for a safety-side batched-versus-single test (and accepts a weaker padding-side fix for the judge); #677 documents the bf16 variance.
Acceptance criteria
- A tiny-model fp32 test runs safety generation and judge generation at
batch_size1, 2, 4 and 8 and asserts identical outputs. - The row is removed from the deferral table and the CHANGELOG names its ID (the table's removal contract).
Recorded in docs/roadmap/risks-and-decisions.md (read at f94595f) and re-checked against the code on 2026-10-09. The row leaves that table only when the fix is on main, the CHANGELOG names its ID and any promised guard exists.
- 主要语言
- Python
- 星标
- 9
- 派生
- 1
- 平均合并
- 17 小时 14 分钟
- 30 天内合并 PR
- 17
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
HodeTech/ForgeLM 的其他 Issue
-
area: site bug good first issue severity: low wave: 4
难度 2/5 1-3 小时 新手友好度 75/100
HodeTech/ForgeLM#961 · 1 条评论 ·
维护者通常 1 天内回复
-
area: dev-tooling bug severity: low source: roadmap wave: 4
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 1 天内回复
-
documentation good first issue severity: low source: review-2026-09 wave: 4
难度 1/5 1-3 小时 新手友好度 80/100
维护者通常 1 天内回复
-
documentation severity: medium source: review-2026-09 wave: 3
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
documentation good first issue severity: medium source: review-2026-09 wave: 3
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
相似的 Issue
-
难度 1/5 1-3 小时 新手友好度 68/100
FavioVazquez/showtime#101 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 83/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 65/100
rossant/awesome-math#199 ·
-
难度 2/5 1-3 小时 新手友好度 68/100
nautechsystems/nautilus_trader#5365 ·
维护者通常 1 天内回复
-
Examples say cnx未关闭documentation embedded
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复