fix(evals): give the methodology rewrite answer a template whose target slot needs a stated basis
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 2/5
- 预计耗时
- 1-3 小时
- 新手友好度
- 72/100
- Issue 类型
- 文档
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
调研方向
阅读 plugins/evals/skills/methodology/SKILL.md，尤其是“Rewrite this criterion”的回答以及第42–52行附近现有的指导。将该回答替换为所要求的填空模板，在 .changes/evals/ 下添加一个片段，然后运行列出的两个验证命令；prompt 和 grader 文件应保持不变。basis: 检查通过、两个验证器均以退出码 0 结束，并且 CI 在 PR head 上为绿色，即表示完成。
由索引模型根据 Issue 内容生成。
描述
Parent: #5973 (slice 1 of 2). Runs first; slice 2 waits on it.
Runs in: lane. Skill text and an offline grader check only; no model run.
Problem
In the measurable-criterion case (plugins/evals/evals/measurable-criterion/) the evals plugin passes the calibrated four-properties grader in 2 of 3 with-plugin runs, on two VALID runs (2026-10-13 and 2026-10-24). The failing reply sets "at least 99%" for red-flag escalation and then says the number comes from severity. The hub already says severity is no basis (plugins/evals/skills/methodology/SKILL.md:42-45) and that the rewrite carries a grounded target (:46-52), and a one-line wording addition tried on 10-24 did not move the result. The model states the rule and still keeps the severity-based number.
Decision (taken from the 10-24 data, not escalated): option (a) of that comment. The rewrite answer gets a template whose target slot can only be filled from a measured baseline, a prior result, a benchmark or expert review, or else the baseline-relative form, so a severity-only number has no slot. Basis: the failing reply named the rule and broke it, which points at the answer's shape, not missing knowledge (#5973 row of the backlog HITL batch).
Fix
In plugins/evals/skills/methodology/SKILL.md, replace the prose of the "Rewrite this criterion" answer with a short fill-in template: quality, number or scale, trial set and size, target, and a basis: slot whose allowed values are exactly the four sources above or the baseline-relative form ("at most half the current rate over N trials"). Say a target with no allowed basis is written in the baseline-relative form, never as a placeholder. Do not change the case prompt or the four-properties rubric. If the change makes any four-properties calibration sample stale, update the sample and say so in the PR (slice 2 recalibrates). Add a .changes/evals/ fragment.
Done when
-
grep -c 'basis:' plugins/evals/skills/methodology/SKILL.mdprints 1 or more, inside the "Rewrite this criterion" answer -
git diff --name-only origin/main...HEAD -- plugins/evals/evals/measurable-criterion/prompt.md plugins/evals/evals/measurable-criterion/graders/prints nothing -
python3 plugins/evals/skills/validate/scripts/validate-cases.py plugins/evals/evalsexits 0 -
bash plugins/skill-quality/scripts/check-skill.sh plugins/evals/skillsexits 0 -
ci-statusis green on the PR head
Split by the backlog HITL session (2026-10-25, owner rule Q93: split large work before a run picks it up). This was generated by AI.
- 主要语言
- Shell
- 星标
- 22
- 派生
- 2
- 平均合并
- 2 小时 48 分钟
- 30 天内合并 PR
- 939
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
melodic-software/claude-code-plugins 的其他 Issue
-
priority: medium status: ready work-class: scoped
难度 2/5 1-3 小时 新手友好度 76/100
melodic-software/claude-code-plugins#10227 · 1 条评论 ·
维护者通常 1 天内回复
-
priority: medium status: ready work-class: scoped
难度 2/5 1-3 小时 新手友好度 68/100
melodic-software/claude-code-plugins#10128 · 1 条评论 ·
维护者通常 1 天内回复
-
pr-pipeline: clear a feedback-not-landed escalation once the head moves and no lane thread is open未关闭priority: medium status: needs-decision work-class: scoped
难度 2/5 1-3 小时 新手友好度 83/100
melodic-software/claude-code-plugins#10074 · 4 条评论 ·
维护者通常 1 天内回复
-
priority: medium work-class: scoped
难度 2/5 1-3 小时 新手友好度 72/100
melodic-software/claude-code-plugins#9842 · 5 条评论 ·
维护者通常 1 天内回复
-
needs-human priority: low status: ready work-class: scoped
难度 2/5 1-3 小时 新手友好度 85/100
melodic-software/claude-code-plugins#9788 · 3 条评论 ·
维护者通常 1 天内回复
查看 melodic-software/claude-code-plugins 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
-
bug priority-medium to-agent
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 70/100
WordPress/Documentation-Issue-Tracker#2540 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
dsx-ai-factory/infra-controller#8301 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 72/100