planning evals: reorder-points case expects a MEDIUM blast radius and no reviewer dispatch, which main's plan skill cannot both satisfy

未关闭 适合新手
#17,039 4 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
66/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
shell

调研方向

阅读 plugins/planning/skills/plan/SKILL.md 中的“Reviewer”部分,并将其与 issue 中提到的两个 grader 和 fixture 进行比较。然后检查 #15465 中相邻的 plan-reviewer-cut 案例,确认是否存在相同的冲突。更新该案例,使两个 grader 都反映 main 上已发布的规则,并在三次运行中验证两者都能在一次运行中通过。

由索引模型根据 Issue 内容生成。

描述

Problem

Two graders of the eval case plugins/planning/evals/plan-reviewer-cut-medium-reorder-points cannot both pass on main:

  • graders/blast-radius-medium.md requires the plan to rate the blast radius MEDIUM (pattern blast[ -]radius[^\n]{0,12}?\bMEDIUM\b).
  • graders/reviewer-not-dispatched.md requires that no plan-reviewer agent is dispatched (min: 0, max: 0, arm: both).
  • plugins/planning/skills/plan/SKILL.md ("Reviewer"): under auto the reviewer "runs when the Step 3b blast radius is MEDIUM or higher". The case's scaffold sets verification: standard (plugins/planning/evals/fixtures/plan-reviewer-cut/init-repo.sh), so a MEDIUM rating dispatches the reviewer.

Both graders came in #15465, written for a cut of the reviewer at MEDIUM. That cut did not ship: the reviewer stays on at MEDIUM until measured. In the run recorded for #16921, blast-radius-medium failed 3 of 3 runs and reviewer-not-dispatched passed 3 of 3. Each grader weighs 0.001, so no score moved, but the case's name no longer says what it exercises.

Options

  • (a) Drop max: 0 from reviewer-not-dispatched in this case, so it matches the shipped rule that MEDIUM runs the reviewer. Recommended (basis: plugins/planning/skills/plan/SKILL.md, "Reviewer").
  • (b) Retarget the case to LOW: rename it and change blast-radius-medium, if the scaffold's four changed files with a clean revert are meant to rate LOW (plugins/planning/skills/plan/context/stress-test-triggers.md rates 3 to 10 files MEDIUM and "Git revert works" lower).

Done when

  • Both graders can pass in one run on main's plan skill, and a run of 3 shows it.
  • The sibling plan-reviewer-cut cases from #15465 are checked for the same conflict.

Source: workshop 01 close-out audit.

主要语言
Shell
星标
22
派生
2
平均合并
1 小时 20 分钟
30 天内合并 PR
952

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

melodic-software/claude-code-plugins 的其他 Issue

查看 melodic-software/claude-code-plugins 的全部 Issue

相似的 Issue

更多 Shell/Bash Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。