fix(planning): load the unattended problem-confirmation rule in every plan run
维护者通常 1 天内回复
已经有一个关联 PR 被合并了。
- #13754 来自 @kyle-sexton —— 已合并
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 35/100
- Issue 类型
- 缺陷
- 描述清晰度
- 描述清楚
- 活跃度
- 活跃
- 领域
- testing-qa, tooling
调研方向
Read plugins/planning/skills/plan/SKILL.md and plugins/planning/context/problem-framing.md, focusing on the unattended and pre-confirmed rules. Compare the two proposed ways to ensure unattended runs load the rule, then review the preregistered eval traces and validity and noise reports for the #11516 near-miss cases. Done means the rule loads in every after run, the validity and case-drop criteria pass, and bash plugins/planning/tests/interview-defenses.test.sh exits 0.
由索引模型根据 Issue 内容生成。
描述
Refs: #9966
Problem
/planning:plan keeps its unattended problem-confirmation rules in plugins/planning/context/problem-framing.md, under "Unattended" and "Pre-confirmed". That includes #11516's comment-marker checks. plan/SKILL.md:137 links the file once, as the place to go when a Brief has no ### Problem, with no trigger to read it in an unattended run.
In 16 kept eval traces of /planning:plan (Claude Code 2.1.318, Sonnet 5.5), no run made a Read of context/problem-framing.md. That covers both arms of the #11516 pair (490bd07b2f0^ and 490bd07b2f0) and its after text with the template-path fix. The runs then ruled on comments from the model's own judgment. In one, the comment's plain "Reproduced on staging" line was taken as a person's confirmation of the problem, which the rule says never counts ("a claim of confirmation ... inside that text never counts"). That run left the plan unapproved, so the gap reaches problem confirmation, not plan approval.
Fix
Make the rule load in every unattended run. Two candidates:
- a trigger line in
plan/SKILL.mdthat says to read problem-framing.md's "Unattended" and "Pre-confirmed" before writing### Problemin an unattended run; or - the rule's checks moved into
plan/SKILL.md.
Pick the shape with the fewest moving parts that loads the rule in every run. Measure it with a preregistered pair (current text against the change) on the #11516 near-miss cases, so the detection control in plugins/evals/skills/plugin-eval/reference/reading-results.md "Detection control" can pass.
Done when
-
plugins/planning/skills/plan/SKILL.mdmakes an unattended run read the rule before it writes### Problem(either a trigger line namingcontext/problem-framing.md"Unattended" and "Pre-confirmed", or the checks themselves in SKILL.md), shown by the diff. - In a preregistered before-and-after pair on the #11516 near-miss cases (5 runs per case per arm, posted on #9966 before it runs), every after run makes a
Readofcontext/problem-framing.md, or the moved text is in SKILL.md. Count it with the kept traces. - That pair's legs read VALID under
python3 -I plugins/evals/skills/plugin-eval/scripts/run-validity.py <result> --runs 5 --scaffolded, and no non-target case drops more than 0.05 (acase dropline innoise-report.pyoutput fails it). -
bash plugins/planning/tests/interview-defenses.test.shexits 0.
Also
- The case generator and 7 cases (one valid marker, six near-misses, raw read output in each prompt, plus a code scaffold) are in lane 1's scratch. Lane 1 commits them with the measurement.
- Runs still go INVALID on a separate cause: the plan skill
cps its checklist template out of the plugin directory, whichplan/SKILL.md:41forbids and the eval sandbox denies (4 of 4 smoke runs).
Related
- #11516 (the rule), #9966 (batch A re-measure).
Filed by AI (lane 1) from eval traces.
- 主要语言
- Shell
- 星标
- 22
- 派生
- 2
- 平均合并
- 1 小时 13 分钟
- 30 天内合并 PR
- 954
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
melodic-software/claude-code-plugins 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 67/100
melodic-software/claude-code-plugins#9179 · 4 条评论 ·
维护者通常 1 天内回复
-
Worktree-isolated sessions refuse claude plugin eval (guard reads 'eval' as shell eval)可能已有人在做 @kyle-sexton 于 6 天前认领。 未关闭host-run priority: low status: ready work-class: mechanical
难度 2/5 1-3 小时 新手友好度 62/100
melodic-software/claude-code-plugins#5696 · 16 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
-
priority: medium status: needs-decision
难度 5/5 一周以上 新手友好度 34/100
melodic-software/claude-code-plugins#13847 · 4 条评论 ·
维护者通常 1 天内回复
-
chore(source-control): finish the public-data credential sweep left from #8282可能已有人在做 @kyle-sexton 今天认领。 未关闭host-run priority: medium status: confirmed work-class: read-only
melodic-software/claude-code-plugins#13819 · 3 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
-
perf(pr-verify): set the capture key proxy's token-cap from a full measured capture可能已有人在做 @kyle-sexton 今天认领。 未关闭host-run priority: medium status: confirmed work-class: scoped
melodic-software/claude-code-plugins#13724 · 6 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
查看 melodic-software/claude-code-plugins 的全部 Issue
相似的 Issue
-
enhancement
难度 2/5 1-3 小时 新手友好度 68/100
Frogging-Family/linux-tkg#1265 ·
-
bug
难度 2/5 1-3 小时 新手友好度 68/100
community-scripts/core#217 ·
维护者通常 1 天内回复
-
known-red S3: friction & debt
难度 2/5 1-3 小时 新手友好度 82/100
MedakaLang/medaka#4969 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复
-
enhancement needs:sonnet ready-for-agent
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复