fix(aidd-dev): review leaves out-of-diff phases and the verified percent undefined
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 74/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- documentation
Research direction
Start with plugins/aidd-dev/skills/05-review/SKILL.md and actions/02-review-functional.md, then compare review-template.md and references/review-rubric.md against the stated scope. Update the Test section to cover out-of-scope phases, deterministic reruns, and percentage calculation. Done means the four acceptance scenarios pass and all four documents agree without contradiction.
Written by the indexing model from the issue text.
Description
Problem
05-review does not say what happens to plan phases outside the reviewed diff, nor how the verified percent is computed. Reviewing one phase of a multi-phase plan, three models on the same skill produced three incompatible review.md files.
The skill contradicts itself:
SKILL.md:30saysreview.mdis overwritten each run: "a snapshot of the current diff, not a history".02-review-functional.md:23says thePhasessection holds "one block per plan phase".- Nothing says whether the previous
review.mdis read before writing. review-template.md:33renders{{pct}}% ({{n_checked}}/{{n_total}}), but nothing says which phases entern_total.not-applicableis only defined as "neither a finding nor a blocker" (review-rubric.md:19), never for the percent, and never when to use it.
Observed on a 7-phase plan, phase 4 reviewed in three worktrees, aidd-dev 2.4.1 (05-review is unchanged in 2.5.0):
| Tool + model | Diff | Phase blocks | Verified line |
|---|---|---|---|
| Mistral Vibe CLI + Mistral Medium | 6623000...HEAD |
7 | 100% (6/6), with 5 of 6 phase-4 boxes checked |
| Mistral Vibe CLI + GLM | 6623000...69e491a |
1 (phase 4 only) | 100% (6/6) |
| OpenCode + Qwen | 0bb8973...HEAD |
7 | 53% (16/34), while the file renders 21 of 36 boxes checked, and 16/34 is 47% |
- Mistral Medium and Qwen have word-for-word identical blocks for phases 1, 2, 3, 5, 6 and 7: both copied the previous
review.md. Mistral's phase 2 still says "ajoutés par ce diff" while its ownFiles checkedlists only phase-4 files. Copied evidence reads as fresh verification. - GLM dropped the six other phases.
- Qwen tagged the 15 criteria of out-of-diff phases
not-applicable, Mistral used the same tag for a partially verifiable phase-4 criterion: one tag, two meanings.
Scope
review.mdstays a snapshot: the skill never reads the previousreview.md, and says so.- Every run renders one block per plan phase, re-derived from the plan.
- A phase the diff does not touch keeps its block, marked out of scope in its heading, with its criteria unchecked and no evidence. Nothing is copied from an earlier review.
- A phase the diff touches is verified as today:
[x]with evidence, or[ ]taggedfix,not-applicableorfixed. - Define
not-applicable: a criterion of a phase in scope that this change cannot satisfy by nature (for example, it depends on a later phase). It is never used to mark a phase out of scope. - The verified percent counts only the criteria of phases in scope, minus
not-applicableones, and matches the boxes rendered. - Update the
Testsection of02-review-functional.mdto check the four rules above.
Acceptance criteria
- Reviewing phase 4 of a 7-phase plan renders 7 phase blocks, 6 of them marked out of scope.
- An out-of-scope block holds no
[x]and nofile:line. - Re-running the review on the same diff gives the same
PhasesandVerification, whatever the content of the previousreview.md. - With 6 criteria in scope, 5 checked and 1
not-applicable, the report shows100% (5/5). -
SKILL.md,02-review-functional.md,review-template.mdandreview-rubric.mdagree on these rules, with no contradiction left.
Prior art in this repository
SKILL.md:30: the snapshot rule, kept.02-review-functional.md:17: "the percent and count of checked criteria", to make precise.review-rubric.md:19: the only definition ofnot-applicable, to extend.
Out of scope
- How "the diff against the repository default branch" resolves (5 occurrences in
05-review, never defined). A clone whoseorigin/HEADpoints at a feature branch diffs against it. Separate issue. - Enforcing
review-validator.yml: a section outside the list survived in the three reviews above. Separate issue.
Reported on Discord: https://discord.com/channels/1173363373115723796/1547134531188822047
- Dominant language
- TypeScript
- Stars
- 481
- Forks
- 45
- Avg merge
- 14h 36m
- Merged PRs (30d)
- 110
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ai-driven-dev/framework
-
refactor(aidd-orchestrator): the check zone says when to stop, and reviews its axes in one round Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
ai-driven-dev/framework#887 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
ai-driven-dev/framework#873 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
ai-driven-dev/framework#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
ai-driven-dev/framework#625 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
ai-driven-dev/framework#467 · 1 comment ·
All issues in ai-driven-dev/framework
Similar issues
-
Browser Waiting for: Product Owner
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
getsentry/sentry-javascript#24577 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
agilepathway/label-checker#640 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
copse-dev/agent-pane#2953 ·
-
agentic-workflows
Difficulty 1/5 Under an hour Newbie friendliness 85/100
githubnext/rig#534 ·
-
automation missing-model model-sync provider:pioneer
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
anomalyco/models.dev#7701 ·