Make REVIEW ask for a change of mechanism on a recurring failure class, and name criteria that have no bound

Aperta
  • Qualcosa non va?
#456 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
1-2 giorni
Idoneità per principianti
22/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
markdown
Ambito
ai-infra-agents

Direzione di ricerca

Start with the review.md prompt template and docs/agent-guides/control-result-protocol.md, which the issue names. The work adds two instructions (a recurring-class rule and an unbounded-criterion rule) and bumps the prompt version, and it depends on #452's fields such as same_root_cause_as, which the payload does not show being merged. Done means both sections render and a structure test pins them, but the wording needs design decisions first, so check with maintainers before starting.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement

Problem

review.md has no completion contract for a failure class that keeps coming back. When a finding is the n-th instance of something an earlier FIX claimed to fix, the reviewer is asked only whether the fixed claim "the code does not bear out" is true. Each instance fix was borne out, so the reviewer records the old finding as resolved and looks for the next instance. Nothing tells the reviewer what to do with an acceptance criterion that cannot be decided as worded.

Evidence

From #451, for Lychee-Technology/forma#801:

  • Every round was the same shape. The reviewer noted the previous round's finding as resolved, ran a probe against the scanner, found one more SQL spelling it missed, and filed it as "strengthen discovery and add a regression for this example".
  • Each new example became a requirement. From R6 on, every finding added "must be addressed in this PR rather than deferred".
  • The criterion had a bound nobody used. Criterion 4 of Lychee-Technology/forma#740 ("cannot quietly evade the inventory") has no fixed point for a text-recognising guard. The issue hinted at a bound ("what coverage guarantees can be enforced cheaply at build time"), and no reviewer ever named one.
  • Severity was not the problem. The findings were valid under the literal wording and were not inflated.

Desired outcome

review.md gains two instructions. Neither of them clears a finding.

  1. A recurring class. When a finding is another instance of an earlier finding whose fixed commit addressed only that instance, link it (same_root_cause_as, #452). Its required_resolution then asks for a change of mechanism that removes the class, not for another regression for this example.
  2. An unbounded criterion. When an acceptance criterion cannot be decided or bounded as worded, say so in spec and in the finding. State the bounded reading the reviewer would accept, so a human can decide. Do not file further instances of the same unbounded demand.

Classification and severity rules are unchanged. A real defect stays a finding.

Acceptance criteria

  • The template renders with both sections, and a structure test pins that they exist.
  • The prompt version is bumped. docs/agent-guides/control-result-protocol.md describes the expectation alongside #452's fields.

Trade-offs and unknowns

  • Under-reporting. A reviewer could label a hard criterion "unbounded" to avoid work. The instruction changes what a finding asks for, never whether it is raised, and a human reads the stated reading at the block.
  • Strength of the evidence. The effect on reviewer behaviour is untested, as it is for #455. #454 is the backstop.

Depends on #452.

Lingua principale
Python
Stelle
0
Fork
0
Merge medio
5h 42m
PR unite (30g)
110

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di iceboundrock/AutoForge

Tutte le issue di iceboundrock/AutoForge

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.