repo compare: flag replay tasks a guidance revision may have seen

Abierto
  • ¿Algo no está bien?
#290 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
data, testing

Línea de trabajo

Start by reading the repo tasks replay, repo run, and repo compare entry points, plus contract sections 4.6 and 9 and the fixture tests. Trace how replay tasks store gold thread data and how compare reads arm metadata; then implement and test the requested review time and guidance history flags. Done means fixture tests cover PR-name and after-review-date flags, unrelated tasks remain unflagged, and the docs explain both flags.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem

A guidance:rev:<sha> arm runs a revision of a repository's guidance against review_replay tasks built from PRs merged earlier. When the guidance was written or edited after a task's review, it can encode that review, and the arm then "catches" the thread because a rule was written from it. Nothing in a batch shows this.

A replay of IntelPython/dpnp#3099 (eight AGENTS.md files; 26 tasks from dpnp PRs merged 2026-09-01..10-28; guidance:none vs guidance:rev:d8fb283) ran into it:

  • The cut looked clean. The guidance's first commit (7781481, 14 Oct 18:45Z) was mined from reviews before it. So a split by review date looked like an out-of-sample test: 9 tasks reviewed after the cut, recall +0.111 against −0.025 for the 17 before.
  • It was not clean. The evaluated revision has later guidance commits that name three of those 9 tasks' PRs:
    • c9b533d "Update test guidance for the fixes in #3103";
    • a9f79ed "Update CMake guidance for the extension helper in #3111";
    • 441690c, with a rule taken from #3098.
  • And 441690c came after every one of them. It was committed after the first review comment of all 9.
  • The batch can't show it. task.json has no review time, and meta.json records the arm's SHA but not its guidance history. The split, and finding the leak, were done by hand.

Proposal

  • Review time: repo tasks replay writes reviewed_at into task.json: the createdAt of the earliest gold thread's first comment. The threads file already holds it.
  • Guidance history: repo run, for a guidance:rev:<rev> arm, records the commits reachable from <rev> that touch the manifest's guidance globs, in meta.json arm_rev.guidance_commits ({sha, date, subject}, newest first, capped).
  • Report it: repo compare flags the tasks such an arm may have seen.
    • By date: some guidance commit is dated after the task's reviewed_at.
    • By name: a guidance commit's message names the task's source_pr (#<n>).
    • The output says how many tasks each flag covers and names the named ones. The rest can be compared on their own (with a repo compare --task filter, or an option for it).
  • Arms with no history: a patch: arm has none, so it reads "unknown". guidance:raw is the base's own guidance and is never flagged.

Open questions

  • The date rule is strict. Here it flags all 26 tasks for d8fb283: the last guidance commit came after every review. Is a per-file rule better, counting only commits to a guidance file the task's diff makes relevant?
  • Which date? Author date or committer date, given that a rebase moves the committer date.
  • Content: should a commit that names no PR, but adds text matching a gold comment, be found too? That is a text-similarity question, likely out of scope.

Done when

The fixture tests cover two cases: a guidance commit that names a task's PR, and one dated after a task's review. repo compare flags both, and leaves the other tasks unflagged. The docs (contract §4.6 and §9) say what the flags mean.

Lenguaje dominante
Python
Estrellas
9
Forks
9
Merge medio
1 d 8 h
PR fusionados (30 d)
267

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de intel/dokimi

Todos los issues de intel/dokimi

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.