Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

repo tasks: authoring replay tasks from merged PRs

Abierto
  • ¿Algo no está bien?
#294 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
42/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
ai-infra-agents, testing-qa

Línea de trabajo

Start by reading the existing repo tasks replay implementation and the task contract’s section 3 to see how review replay builds tasks and how feature or fix tasks are graded. Then inspect the manifest handling and repo tasks check path, along with the network-free fixture tests. Done means authoring tasks from merged PRs are accepted by repo tasks check, with the untouched base failing and the PR diff passing as the oracle.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Problem

repo tasks replay builds dokimi:review_replay tasks, which measure what guidance does to an agent reviewing code. Most guidance is read while writing code. Of the eight AGENTS.md files in IntelPython/dpnp#3099, most lines say where a new function's kernel and dispatch helpers go, how tests are laid out, and what a changelog entry looks like. The review replay of that PR exercised little of it.

dokimi can already grade such work. A task.json with family: feature or fix and the repository's own grade entry point does it (contract §3). But those tasks are written by hand, so this measurement is skipped too, just as review counting was before repo tasks replay.

Proposal

Build authoring tasks from merged PRs, with the same input and the same rules for what to leave out as review replay: repo tasks replay --kind authoring, or a sibling subcommand. Per merged PR:

  • Base: the merge base of the PR with mergeCommit^1, as for review replay.
  • Prompt: the PR's title and description, with links to its commits and diff and anything quoting the change removed. --prompt-file replaces the default.
  • Hidden tests: the test files the PR added or changed. They are held out of the workspace and applied only at grade time.
  • Grade: the repository's test command from the manifest, on the hidden tests; the gate is that they pass. oracle is the PR's own diff, so repo tasks check can show that the base fails and the oracle passes.
  • Left out: guidance files and eval_dir, as review replay leaves them out.

Open questions

  • Which PRs qualify: with tests, under a size limit, not only refactors.
  • Leaked tests: a description that names the test functions.
  • Build cost: dpnp needs its SYCL extensions built for every cell. How much can the manifest's caches and a prebuilt base image save?
  • Partial credit: some hidden tests passing. Should it count, as a score rather than a gate?
  • Flaky tests: run the oracle twice in tasks check?

Done when

On a window of merged PRs of a real repository, the command writes tasks that repo tasks check accepts (the untouched base fails, the oracle passes). The tests use a fixture repository with no network.

Lenguaje dominante
Python
Estrellas
9
Forks
9
Merge medio
1 d 7 h
PR fusionados (30 d)
231

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de intel/dokimi

Todos los issues de intel/dokimi

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.