feat(bench): track benchmark and memory results over time instead of a single-run gate
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 12/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Tranquilo
- Stack tecnológico
- github, github-actions, go
- Área
- build-system, observability, performance
Línea de trabajo
Start with the regression job in .github/workflows/bench.yml and the baseline in tools/bench/baseline.json, then find CheckRegression to see why it stops at the first failure. The issue asks for an OpenSpec change to PERF-02 and INDX-06 before any code, and the tool choice (github-action-benchmark or Bencher) is left to the spec. Done means a spec is agreed, the tool is picked, and a per-phase metrics output is designed, which is design work rather than a first contribution.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
What exists today?
The regression job in .github/workflows/bench.yml runs each night. It indexes the synthetic corpus three times, takes the median, and compares it with one stored baseline in tools/bench/baseline.json. It fails at a 10% throughput drop, at 15% more peak RSS, or above 4 GiB. It never blocks a merge. The publish job reports absolute numbers each week.
How does it behave now?
The gate measures the host as much as the code, and it keeps no history.
- Two runs on the same commit differed by 21% in throughput (#252).
- After #250, the three trials of one night spread by up to 12% in peak RSS.
- The job failed every night from 3 to 7 November. On three of those nights, its message named only throughput, because
CheckRegressionreturns at its first failure. The 40% RSS growth from #250 stayed hidden. - No one can see when a number started to move, or which phase of an index moved it. The #265 investigation rebuilt that by hand from profiles.
How should it behave?
- Each nightly run and each dispatch records its results as a time series. The series holds throughput, peak RSS, query latency, node and edge counts, and numbers per phase of an index (extract, resolve, write): bytes allocated, live heap at the end of the phase, GC cycles and batch bytes.
- The binary reports the per-phase numbers from
runtime/metrics, so the runner reads them and no profile is needed. - A page shows the series as charts. An alert fires on a change that lasts over several runs, with a statistical threshold, and not on one run.
- Optional: a pull request that changes Go code gets a report of a paired comparison. The job builds the base and the head on one runner, alternates their trials, and compares them with
benchstat. The report does not block the merge. - The tool comes from OSS. The candidates are
github-action-benchmarkand Bencher. The spec picks one, and the implementation checks its current version.
This issue replaces #252.
What kind of improvement?
Performance
Would this change existing behavior?
Yes. The nightly job stops being a pass or fail result from one run against one baseline. That changes PERF-02 and INDX-06, so it needs an OpenSpec change first. If the binary reports per-phase numbers through a new flag or output, that flag is a change to the CLI surface, and the spec names it.
🤖 Generated with Claude Code
- Lenguaje dominante
- Go
- Estrellas
- 0
- Forks
- 0
- Merge medio
- 7 h 42 min
- PR fusionados (30 d)
- 111
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de seanb4t/codegraph-go
-
chore needs-triage
Dificultad 1/5 Menos de una hora Aptitud para principiantes 88/100
seanb4t/codegraph-go#380 ·
Los mantenedores suelen responder en 1 día
-
chore needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
seanb4t/codegraph-go#379 ·
Los mantenedores suelen responder en 1 día
-
area:query chore needs-triage spec-needed
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
seanb4t/codegraph-go#377 ·
Los mantenedores suelen responder en 1 día
-
chore needs-triage
Dificultad 1/5 1-3 horas Aptitud para principiantes 84/100
seanb4t/codegraph-go#362 ·
Los mantenedores suelen responder en 1 día
-
area:query bug needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
seanb4t/codegraph-go#355 ·
Los mantenedores suelen responder en 1 día
Todos los issues de seanb4t/codegraph-go
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 77/100
Los mantenedores suelen responder en 1 día
-
automated documentation maintenance
Dificultad 1/5 1-3 horas Aptitud para principiantes 75/100
github/gh-aw-mcpg#16717 ·
Los mantenedores suelen responder en 1 día
-
Docs: Recording rules pages don't say what happens to a recorded metric when its rule is deletedAbiertointernal
Dificultad 1/5 1-3 horas Aptitud para principiantes 84/100
Los mantenedores suelen responder en 1 día
-
enhancement
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
jesseduffield/lazygit#6229 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
prometheus/prometheus#20482 ·
Los mantenedores suelen responder en 2 días