Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Benchmark every PR on deterministic costs, with on-demand stable-hardware runs and a results history

Abierto
  • ¿Algo no está bien?
#90 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
docker, github-actions, go
Área
ci-cd, performance, testing-qa

Línea de trabajo

Start by inventorying the Go micro-benchmarks, tests/performance, tests/stress, tests/resources, tests/enry-performance, and BenchmarkProfileScanner, then inspect how CI currently runs them. Use the related #89 and #75 proposals to clarify dependencies. Done means the three benchmark tiers, matching local target, public history and regression reporting are implemented without a schedule trigger.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement follow-up

Why

The raw material exists, but nothing watches performance continuously:

  • 16 Go micro-benchmarks, none run in CI;
  • BenchmarkProfileScanner, with pprof capture, disabled unless DIRCUE_PROFILE_ROOT is set;
  • Docker-based harnesses (tests/performance against Linguist, tests/stress, tests/resources with cgroup limits, tests/enry-performance), all run by hand, with results committed as JSON.

There is no history, no baseline comparison and no alerting. That's how the Git-path cliff (#73) shipped unnoticed: 23.5 s vs 6.5 s for the language pass, and 205.6 s vs 6.7 s for metrics, on the Linux kernel.

Shared CI runners are too noisy for wall-time gates (±20% is common), so the design splits into what is deterministic and what needs stable hardware. All of it is event-driven: nothing runs on a schedule (#92).

Proposal: three tiers

  1. Every PR (hosted runners, noise-free signals).
    • Logical-work counters (#89) on small pinned fixtures, gated against committed budgets. Physical costs are gated only under a pinned pack layout and controlled cache conditions, e.g. a small delta-replay fixture with an explicit byte-amplification budget.
    • Go micro-benchmarks compared with benchstat against the merge base, gating on allocs/op and B/op, which are stable on shared runners.
    • Wall time reported but not gated.
  2. On demand (stable hardware). A workflow_dispatch-only workflow, plus a local make target that produces the same result format. It targets a self-hosted runner label when a dedicated, quiet machine exists (self-hosted runners are free for public repositories), and runs macro-benchmarks on the golden corpus (#75):
    • N repeated runs per case, covering both source modes (Git and directory), each preset, and key modules;
    • wall time, CPU, peak RSS and bytes read;
    • pprof profiles archived.
    • Each run appends to a public history (e.g. github-action-benchmark to GitHub Pages, or Bencher). Regressions are detected statistically against the previous recorded runs and reported on the run, or by opening an issue.
  3. Release.
    • The full matrix against the previous release, Linguist and scc.
    • Constrained-resource runs (cgroup memory and CPU limits) that prove each preset's and memory ceiling's claims.
    • BENCHMARKS.md regenerated from data.

Budgets. Publish per-corpus-entry targets for reference hardware, e.g. the kernel language pass from Git, peak RSS, and analyze all time, and check them at release.

Acceptance

  • PR-tier counter and allocation gates live in CI.
  • On-demand stable-hardware workflow (workflow_dispatch only) and a matching local target, feeding a public history with regression detection. No schedule: trigger.
  • Release-tier report generated automatically and attached to each release.
  • Existing harnesses (tests/performance, tests/resources, BenchmarkProfileScanner) folded into the tiers rather than duplicated.

Related: #39 (optimization campaign methodology), #73.

Part of the 1.0 quality program: #83. Epic: #81.

Lenguaje dominante
Go
Estrellas
0
Forks
0
Merge medio
5 h 8 min
PR fusionados (30 d)
54

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de war-and-code/dircue

Todos los issues de war-and-code/dircue

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.