Benchmark every PR on deterministic costs, with on-demand stable-hardware runs and a results history
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 35/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- docker, github-actions, go
- 領域
- ci-cd, performance, testing-qa
調査の方向性
Start by inventorying the Go micro-benchmarks, tests/performance, tests/stress, tests/resources, tests/enry-performance, and BenchmarkProfileScanner, then inspect how CI currently runs them. Use the related #89 and #75 proposals to clarify dependencies. Done means the three benchmark tiers, matching local target, public history and regression reporting are implemented without a schedule trigger.
索引モデルが issue の本文から書いたものです。
説明
Why
The raw material exists, but nothing watches performance continuously:
- 16 Go micro-benchmarks, none run in CI;
BenchmarkProfileScanner, with pprof capture, disabled unlessDIRCUE_PROFILE_ROOTis set;- Docker-based harnesses (
tests/performanceagainst Linguist,tests/stress,tests/resourceswith cgroup limits,tests/enry-performance), all run by hand, with results committed as JSON.
There is no history, no baseline comparison and no alerting. That's how the Git-path cliff (#73) shipped unnoticed: 23.5 s vs 6.5 s for the language pass, and 205.6 s vs 6.7 s for metrics, on the Linux kernel.
Shared CI runners are too noisy for wall-time gates (±20% is common), so the design splits into what is deterministic and what needs stable hardware. All of it is event-driven: nothing runs on a schedule (#92).
Proposal: three tiers
- Every PR (hosted runners, noise-free signals).
- Logical-work counters (#89) on small pinned fixtures, gated against committed budgets. Physical costs are gated only under a pinned pack layout and controlled cache conditions, e.g. a small delta-replay fixture with an explicit byte-amplification budget.
- Go micro-benchmarks compared with
benchstatagainst the merge base, gating on allocs/op and B/op, which are stable on shared runners. - Wall time reported but not gated.
- On demand (stable hardware). A
workflow_dispatch-only workflow, plus a local make target that produces the same result format. It targets a self-hosted runner label when a dedicated, quiet machine exists (self-hosted runners are free for public repositories), and runs macro-benchmarks on the golden corpus (#75):- N repeated runs per case, covering both source modes (Git and directory), each preset, and key modules;
- wall time, CPU, peak RSS and bytes read;
- pprof profiles archived.
- Each run appends to a public history (e.g.
github-action-benchmarkto GitHub Pages, or Bencher). Regressions are detected statistically against the previous recorded runs and reported on the run, or by opening an issue.
- Release.
- The full matrix against the previous release, Linguist and scc.
- Constrained-resource runs (cgroup memory and CPU limits) that prove each preset's and memory ceiling's claims.
BENCHMARKS.mdregenerated from data.
Budgets. Publish per-corpus-entry targets for reference hardware, e.g. the kernel language pass from Git, peak RSS, and analyze all time, and check them at release.
Acceptance
- PR-tier counter and allocation gates live in CI.
- On-demand stable-hardware workflow (
workflow_dispatchonly) and a matching local target, feeding a public history with regression detection. Noschedule:trigger. - Release-tier report generated automatically and attached to each release.
- Existing harnesses (
tests/performance,tests/resources,BenchmarkProfileScanner) folded into the tiers rather than duplicated.
Related: #39 (optimization campaign methodology), #73.
Part of the 1.0 quality program: #83. Epic: #81.
- 主要言語
- Go
- スター
- 0
- フォーク
- 0
- 平均マージ
- 11時間 3分
- マージ済み PR（30日）
- 5
環境構築
- Dockerfile または Docker Compose ファイルあり
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
war-and-code/dircue のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 86/100
war-and-code/dircue#200 ·
-
enhancement follow-up
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
war-and-code/dircue#201 ·
-
follow-up
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
war-and-code/dircue#179 ·
-
follow-up
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
war-and-code/dircue#178 ·
-
enhancement follow-up
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
war-and-code/dircue#172 ·
war-and-code/dircue の issue をすべて見る
似ている issue
-
agent-butler-finding bug
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
jordansmall/spindrift#6414 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
Icinga/icinga-kubernetes#294 ·
メンテナーはふだん 4 日以内に返信
-
Cluster details: the Control Plane version field has a different border color than its icon boxオープンkind/design sig/ui
難易度 1/5 1時間未満 初心者へのやさしさ 92/100
kubermatic/dashboard#8449 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
filecoin-project/curio#1632 ·
メンテナーはふだん 1 日以内に返信