feat(bench): track benchmark and memory results over time instead of a single-run gate
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 12/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 静か
- 技術スタック
- github, github-actions, go
調査の方向性
Start with the regression job in .github/workflows/bench.yml and the baseline in tools/bench/baseline.json, then find CheckRegression to see why it stops at the first failure. The issue asks for an OpenSpec change to PERF-02 and INDX-06 before any code, and the tool choice (github-action-benchmark or Bencher) is left to the spec. Done means a spec is agreed, the tool is picked, and a per-phase metrics output is designed, which is design work rather than a first contribution.
索引モデルが issue の本文から書いたものです。
説明
What exists today?
The regression job in .github/workflows/bench.yml runs each night. It indexes the synthetic corpus three times, takes the median, and compares it with one stored baseline in tools/bench/baseline.json. It fails at a 10% throughput drop, at 15% more peak RSS, or above 4 GiB. It never blocks a merge. The publish job reports absolute numbers each week.
How does it behave now?
The gate measures the host as much as the code, and it keeps no history.
- Two runs on the same commit differed by 21% in throughput (#252).
- After #250, the three trials of one night spread by up to 12% in peak RSS.
- The job failed every night from 3 to 7 November. On three of those nights, its message named only throughput, because
CheckRegressionreturns at its first failure. The 40% RSS growth from #250 stayed hidden. - No one can see when a number started to move, or which phase of an index moved it. The #265 investigation rebuilt that by hand from profiles.
How should it behave?
- Each nightly run and each dispatch records its results as a time series. The series holds throughput, peak RSS, query latency, node and edge counts, and numbers per phase of an index (extract, resolve, write): bytes allocated, live heap at the end of the phase, GC cycles and batch bytes.
- The binary reports the per-phase numbers from
runtime/metrics, so the runner reads them and no profile is needed. - A page shows the series as charts. An alert fires on a change that lasts over several runs, with a statistical threshold, and not on one run.
- Optional: a pull request that changes Go code gets a report of a paired comparison. The job builds the base and the head on one runner, alternates their trials, and compares them with
benchstat. The report does not block the merge. - The tool comes from OSS. The candidates are
github-action-benchmarkand Bencher. The spec picks one, and the implementation checks its current version.
This issue replaces #252.
What kind of improvement?
Performance
Would this change existing behavior?
Yes. The nightly job stops being a pass or fail result from one run against one baseline. That changes PERF-02 and INDX-06, so it needs an OpenSpec change first. If the binary reports per-phase numbers through a new flag or output, that flag is a change to the CLI surface, and the spec names it.
🤖 Generated with Claude Code
- 主要言語
- Go
- スター
- 0
- フォーク
- 0
- 平均マージ
- 7時間 42分
- マージ済み PR(30日)
- 111
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
seanb4t/codegraph-go のほかの issue
-
chore needs-triage
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
seanb4t/codegraph-go#380 ·
メンテナーはふだん 1 日以内に返信
-
chore needs-triage
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
seanb4t/codegraph-go#379 ·
メンテナーはふだん 1 日以内に返信
-
area:query chore needs-triage spec-needed
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
seanb4t/codegraph-go#377 ·
メンテナーはふだん 1 日以内に返信
-
chore needs-triage
難易度 1/5 1〜3時間 初心者へのやさしさ 84/100
seanb4t/codegraph-go#362 ·
メンテナーはふだん 1 日以内に返信
-
area:query bug needs-triage
難易度 2/5 1〜3時間 初心者へのやさしさ 80/100
seanb4t/codegraph-go#355 ·
メンテナーはふだん 1 日以内に返信
seanb4t/codegraph-go の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 77/100
メンテナーはふだん 1 日以内に返信
-
automated documentation maintenance
難易度 1/5 1〜3時間 初心者へのやさしさ 75/100
github/gh-aw-mcpg#16717 ·
メンテナーはふだん 1 日以内に返信
-
Docs: Recording rules pages don't say what happens to a recorded metric when its rule is deletedオープンinternal
難易度 1/5 1〜3時間 初心者へのやさしさ 84/100
メンテナーはふだん 1 日以内に返信
-
enhancement
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
jesseduffield/lazygit#6229 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
prometheus/prometheus#20482 ·
メンテナーはふだん 2 日以内に返信