Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

perf(query): sustain dashboard reads and bound peak RSS during mixed ingest

オープン
  • 何か問題がありますか？
#278 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
25/100
issue の種類
バグ
明瞭さ
説明が足りない
活発さ
活発
技術スタック
go, linux
領域
backend, databases, observability, performance

調査の方向性

Start with docs/benchmarks/duckdb2-review-2026-10-01.md and its retained harness patch, then rerun the fixed five-minute Linux ARM64 preset with the gauge and actual timestamps. Profile the named ranking/file-binding/scanning, analytical service/edge, publication scheduling, and retained transaction-memory work. Done means the sustained 20-read/s gate passes with zero errors or shedding and bounded cache backlog, with RSS, accepted rate, CPU per row, and latency reported separately.

索引モデルが issue の本文から書いたものです。

説明

PR #272 review follow-up: https://github.com/labstack/fanout/pull/272#discussion_r4158977357 and https://github.com/labstack/fanout/pull/272#discussion_r4158977411

The completed-batch cache now keeps up with sustained ingestion, but the full dashboard-read capacity gate and peak process memory remain unresolved. Do not equate cache backlog stability or a smaller persisted catalog with sustained read capacity or lower peak RSS.

The fixed Linux ARM64 trial uses 4 CPUs / 6 GiB, same-host telemetrygen 0.157.0, nominal 120k offered rows/s, 20 open-loop reads/s, a 300.154-second mixed phase, two-second requested sampling and thirty-second cooldown. Fanout accepted 66,086 rows/s; the target offered rate was not actually sustained. Source and binary identities, complete harness patch, raw measured JSON, and failed iterations are retained in the review report.

  • Pending batches: minute medians 7, 11, 12, 16, 20.5; sampled maximum 63; last sustained sample 16; manual cooldown observation 0. The preceding iteration ended at 1,690 pending batches.
  • Successful reads: 4,931 (16.4/s); 19 failed and 1,099 shed. Successful-client p50/p95/p99: 88.12ms / 5.955s / 10.630s.
  • Server-handler p50 deteriorates from 32.9ms in minute one to 4.45s in minute five despite the stable cache backlog. Those values come from request logs grouped by request start, not client latencies.
  • Peak process RSS: 3.32 GiB; cooldown RSS: 0.98 GiB. The preceding failed iteration peaked at 3.03 GiB. The smaller cache does not establish a lower process peak.
  • Identical persisted-cache fixture: catalog 180,891,648 -> 70,004,736 bytes (-61.3%), with identical 52,287,828-byte Parquet data. The final catalog remains 1.34x the compressible fixture.
  • Generator exits, dropped-row counters, restarts and full authoritative storage verification pass; the sustained-query-load gate fails.

Profile the remaining ranking/file-binding/scanning work, analytical service/edge work, publication scheduling and retained transaction memory before assigning a cause. Use the fixed five-minute preset with the gauge and actual timestamps, matched accepted data where possible, and a separate-generator run to distinguish contention from backend limits. Preserve exact scoped/clipped results, stable snapshots, typed format 3 and bounded storage publication. Success requires the sustained 20-read/s gate to pass with zero errors/shedding and cache backlog bounded; separately report peak/cooldown RSS, accepted rate, CPU per row and query latency. Do not improve a benchmark by changing its admission/workload limits without disclosing that change.

This issue is separate from #277's ingestion encoding CPU regression. Both remain open after the cache follow-up; neither should be closed by merging #272.

主要言語
Go
スター
6
フォーク
0
平均マージ
2時間 25分
マージ済み PR（30日）
25

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

labstack/fanout のほかの issue

labstack/fanout の issue をすべて見る

似ている issue

Go の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。