perf(bench): add a production readEvents scaling benchmark
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 30/100
- Issue 類型
- 功能
- 描述清晰度
- 基本清楚
- 活躍度
- 活躍
- 技術堆疊
- nodejs, typescript
- 領域
- performance, testing
研究方向
Start from the existing runner in scripts/run-benchmarks.mjs and the built-in cases in packages/squad-sdk/src/runtime/benchmarks.ts, then find LocalSquadBackend.readEvents for the local file-state backend. Add the six fixture cases (100/1,000/10,000 events, plus limit 1 with one-event and empty tails) to the existing runner and keep its output shape compatible. Done when the built method runs with 2 warmups and 3 rounds of 10 timed samples, a deliberately broken result fails the run, and a test-only 2x slowdown exceeds the comparison threshold.
由索引模型根據 Issue 內容生成。
描述
Summary
Add one production readEvents scaling workload to the existing performance runner. The current built-in configLoad, charterCompile, routing, model-selection and export/import cases largely simulate JSON/string/regex work rather than invoke the corresponding product implementations.
Proposed primary owner: unit-contract-tester.
Evidence and scope
Audited revision: 47082a81799710c3e457b594123d190c00596ebf, observed 2026-11-12.
- Built-in benchmark cases were exercised for three baseline runs of 1,000 iterations per case. Their output is not evidence of equivalent production-path performance.
- Existing entry points include
scripts/run-benchmarks.mjsandscripts/measure-cold-start.mjs. No workflow invocation of those performance runners was found at the audited revision. Smoke/output-shape tests do not demonstrate sensitivity to realistic product regressions. - The local packaged build could not run: TypeScript/Vitest were unavailable and locked dependency restoration returned a configured-feed E404 for
hono-4.13.17.tgz. Missingdistalso blocked the built benchmark and cold-start runners. This is a measurement limitation, not evidence that the CLI is slow.
The audit's fallback used Node's built-in TypeScript transformation and isolated relative-import resolution. Those function-level diagnostics exclude import/setup time and must not certify packaged startup, application throughput, total allocations, token efficiency, or whole-agent-session memory.
Scope
Parent: github/squad#2481.
One production API only: LocalSquadBackend.readEvents with the local file-state backend. Add a fixture/case to the existing runner foundation; do not build a new benchmark framework. Packaged help/version startup is split into github/squad#2482.
Out of scope: config, charter, routing, model selection, memory search, shell persistence, backend optimization, CI gates, nightly automation and cross-platform rollout. The optimization belongs to github/squad#2479.
Measurable acceptance and tests
- Run the built production method, not a JSON/string simulation or source-loader shim. Fixture sizes: 100/1,000/10,000 valid events;
limit: 1with one-event-tail and empty-tail cursors, six cases total. - For each case, collect three rounds of ten timed observations after two untimed warmups. Exclude fixture/import setup and retain all 30 raw times plus p50/p95, exact SHA, Node/platform/build mode and cache policy.
- Assert event count, final sequence, cursor and resumability in the fixtures. A deliberately broken result must fail the run rather than produce a successful timing.
- Deterministic output tests cover sample counts/quantiles, nonfinite or missing data, fixture failure and a nonzero runner exit. Existing runner consumers/output contracts remain compatible.
- Exercise the comparison/reporting path with a calibrated test-only slowdown of at least 2x; its reported median ratio must exceed the declared comparison threshold. This proves regression sensitivity, not production optimization benefit; do not add a blocking CI threshold in this issue.
- Check in the portable fixture and document one exact build/run command with a passing sample artifact. Missing dependencies/dist remain a failing or blocked measurement, never a successful substitute.
Done means one trustworthy six-case production benchmark and its runner/result tests, not coverage of every product path.
Related work and non-goals
Follow-up to the already-merged runner foundation github/squad#1111, not another request to add it. github/squad#2449 is a distinct model-quality paired A/B workload.
Methodology: dotnet/aspnetcore#70219, microsoft/waza#846, and the packaged-path correction in microsoft/aspire#16704. No product optimization or end-to-end speedup has been demonstrated by this issue.
- 主要語言
- TypeScript
- 星號
- 3.4k
- 分支
- 516
- 平均合併
- 1 天 15 小時
- 30 天內合併 PR
- 116
環境準備
- 提供 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
github/squad 的其他 Issue
-
bug
難度 2/5 1-3 小時 新手友好度 84/100
維護者通常 1 天內回覆
-
bug
難度 2/5 1-3 小時 新手友好度 77/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 82/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 72/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 82/100
vercel-labs/ui#809 ·
維護者通常 1 天內回覆
-
bot:ai-assisted component:platform-js status:untriaged
難度 1/5 1 小時以內 新手友好度 74/100
midnightntwrk/platform-js#38 ·
-
[bug] Second false positive on high-cost plugin alert: detection only compares patch entry id, community plugin dsh-auto | [bug] 高成本插件告警第二次误报:检测只比对 patch 条目 id,社区插件 dsh-auto-review 被认成 @deepseek-ai/dsh-experimental-auto-review可能已有人在做 關聯的 PR 仍在進行中或已合併。 未關閉
難度 2/5 1-3 小時 新手友好度 68/100
ranxianglei/billion-context#7068 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
mybigday/whisper.rn#348 ·
-
難度 2/5 1-3 小時 新手友好度 78/100
six-group/six-webcomponents#600 ·