Workbench: live measurements leave no versioned trace
維護者通常 1 天內回覆
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 48/100
- Issue 類型
- 功能
- 描述清晰度
- 基本清楚
- 活躍度
- 活躍
- 技術堆疊
- typescript
研究方向
Start with workbench/README.md, especially “La prise réelle,” then inspect baseline.ts and workbench/lib/cassette.ts. The issue calls for manually verified --update-baseline runs for the real-take scenarios, followed by capturing the provider’s usage block in the recording proxy. Done should include tracked, verified measurements and a clear decision in the README about whether judged-PR cassettes should be committed.
由索引模型根據 Issue 內容生成。
描述
Split out of #428, which is being closed — its core ask is done, this is the adjacent point that was never in scope for it.
What's left
workbench/runs/ and workbench/reports/ are gitignored, and only three baselines are committed (consent.json, describe-project.json, wizard-enhance.json). Every live measurement made against this bench — including the ones behind #431, #432, #440, #452, #453, #454 and #456 — leaves no trace anyone else can replay without re-running it themselves, on their own key, against whatever the provider resolves that day.
Concretely, that means:
- A cassette recorded during a PR review (e.g.
workbench/cassettes/judge-migration-*.json) replays offline for whoever has the file, but nothing forces it to reachmain— it's a byproduct of one session, not a tracked measurement. - The three real-take scenarios (
real-wizard-enhance,real-zooms,real-zoom-grounding) have never been judged at all, because the take itself (workbench/fixtures/) is gitignored and absent from every clone. See #457's sibling problem andworkbench/README.md§ "La prise réelle" for what that costs on a fresh checkout (44 L0 tests fail on the missing fixture, silently, since CI never runs the bench). - A baseline's
behaviour/dslnumbers, once written, are never re-verified against anything — nothing reads them back (baseline.tsonly compares check ids), so they drift into being an unlabelled archive, which is exactly what happened towizard-enhance.jsonbefore #453 said so explicitly.
First step, as noted when this was flagged
Freeze the real-take scenario baselines (--update-baseline, verified by hand rather than trusted), then capture the provider's usage block in the recording proxy (workbench/lib/cassette.ts) — the token count every context-size argument in this codebase wants is already in the response the recorder sees pass through, uncaptured.
Also worth deciding here, not in #428
Whether cassettes from a judged PR review should be committed as a matter of course (the way judge-migration-*.json were, by convention rather than by rule) or whether that's noise the repo doesn't want. workbench/README.md doesn't currently say either way.
- 主要語言
- TypeScript
- 星號
- 3.7k
- 分支
- 231
- 平均合併
- 16 小時 38 分鐘
- 30 天內合併 PR
- 327
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
getopenscreen/openscreen 的其他 Issue
-
難度 2/5 1-3 小時 新手友好度 78/100
getopenscreen/openscreen#1423 ·
維護者通常 1 天內回覆
-
Native review wanted: machine-translated recording bar tooltips (14 locales)可能重新可做 @auberginewly 於 21 天前認領,目前沒有進行中的 PR。 未關閉good first issue help wanted
難度 2/5 1-3 小時 新手友好度 68/100
getopenscreen/openscreen#913 · 5 則留言 ·
維護者通常 1 天內回覆
-
QA: e2e pass on v2.2.0-rc.1 (macOS, Windows)可能已有人在做 關聯的 PR 仍在進行中或已合併。 未關閉
難度 2/5 1-3 小時 新手友好度 55/100
getopenscreen/openscreen#1431 · 1 則留言 ·
維護者通常 1 天內回覆
-
bug
難度 3/5 半天 新手友好度 68/100
getopenscreen/openscreen#1428 · 1 則留言 ·
維護者通常 1 天內回覆
-
enhancement
難度 5/5 一週以上 新手友好度 35/100
getopenscreen/openscreen#1426 ·
維護者通常 1 天內回覆
查看 getopenscreen/openscreen 的全部 Issue
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 82/100
better-auth/better-auth#12132 · 1 個 reaction ·
維護者通常 1 天內回覆
-
o8 next debug build aborts on Glass to Dark appearance switch (nil identifier messaged before check)未關閉bug
難度 2/5 1-3 小時 新手友好度 78/100
hurttlocker/o8#5112 ·
-
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
webrecorder/wabac.js#386 ·
-
needs-triage
難度 1/5 1-3 小時 新手友好度 92/100
agent-of-empires/agent-of-empires#5098 · 1 則留言 · 1 個 reaction ·
維護者通常 1 天內回覆