perf(discovery): slice-only docs reads lose recall on scattered questions
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 42/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- markdown
- 领域
- performance, search
调研方向
Start from the artifact introduced in #6348: the shared docs lookup skill /discovery:read-docs and its escalate_bytes (default 60 KB) and section-count (25%) escalation rules, plus the Phase 6 acceptance kit, note, cache and grader used for the confirmation run. Identify where the thresholds and the slice-reading procedure live, then run the measured comparison of default vs. lower threshold vs. procedure nudge vs. combination, reporting n per arm and the same interval rule as the confirmation run. Done means the slice-only versus escalated recall gap closes within noise at acceptable bytes/tokens cost, or the null result is recorded and defaults kept.
由索引模型根据 Issue 内容生成。
描述
No related issue: measured follow-up to #6348 (part of #6020).
Summary
The shared docs lookup (/discovery:read-docs, #6348) returns verbatim sections and escalates to the whole page past a size or section-count threshold (escalate_bytes, default 60 KB; 25% of sections). The Phase 6 acceptance run showed that runs reading only slices lose recall on questions whose answer is scattered across a page:
- Slice-only runs: new-core recall .838 (n=4).
- Runs that escalated to the whole page: .927 (n=6).
- The worst slice-only run scored .74; it never read 3 scattered sections.
- The same direction appeared in the original batch (.93 vs .86).
The pre-registered confirmation passed (10 fresh runs, mean .891 vs the no-cache arm's .936, lower bound -.072 against a -.10 margin) and thresholds were kept at defaults, so this is an improvement, not a regression fix.
Fix
Candidates, each to be measured before adopting:
- Lower the
escalate_bytesthreshold so more reads return the whole page. - Add a procedure nudge: when a question spans many sections, read the whole page instead of slices.
Verification
- A measured comparison on the same acceptance kit (same questions, same note and cache, same grader) of the default, each candidate, and the combination.
- Acceptance: the slice-only versus escalated recall gap closes (target: slice-only runs within the noise of escalated runs) without losing the cost benefit, measured as bytes or tokens read per run against the current default. Report n per arm and use the same interval rule as the confirmation run.
- If neither candidate closes the gap at acceptable cost, record the result and keep the defaults.
Related
- #6020 (umbrella)
- #6348 (shared docs cache, fetcher,
/discovery:read-docs)
- 主要语言
- Shell
- 星标
- 22
- 派生
- 2
- 平均合并
- 8 小时 34 分钟
- 30 天内合并 PR
- 946
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
melodic-software/claude-code-plugins 的其他 Issue
-
needs-human needs-triage
难度 2/5 1-3 小时 新手友好度 82/100
melodic-software/claude-code-plugins#7636 · 1 条评论 ·
维护者通常 1 天内回复
-
agent-ready needs-human
难度 2/5 1-3 小时 新手友好度 77/100
melodic-software/claude-code-plugins#7598 · 4 条评论 ·
维护者通常 1 天内回复
-
good first issue priority: low
难度 2/5 1-3 小时 新手友好度 62/100
melodic-software/claude-code-plugins#7530 · 1 条评论 ·
维护者通常 1 天内回复
-
needs-human priority: medium
难度 2/5 1-3 小时 新手友好度 65/100
melodic-software/claude-code-plugins#7455 · 1 条评论 ·
维护者通常 1 天内回复
-
priority: medium
难度 2/5 1-3 小时 新手友好度 68/100
melodic-software/claude-code-plugins#7351 · 2 条评论 ·
维护者通常 1 天内回复
查看 melodic-software/claude-code-plugins 的全部 Issue
相似的 Issue
-
documentation needs-triage priority:medium
难度 2/5 1-3 小时 新手友好度 85/100
yeongseon/azure-functions-validation-python#546 ·
维护者通常 1 天内回复
-
help wanted new command
难度 2/5 1-3 小时 新手友好度 72/100
tldr-pages/tldr#24755 ·
维护者通常 1 天内回复
-
afk-ok area:agents size:S
难度 2/5 1-3 小时 新手友好度 82/100
enorm-labs/event-junkie#3523 ·
维护者通常 1 天内回复
-
Container fails to start when a mounted neo4j.conf uses spaces around `=`可能已有人在做 @RaphaelFakhri 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
neo4j/docker-neo4j#596 ·
-
documentation good first issue
难度 2/5 半天 新手友好度 78/100
devdavidejesus/btc-toolkit#63 ·
维护者通常 1 天内回复