Scope the builder Pre-Execution Protocol to task shape
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 66/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- sql, typescript
调研方向
从 packages/opencode/src/altimate/prompts/builder.txt 开始，检查现有的 SessionTermination.completionInstruction 先例，了解针对运行模式的 prompt 注入。追踪如何确定 headless mode、builder agent 以及是否存在 dbt 项目。完成标准是：仅在没有 dbt 项目的 headless builder 运行中省略 Pre-Execution Protocol，在所有其他情况下都保留它，包括未分类的 workspace。
由索引模型根据 Issue 内容生成。
描述
Problem
packages/opencode/src/altimate/prompts/builder.txt carries a ## Pre-Execution Protocol section that makes sql_analyze + altimate_core_validate mandatory before every sql_execute. builder is a PRIMARY agent, so that section reaches every builder surface at once: dbt authoring, interactive chat, and headless question-answering runs.
An internal pre-registered paired prompt ablation (540 trials on a public data-question benchmark, one binary across both arms, per-trial system prompts verified by sha256) measured what the section costs on the question-answering surface:
| macro Pass@1 | |
|---|---|
| control | 0.6667 |
| treatment (protocol removed, among other changes) | 0.6807 |
| delta | +0.0140 |
- query-blocked sign-flip permutation, 20,000 resamples: p = 0.7358
- cluster-bootstrap 95% CI: [-0.0400, +0.0674]
That is a null on score. The point estimate is positive and the interval comfortably contains zero — it is not an improvement and must not be reported as one.
What did move:
| control | treatment | delta | |
|---|---|---|---|
| wall clock per trial (restricted mean) | 440.9s | 319.4s | -27.6% |
| model turns per trial | 11.9 | 8.6 | -27.7% |
| generation seconds per trial | 187.6 | 127.2 | -32.2% |
altimate_core_validate calls (arm total) |
1,476 | 0 | -1,476 |
sql_analyze calls (arm total) |
1,329 | 0 | -1,329 |
sql_execute calls (arm total) |
1,716 | 2,554 | +49% |
| trials hitting the 900s timeout | 27 | 16 | -11 |
The 2,805 ritual tool calls going to zero is the number directly attributable to this text. The ritual is prompt-ordered: deleting the order deletes it completely, not partially. The freed budget went into actual querying (sql_execute +49%).
Why not just delete it
Two reasons, both from the experiment's own caveats:
- The latency win is not attributable to this section alone. The treatment arm bundled five coupled changes; the experiment explicitly declined to attribute and a factorial was not run.
- The measurement covers data questions only. dbt authoring and interactive chat are unmeasured builder surfaces where a pre-execution discipline may genuinely earn its place. Deleting on this evidence would over-generalise from one workload.
Proposal
Scope the section to task shape rather than delete it, following the precedent already in the tree: SessionTermination.completionInstruction moved a run-mode-only instruction out of builder.txt and injects it only when headless AND the agent is builder.
Drop the protocol only in the cell that was measured (headless + builder + no dbt project in the workspace) and keep it everywhere else, including whenever the workspace cannot be classified. The cost of keeping it is 27% latency on one workload; the cost of wrongly dropping it is unmeasured.
Out of scope
main also carries a ## Finish Protocol section (shipped in #1171) — a second mandatory ritual in the same family, added after the binary the ablation measured was built. No measurement covers it and this issue does not propose touching it.
A follow-up measurement on dbt tasks is needed before the gate could be widened, or the section deleted outright.
- 主要语言
- TypeScript
- 星标
- 815
- 派生
- 135
- 平均合并
- 1 天 12 小时
- 30 天内合并 PR
- 53
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
AltimateAI/altimate-code 的其他 Issue
-
test: MCP tests fail when the developer's ~/.claude.json has MCP servers (HOME is not sandboxed)未关闭
难度 2/5 1-3 小时 新手友好度 78/100
AltimateAI/altimate-code#1386 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
AltimateAI/altimate-code#1384 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 74/100
AltimateAI/altimate-code#1378 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
AltimateAI/altimate-code#1359 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
AltimateAI/altimate-code#1323 ·
维护者通常 1 天内回复
查看 AltimateAI/altimate-code 的全部 Issue
相似的 Issue
-
Mend: dependency security vulnerability untriaged
难度 2/5 1-3 小时 新手友好度 68/100
opensearch-project/security-dashboards-plugin#2545 ·
维护者通常 1 天内回复
-
Add: Dream TR SD未关闭check:passed streams:add
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 1 天内回复
-
doctor integrity sample scans soft-deleted pages on Postgres (batch path has no deleted_at filter)未关闭
难度 2/5 1-3 小时 新手友好度 88/100
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 72/100
SocialGouv/egapro#4672 · 1 条评论 ·
维护者通常 2 天内回复
-
area:agents area:tui bug
难度 2/5 1-3 小时 新手友好度 74/100
anthropics/claude-code#98358 ·
维护者通常 1 天内回复