eval gate is a source reader: intercept spawns in the eval kernels instead
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 35/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- bun, javascript, python, typescript
- 领域
- security
调研方向
Start in packages/coding-agent/src/eval/py and packages/coding-agent/src/eval/js, then read eval/agent-bridge.ts and eval/tool-bridge.ts to understand the existing host bridge. Implement execution-time interception for the listed Python and JavaScript spawn APIs through that bridge, then perform the required binary rebuild and verify the fork remains compatible with upstream merges.
由索引模型根据 Issue 内容生成。
描述
PR #12 gates eval's subprocess spawns by reading the submitted source and extracting every spawn whose command is written as a literal. That closes the case that shows up in transcripts, where an agent blocked at bash rewrites the same command in Python. It does not close the case where the code hides the spawn from a source reader.
A command assembled at runtime is handled: it reads as an opaque site and raises a permission request. The gap is a spawn the reader never sees at all, which fails open:
getattr(os, "sys" + "tem")(cmd)
__import__("os").system(cmd)
exec(compile(payload, "<s>", "exec"))
globalThis[["child", "_process"].join("")]
None of those match a callee name, so no site is produced and nothing is judged.
The fix
Intercept spawns inside the eval kernels rather than in the source. Every spawn then calls back to the host classifier at execution time, with the real argv, and there is nothing to hide from.
Touch points in the OMP fork:
packages/coding-agent/src/eval/py— patchsubprocessand theos.system/os.exec*family in the Python kernel.packages/coding-agent/src/eval/js— patchchild_processandBun.spawnin the worker.
The runtimes already have a host bridge (eval/agent-bridge.ts, eval/tool-bridge.ts); route the classifier call over it rather than opening a second channel.
Cost
A fork edit plus a binary rebuild, and the patch has to survive upstream merges. That is why PR #12 took the plugin-sized version first.
When to do this
When session logs show agents building command strings to slip past the parser. Static extraction is the cheap version and it is enough while the observed behavior is agents taking the shortest path rather than evading a gate. A run that shows reflection or string arithmetic around a spawn is the signal that this is worth the fork.
- 主要语言
- TypeScript
- 星标
- 0
- 派生
- 1
- 平均合并
- 2 小时 10 分钟
- 30 天内合并 PR
- 64
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
STRML/omp-classifier 的其他 Issue
-
Decide whether a coordinator may lift a headless worker's refusal (the trust boundary #68 defers)未关闭enhancement ready-for-human
难度 5/5 一周以上 新手友好度 25/100
STRML/omp-classifier#142 ·
维护者通常 1 天内回复
-
enhancement ready-for-human
难度 4/5 3-5 天 新手友好度 45/100
STRML/omp-classifier#116 · 7 条评论 ·
维护者通常 1 天内回复
查看 STRML/omp-classifier 的全部 Issue
相似的 Issue
-
refactor
难度 2/5 半天 新手友好度 84/100
维护者通常 5 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
OHDSI/Data2Evidence#3450 ·
维护者通常 2 天内回复
-
e2e-failure ready-to-code
难度 2/5 1-3 小时 新手友好度 90/100
redhat-developer/rhdh-plugin-export-overlays#4011 · 1 条评论 ·
维护者通常 1 天内回复
-
automation missing-model model-sync provider:ofox
难度 2/5 1-3 小时 新手友好度 72/100
anomalyco/models.dev#8421 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 1 天内回复