Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

eval gate is a source reader: intercept spawns in the eval kernels instead

Aperta
  • Qualcosa non va?
#13 6 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
bun, javascript, python, typescript
Ambito
security

Direzione di ricerca

Start in packages/coding-agent/src/eval/py and packages/coding-agent/src/eval/js, then read eval/agent-bridge.ts and eval/tool-bridge.ts to understand the existing host bridge. Implement execution-time interception for the listed Python and JavaScript spawn APIs through that bridge, then perform the required binary rebuild and verify the fork remains compatible with upstream merges.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement ready-for-human

PR #12 gates eval's subprocess spawns by reading the submitted source and extracting every spawn whose command is written as a literal. That closes the case that shows up in transcripts, where an agent blocked at bash rewrites the same command in Python. It does not close the case where the code hides the spawn from a source reader.

A command assembled at runtime is handled: it reads as an opaque site and raises a permission request. The gap is a spawn the reader never sees at all, which fails open:

getattr(os, "sys" + "tem")(cmd)
__import__("os").system(cmd)
exec(compile(payload, "<s>", "exec"))

globalThis[["child", "_process"].join("")]

None of those match a callee name, so no site is produced and nothing is judged.

The fix

Intercept spawns inside the eval kernels rather than in the source. Every spawn then calls back to the host classifier at execution time, with the real argv, and there is nothing to hide from.

Touch points in the OMP fork:

  • packages/coding-agent/src/eval/py — patch subprocess and the os.system/os.exec* family in the Python kernel.
  • packages/coding-agent/src/eval/js — patch child_process and Bun.spawn in the worker.

The runtimes already have a host bridge (eval/agent-bridge.ts, eval/tool-bridge.ts); route the classifier call over it rather than opening a second channel.

Cost

A fork edit plus a binary rebuild, and the patch has to survive upstream merges. That is why PR #12 took the plugin-sized version first.

When to do this

When session logs show agents building command strings to slip past the parser. Static extraction is the cheap version and it is enough while the observed behavior is agents taking the shortest path rather than evading a gate. A run that shows reflection or string arithmetic around a spawn is the signal that this is worth the fork.

Lingua principale
TypeScript
Stelle
0
Fork
1
Merge medio
2h 10m
PR unite (30g)
64

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di STRML/omp-classifier

Tutte le issue di STRML/omp-classifier

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.