feat(runtime): option to isolate local runs from the user's own agent setup

Aperta
#404 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
32/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
go
Ambito
devtools

Direzione di ricerca

Start in internal/runtime/runtime.go, where mergeEnv builds the environment for the none runtime from os.Environ(), then read internal/agent/claude_code.go to see how the Skill is installed into the workspace. Agree on the setting name and the per-case temporary home with the maintainers before coding, since the proposal leaves the design open. Done means an opt-in setting gives each case its own HOME and engine config directory, removed with the workspace unless --no-delete is set.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem / motivation

On environment.type: none, every agent command inherits the host environment: mergeEnv starts from os.Environ() (runtime.go#L31). So the agent runs with the user's real home directory and loads the user's own setup into the eval: settings, MCP servers, memory files such as ~/.claude/CLAUDE.md or ~/.codex/AGENTS.md, and user-level Skills.

That can make the with/without comparison wrong:

  1. The without_skill arm can still have the Skill. A Skill author often has the Skill installed globally while developing it, and the README installs skill-upper globally (npx skills add ... -g). A user-level copy is visible in both arms.
  2. The with_skill arm can run a different copy. skill-up installs the Skill under test into the workspace's .claude/skills (claude_code.go#L43). Claude Code's docs say a personal Skill wins over a project Skill with the same name: "With deploy in both ~/.claude/skills/ and the project's .claude/skills/, /deploy runs the personal one" (Claude Code docs). So an edited copy under evaluation can lose to an older installed copy.
  3. Results depend on whose machine ran them, and runs write into the user's own agent state (#360, #366).

I have not reproduced point 2 with a model run. It follows from the install path and the Claude Code docs.

Users already work around this by hand:

  • powercontext's eval README: "Use a dedicated evaluation account and profile: Claude inherits user settings, native memory, and global Skills/MCP." (README)
  • I ran my own evals with a temporary HOME for the same reason.

Proposed solution

An opt-in setting for the none runtime, for example environment.isolate_home: true (name open). For each case it would:

  1. Create a temporary home directory next to the case workspace, and point HOME and the engine's own config variable at it (CLAUDE_CONFIG_DIR, CODEX_HOME, and so on).
  2. Carry over only what the engine needs to log in. API keys in the environment pass through as today. Each adapter decides which credential files to copy or link.
  3. Remove it together with the workspace, keeping it under --no-delete.

Docker and OpenSandbox already start from a clean home, so they would not change.

Login storage differs per engine: some keep it in files under the home directory, others in the OS keychain. An engine whose login cannot be carried over should fail early with a clear message instead of running logged out. I can start with Claude Code and Codex, which I can test.

Alternatives considered

  • Run in Docker or OpenSandbox. That isolates, but it is heavier, and the docs recommend none for plain-text Skills.
  • Clean up after the run (#394, #397). That removes leftovers, but the run itself still reads the user's setup.
  • Tell users to use a separate OS account. That works, but it puts the setup on every user.

Additional context

  • #267 (closed as not planned) was about cleaning up after a run. This issue is about what the agent reads during the run.
  • #360 and PR #398 stop the Codex adapter from writing mocked servers into the user's global config.
Lingua principale
Go
Stelle
1.5k
Fork
134
Merge medio
1g 22h
PR unite (30g)
61

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di alibaba/skill-up

Tutte le issue di alibaba/skill-up

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.