feat(runtime): option to isolate local runs from the user's own agent setup
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 32/100
Research direction
Start in internal/runtime/runtime.go, where mergeEnv builds the environment for the none runtime from os.Environ(), then read internal/agent/claude_code.go to see how the Skill is installed into the workspace. Agree on the setting name and the per-case temporary home with the maintainers before coding, since the proposal leaves the design open. Done means an opt-in setting gives each case its own HOME and engine config directory, removed with the workspace unless --no-delete is set.
Written by the indexing model from the issue text.
Description
Problem / motivation
On environment.type: none, every agent command inherits the host environment: mergeEnv starts from os.Environ() (runtime.go#L31). So the agent runs with the user's real home directory and loads the user's own setup into the eval: settings, MCP servers, memory files such as ~/.claude/CLAUDE.md or ~/.codex/AGENTS.md, and user-level Skills.
That can make the with/without comparison wrong:
- The without_skill arm can still have the Skill. A Skill author often has the Skill installed globally while developing it, and the README installs skill-upper globally (
npx skills add ... -g). A user-level copy is visible in both arms. - The with_skill arm can run a different copy. skill-up installs the Skill under test into the workspace's
.claude/skills(claude_code.go#L43). Claude Code's docs say a personal Skill wins over a project Skill with the same name: "Withdeployin both~/.claude/skills/and the project's.claude/skills/,/deployruns the personal one" (Claude Code docs). So an edited copy under evaluation can lose to an older installed copy. - Results depend on whose machine ran them, and runs write into the user's own agent state (#360, #366).
I have not reproduced point 2 with a model run. It follows from the install path and the Claude Code docs.
Users already work around this by hand:
- powercontext's eval README: "Use a dedicated evaluation account and profile: Claude inherits user settings, native memory, and global Skills/MCP." (README)
- I ran my own evals with a temporary
HOMEfor the same reason.
Proposed solution
An opt-in setting for the none runtime, for example environment.isolate_home: true (name open). For each case it would:
- Create a temporary home directory next to the case workspace, and point
HOMEand the engine's own config variable at it (CLAUDE_CONFIG_DIR,CODEX_HOME, and so on). - Carry over only what the engine needs to log in. API keys in the environment pass through as today. Each adapter decides which credential files to copy or link.
- Remove it together with the workspace, keeping it under
--no-delete.
Docker and OpenSandbox already start from a clean home, so they would not change.
Login storage differs per engine: some keep it in files under the home directory, others in the OS keychain. An engine whose login cannot be carried over should fail early with a clear message instead of running logged out. I can start with Claude Code and Codex, which I can test.
Alternatives considered
- Run in Docker or OpenSandbox. That isolates, but it is heavier, and the docs recommend
nonefor plain-text Skills. - Clean up after the run (#394, #397). That removes leftovers, but the run itself still reads the user's setup.
- Tell users to use a separate OS account. That works, but it puts the setup on every user.
Additional context
- #267 (closed as not planned) was about cleaning up after a run. This issue is about what the agent reads during the run.
- #360 and PR #398 stop the Codex adapter from writing mocked servers into the user's global config.
- Dominant language
- Go
- Stars
- 1.5k
- Forks
- 134
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 61
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from alibaba/skill-up
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
OpenSandbox runtime: expect.files_exist and file_contains fail because WorkspacePath is remotePossibly taken @bababoom721 claimed this 1 day ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 25/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
alibaba/skill-up#408 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
alibaba/skill-up#407 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
alibaba/skill-up#406 · 1 comment ·
Maintainers usually reply within 1 day
All issues in alibaba/skill-up
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
OneBusAway/maglev#2145 ·
Maintainers usually reply within 2 days
-
priority/medium status/ready type/chore
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
discobox-ai/discobox#1357 ·
Maintainers usually reply within 1 day
-
TestPseudoLocaleRemote fails now and then: it types the code before the laptop asks for itPossibly taken @fzn0x claimed this today. Openbug priority: low
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
edit_file diff rendering panics on invalid UTF-8 textPossibly taken @TDnorthgarden claimed this today. Openarea/tui
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
docker/docker-agent#4955 ·
Maintainers usually reply within 1 day
-
enhancement ui/ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
jbigler/railswitch#243 ·
Maintainers usually reply within 1 day