bug(codex): mocked MCP servers are written into the user's global ~/.codex/config.toml and left there

Aperta
#360 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
30/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
go

Direzione di ricerca

Start in internal/agent/codex.go: the install path around lines 146-149 (codex mcp remove then codex mcp add) and the session lookup near line 1130 that hard-codes $HOME/.codex/sessions. Reproduce by copying ~/.codex/config.toml, running a Codex case with a mocked server on environment.type: none, and diffing the file. Done when the install and the session lookup both use one per-case CODEX_HOME, and the global config is unchanged after the run.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Description

On environment.type: none, the Codex engine installs MCP servers with codex mcp add (codex.go#L149), which writes to the user's global ~/.codex/config.toml. Before adding, it runs codex mcp remove for the same name (codex.go#L146), so a user's own server with that name, say a real github server, is replaced by the mock. Nothing removes the entry after the run, so every later Codex session on that machine, outside skill-up too, starts the mocked server. Claude Code installs with --scope project, so this only affects Codex.

Steps to reproduce

  1. Copy ~/.codex/config.toml aside.
  2. Run any Codex case with a mocked server on environment.type: none. Mine:
# SKILL.md body: call get_order on the shop MCP server and report the status it returns.

# evals/fixtures/mcp/shop.yaml
tool_responses:
  get_order:
    default:
      order_id: "{{params.order_id}}"
      status: SHIPPED-7F3Q

# evals/cases/lookup.yaml
id: lookup
input:
  prompt: What is the status of order 42? Use the shop tools.
judge:
  type: rule_based
  success:
    - tool_called:
        name: mcp__shop__get_order
    - output_contains:
        all: ["SHIPPED-7F3Q"]

# evals/eval.yaml
schema_version: v1alpha1
environment:
  type: none
skills:
  - source: local_path
    path: .
mcp:
  servers:
    - name: shop
      mode: mocked
      config_ref: evals/fixtures/mcp/shop.yaml
engine:
  name: codex
  model: {}
cases:
  files: [evals/cases/lookup.yaml]
  1. Diff the config against the copy.

Expected behavior

The user's Codex config is unchanged after the run, or skill-up uses a Codex home of its own.

Actual behavior

  • A new [mcp_servers.shop] table holds the whole generated Node script (136 added lines) and stays after the run.
  • Codex rewrites the file while adding it. Empty http_headers = {} tables and an empty args = [] disappear from unrelated servers, and a trust entry for the temporary workspace is added under [projects].
  • The eval session also loads every MCP server and the global instructions from the user's own Codex setup, so results depend on the machine.

Environment

  • skill-up 0.14.0-14-g99894b3 (main, built from source)
  • Codex CLI 0.166.0, preinstalled and logged in, account default model (gpt-6-astra in every run) (skill-up's default pin is 0.80.0, codex.go#L35)
  • Go 1.26.0, macOS arm64, environment.type: none

Additional context

Setting environment.env.CODEX_HOME to a separate folder keeps the global file untouched, but the session lookup hard-codes $HOME/.codex/sessions (codex.go#L1130), so it then misses the session file. A CODEX_HOME per case runtime, used by both install and lookup, with the user's auth.json copied in, would isolate local runs. Per case rather than per run, since two cases can mock the same server name with different fixtures.

Lingua principale
Go
Stelle
1.5k
Fork
134
Merge medio
1g 22h
PR unite (30g)
61

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di alibaba/skill-up

Tutte le issue di alibaba/skill-up

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.