Refuse the XCTestDevices redirect when an Xcode xcrun shim (simctl or devicectl) has an armed first-launch hook: scoped simulator set deletion

Start with packages/platform-apple/src/runner/runner-device-set.ts and the shared tool definitions in packages/platform-apple/src/core/tool-provider.ts, then follow the host.ts delegates and runner error classification files named in the issue. Run the mirrored probe and runner-device-set tests; done means both simctl and devicectl hooks are evaluated fail-closed, refusal occurs before xcodebuild or runner launch, and the typed error details and classification are covered.

Purpose

Bug class: agent-device points ~/Library/Developer/XCTestDevices at a user-owned simulator set while Apple's first-launch cleanup can run. That cleanup deletes every device in the user's set (data loss).

Found while validating #2917 (see the #2878 evidence, "Live: scoped simulator set"). Reproduced on origin/main dbc08a417a, re-checked on 2db7c42a8b and 6428c54853. The code has not changed.

What happens (watcher log 2878b-watch.log, request log 2878b-state-main/sessions/s2878bm/requests/*.ndjson):

1. A cold open com.apple.Preferences --ios-simulator-device-set <set> --udid <udid> finds no cached runner (runner_xctestrun_cache reason: missing_xctestrun). It starts the runner xcodebuild build-for-testing inside withXcodebuildSimulatorSetRedirect (packages/platform-apple/src/runner/runner-artifact.ts:475), which calls acquireXcodebuildSimulatorSetRedirect (runner-device-set.ts:68). That call swaps XCTestDevices for a symlink to the user's set (runner-device-set.ts:116-119, installDeviceSetRedirect at :164).
2. At the same time, open runs xcrun simctl --set <set> launch <udid> …. xcrun simctl resolves to the Xcode shim $DEVELOPER_DIR/usr/bin/simctl, a bash script. The shim compares its hard-coded EXPECTED_VERSION (Xcode 26.2: 1051.17.7) with the installed CoreSimulator CFBundleVersion (this host: 1155.4). When they differ, the shim runs xcodebuild -runFirstLaunch before every simctl call.
3. xcodebuild -runFirstLaunch runs simctl --set ~/Library/Developer/XCTestDevices delete all. The watcher caught this at 16:44:50 as pid 67557. The cleanup list in DVTSystemPrerequisites names ~/Library/Developer/XCTestDevices. Because of the symlink, this deletes the user's scoped set. The device directory disappears about 4 s after the redirect.
4. The launch fails with Invalid device: <udid> (xcrun exit 148). The build's -destination platform=iOS Simulator,id=<udid> now names a device that no longer exists, so the build never produces Build/Products. Every cold open repeats steps 1-4. The run lost 3 throwaway devices across 3 runs (UDIDs 83D4E483…, 7D7DBA07…, 88F78C99…).

simctl is not the only shim with this hook. On the same host, $DEVELOPER_DIR/usr/bin/devicectl is a zsh script with the same pre-exec check: EXPECTED_VERSION="506.6" against the CFBundleVersion of /Library/Developer/PrivateFrameworks/CoreDevice.framework/Versions/A/Resources/Info.plist (this host: 629.3), then xcodebuild -runFirstLaunch when they differ. agent-device calls devicectl for scoped simulators during a runner session, and nothing gates that call on the scoped set:

• every simulator screenshot: captureSimulatorScreenshotWithFallback → resolveCaptureDisplay (core/screenshot.ts:141) → queryAppleDisplayInventory → devicectl device info displays --device <udid> (core/display-inventory.ts:101);
• every simulator recording: buildAppleSimulatorRecordVideoArgs → resolveAppleCaptureDisplay (simctl-facade.ts:24-26), the same devicectl call.

So a host whose CoreSimulator version matches its Xcode but whose CoreDevice version does not wipes the user's set on the first screenshot of a runner session. A gate that reads only the simctl shim would call that host safe.

Why the redirect exists: #353 / PR #354 (48aa32921d). xcodebuild resolves an id=<udid> destination only through the default set and XCTestDevices. It has no --set option. Two phases pass id=<udid> for a simulator today:

• build-for-testing: resolveRunnerBuildDestination (apple-runner-platform.ts:228-237) returns platform=<P> Simulator,id=<udid>. Whether the build needs a device at all is a separate proposal (see "Dependencies / related"); this issue does not change the build.
• test-without-building: resolveRunnerDestination (apple-runner-platform.ts:217-226) must name the booted device. runner-session.ts:286-290 acquires the redirect before launchRunnerProcess and holds it for the whole runner lifetime. It is released at :319 on a failed start and at session close. Handoff is refused while it is held (runner-session.ts:793-795, reason simulator_set_redirect), and adoption is refused for scoped sets (runner-adoption.ts:172-173). On an armed host, any call through an armed shim during that lifetime wipes the user's set: the AX-bridge simctl spawn (#2784), app launches, and the screenshot and recording devicectl calls above.

What lists the sites:

• Redirect installs. git grep -n "acquireXcodebuildSimulatorSetRedirect\|withXcodebuildSimulatorSetRedirect" -- packages src ':!**/__tests__/**'. Two production callers reach acquireXcodebuildSimulatorSetRedirect: the build, through withXcodebuildSimulatorSetRedirect (runner-artifact.ts:475 → runner-device-set.ts:68), and the session start (runner-session.ts:289). The session imports it through the re-export runner-xctestrun.ts:23 (runner-session.ts:23-33). That re-export is the production import path, and the session tests mock that module. Both callers pass through the one function, so a gate inside it covers both.
• Shims that can run first-launch. The xcrun tools agent-device names are declared by the argv types in core/tool-provider.ts: ScopedSimctlCommand (simctl) and XcrunToolArgs (devicectl | xcdevice | xctrace, :141). AppleToolProvider routes simctl and devicectl (:100-106). On Xcode 26.2, simctl and devicectl are the only files in $DEVELOPER_DIR/usr/bin that contain -runFirstLaunch. xcdevice and xctrace are Mach-O binaries with no such text. xcodebuild is run directly, not through a shim.

Required behavior

1. The redirect is refused when any xcrun shim has an armed first-launch hook (fail closed)

Rule: acquireXcodebuildSimulatorSetRedirect installs the redirect only when no Xcode shim that agent-device calls through xcrun has an armed first-launch hook. Both production callers go through this function, so the rule gates the build redirect and the session redirect alike.

Where it runs. Inside acquireXcodebuildSimulatorSetRedirect, after the lock is held and reconcileXcodebuildSimulatorSetRedirect has run, and only when needsRedirect is true (runner-device-set.ts:116). Default-set simulators return before the lock (:80-83) and a scoped set that already is XCTestDevices has needsRedirect === false. Neither pays for the probe. When the probe reports an armed shim, nothing is renamed or symlinked. The lock is given back through the existing handBackDeviceSet, and the refusal is thrown as is. It is not wrapped by redirectFailure ("Failed to redirect XCTest device set path"), because nothing failed to redirect.

Timing: the refusal comes before any xcodebuild phase. On a cold start the build path reaches the gate first (runner-artifact.ts:475 acquires before runCmdStreaming spawns build-for-testing). The refusal leaves withXcodebuildSimulatorSetRedirect before its task runs, so the build catch (runner-artifact.ts:517-537) does not rewrap it as xcodebuild build-for-testing failed. On a warm start, the session acquire (runner-session.ts:286-290) refuses before launchRunnerProcess spawns test-without-building. An armed host therefore never spends a runner build it cannot use. The progress line "Building Apple runner..." (runner-artifact.ts:251-255) is emitted before the gate and may still appear.

The shim list comes from one declaration. Add a const tuple in core/tool-provider.ts, for example XCRUN_TOOL_NAMES = ['simctl', 'devicectl', 'xcdevice', 'xctrace'] as const. Derive the XcrunToolArgs tool union from it, so a tool that runXcrun learns to run is also probed. Minimum required coverage: simctl and devicectl, the two AppleToolProvider routes (:100-106). A tool whose resolved file is not a text script, or whose text has no -runFirstLaunch, has no hook.

Per-tool probe. For each tool:

1. Locate the shim with runAppleToolCommand('xcrun', ['--find', <tool>], { allowFailure: true, timeoutMs }) (runner host.ts:184). runXcrun (host.ts:185) cannot run this: its argv type is ScopedSimctlCommand | XcrunToolArgs, and ['--find', …] is neither.
2. Read the shim text from the local filesystem. The redirect itself is a local-filesystem change on this same host (runner-device-set.ts uses node:fs).
3. When the text contains -runFirstLaunch, read both values from the shim text. Do not hard-code them. EXPECTED_VERSION="<v>" gives the expected version. The quoted path that ends in Info.plist on the CURRENT_VERSION= line gives the framework Info.plist (CoreSimulator for simctl, CoreDevice for devicectl on Xcode 26.2).
4. Read the installed CFBundleVersion with readApplePlistJson(<that path>) (runner host.ts:186).

The hook is armed when the two versions differ, or when any of them cannot be read: shim path not found, text unreadable, EXPECTED_VERSION missing, no Info.plist path on the CURRENT_VERSION line, plist unreadable, or CFBundleVersion missing. A shim that contains -runFirstLaunch but cannot be evaluated fails closed. The whole probe shares one budget of 2 s or less. A probe that times out counts as armed.

xcodebuild -checkFirstLaunchStatus is not a valid probe. It exits 0 on the reproducing host while both shims run first-launch on every call.

Data shape (a per-tool list; the gate refuses when any entry is armed):

type XcrunToolName = (typeof XCRUN_TOOL_NAMES)[number];

type XcrunShimFirstLaunchHook =
  | { tool: XcrunToolName; shimPath: string; hook: 'none' } // not a script, or no -runFirstLaunch
  | {
      tool: XcrunToolName;
      shimPath: string;
      hook: 'disarmed'; // versions equal
      expectedVersion: string;
      frameworkInfoPlistPath: string;
      installedVersion: string;
    }
  | {
      tool: XcrunToolName;
      shimPath: string | null; // null: xcrun --find failed or timed out
      hook: 'armed';
      expectedVersion: string | null; // null: unparsable
      frameworkInfoPlistPath: string | null; // null: unparsable
      installedVersion: string | null; // null: unreadable
    };

type XctestDeviceSetCleanupArming = readonly XcrunShimFirstLaunchHook[];

The probe lives in packages/platform-apple/src/core/ beside tool-provider.ts (a module named for its question, for example xcrun-shim-first-launch.ts, with a mirrored test). The runner reaches it through a new host.ts delegate, as it reaches runXcrun and readApplePlistJson today.

Refusal error. AppError('COMMAND_FAILED', …) with details.reason: 'xctest_device_set_cleanup_armed', a hint, and details.xcrunShims: XctestDeviceSetCleanupArming (every entry, so the reader sees which shim is armed and which are not). The message names each armed tool with its expected and installed versions, for example Xcode's devicectl expects CoreDevice 506.6; installed 629.3.

Reason and hint come from one declaration. Follow runner-dev-tools-security.ts:35-44. Add a row to RUNNER_ERROR_RULES (runner-error-classification.ts) that matches code: 'COMMAND_FAILED' and a typed details predicate on xcrunShims, and add the reason to RUNNER_STARTUP_FAILURE_REASONS (:154-162). The refusal builder constructs the observed error with the typed detail and calls classifyRunnerStartupFailure. It then throws with the reason and hint that the call returns, and never names either itself. The row's hint is static text. The runtime versions travel in details.xcrunShims and in the message, never in the hint. So the row and the hint stay one declaration, and the versions are still reported. The hint says two things: while the selected Xcode does not match the installed CoreSimulator or CoreDevice framework, every call through that Xcode's shim runs xcodebuild -runFirstLaunch, which deletes all devices in ~/Library/Developer/XCTestDevices; and the fix is to select the Xcode that installed those frameworks (xcode-select -s or DEVELOPER_DIR), with details.xcrunShims naming each version. Callers branch on details.reason, never on message text.

Injection for tests. Tests must not touch the host. Replace the xcrun --find step with an injected per-tool map, for example xcrunShimPaths?: Readonly<Partial<Record<XcrunToolName, string>>> beside the existing xctestDeviceSetPath, backupPath and lockDirPath options. A tool that is missing from the map is treated as "not found" (armed), so a test cannot pass by leaving a tool out. The plist path comes from the fake shim text, as in production. Tests read it through a scoped AppleToolProvider (withAppleToolProvider) whose plist.readJson is a fake, so no test needs plutil.

2. Help / docs

Update the --ios-simulator-device-set help (src/commands/schema/cli-help.ts) and website/docs/docs/commands.md. State two things, one sentence each. First: runner-backed commands refuse a scoped set with xctest_device_set_cleanup_armed when the selected Xcode does not match the installed CoreSimulator or CoreDevice framework. Second: a scoped set is not protected from a first-launch cleanup that starts outside agent-device during a live runner session (non-goal below).

Completion conditions

Unit (each must fail on origin/main):

1. Per-tool arming refusal (runner-device-set.test.ts, with injected shim paths and a fake plist reader). Every refused case also asserts that XCTestDevices is still the original directory (not a symlink), that no backup exists, and that the lock dir is released:
   • simctl EXPECTED_VERSION="1051.17.7" + -runFirstLaunch with plist 1155.4, devicectl without a hook → throws details.reason === 'xctest_device_set_cleanup_armed'
   • simctl versions equal, devicectl EXPECTED_VERSION="506.6" + -runFirstLaunch with plist 629.3 → refused, and details.xcrunShims has simctl disarmed and devicectl armed
   • both shims with equal versions → redirect installed as today
   • either shim without -runFirstLaunch (run once per tool, with the other shim at equal versions) → redirect installed
   • -runFirstLaunch present but EXPECTED_VERSION missing, or no Info.plist path on the CURRENT_VERSION line, or the plist unreadable → refused (fail closed), once per tool
   • a tool that xcrun --find cannot locate (absent from the injected map) → refused
   • the plist path is read from the shim text: a fake shim naming <tmp>/Other.framework/Info.plist reads that path (assert the fake reader's argument)
   • default-set simulator and a scoped set equal to XCTestDevices → probe never called (spy)
2. Refusal before the build. An ensureXctestrunArtifact test for a scoped-set simulator on a cache miss with an armed shim. It throws details.reason === 'xctest_device_set_cleanup_armed' and the mocked runCmdStreaming is never called (no build-for-testing). It fails on origin/main because origin/main redirects and builds.
3. Refusal before the runner launch. A session start with a reused artifact and an armed shim. It throws the reason, and launchRunnerProcess is never called (no test-without-building).
4. Reason and hint pair. HINT_FOR_REASON in runner-startup-failure-reasons.test.ts:56 (Record<RunnerStartupFailureReason, string>) gains the reason, and the type system forces it. Assert that the thrown hint equals the row's hint for two different version pairs (the hint does not change with versions) and that the message contains both versions.
5. Shim list from the declaration. A test asserts that the probe visits every XCRUN_TOOL_NAMES entry, so a tool added to the union is probed without a second edit.

Live (throwaway scoped set; never the default set, never reserved CI devices). Prepare: xcrun simctl --set <tmp-set> create … + boot, plus a fresh AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH to force missing_xctestrun. Run a watcher that logs readlink ~/Library/Developer/XCTestDevices, ls <tmp-set> and pgrep -fl "xcodebuild|simctl|devicectl" every 0.5 s. Record the probe output for both shims first: xcrun --find simctl, xcrun --find devicectl, the EXPECTED_VERSION line and the CURRENT_VERSION Info.plist path from each shim, and CFBundleVersion of each named plist.

• Leg A: armed host (the maintainer host with Xcode 26.2 qualifies: simctl 1051.17.7 vs CoreSimulator 1155.4, devicectl 506.6 vs CoreDevice 629.3):
  • Pre-fix (origin/main): the device directory disappears during open, and Invalid device is reported. This reproduces the bug.
  • Post-fix: run cold open → snapshot → screenshot → close. The runner start that the run triggers ends with details.reason: xctest_device_set_cleanup_armed, the static hint, and details.xcrunShims listing both shims as armed with the versions above. The request log shows the refusal before build-for-testing starts: the watcher never shows an xcodebuild build-for-testing process for the fresh derived path, and that path has no Build/Products. The watcher never shows XCTestDevices as a symlink. After close, the device directory is still present and simctl --set <tmp-set> list still shows the device. The screenshot step runs devicectl device info displays through the armed shim; the device must survive it. open itself may succeed, because its simctl launch is not runner-backed. If snapshot has a non-runner backend available (AX bridge), it succeeds.
• Leg B: non-armed host (every shim none or disarmed, for example a Mac whose selected Xcode installed the current CoreSimulator and CoreDevice, or the macOS CI image): cold open + snapshot + screenshot + close on a scoped set succeed. The symlink appears only while an xcodebuild phase holds the redirect, as today, and the device survives. If the implementer's host is armed, post the probe output and Leg A, and hand Leg B to the maintainer as the one remaining merge condition.
• Attach the probe output, both watcher logs and the request logs to the PR.

Gates: pnpm check:affected (or the gate docs/agents/testing.md selects for packages/platform-apple/src/runner and src/core), fallow production-unused exports, and the layering scan with the new files staged. No change to package entry points, so no eager-closure change.

Non-goals

• Changing the runner build destination or build argv. Building with generic/platform=<P> Simulator so that build-for-testing needs no redirect is a separate proposal with its own risks. See "Dependencies / related". This issue closes the reported repro without it, because the build's own acquire refuses before build-for-testing.
• Protecting a scoped set from a first-launch cleanup that something outside agent-device starts while a runner session holds the redirect (a user opening a newly installed Xcode, or running xcodebuild -runFirstLaunch by hand). The redirect is required by test-without-building, and xcodebuild has no other way to find a device in a custom set. Closing this needs a spike with two questions. Does a runner started through the redirect keep working after XCTestDevices is restored at readiness? After the restore, does CoreSimulatorService still map the XCTestDevices path to the user's devices in memory? If it does, restoring gives no protection. File that spike separately if this lands. Do not fold it in.
• Noticing a change of Xcode, CoreSimulator or CoreDevice during a live session. Each acquire probes once, and the result holds until that redirect is released. A session that holds the redirect does not probe again. Selecting another Xcode or installing new frameworks mid-session is the external-trigger case above.
• Bypassing the Xcode shims (running CoreSimulator's simctl or CoreDevice's devicectl directly) to stop agent-device from triggering first-launch. This is an unapproved compatibility change to every simctl and devicectl call, and it would still leave external triggers open.
• A shadow set with per-device symlinks, or clones in an agent-device-owned set. How CoreSimulator deletes a symlinked device is unverified, and a clone has a different UDID and different app state.
• A scoped set that is XCTestDevices (sameResolvedPath → no redirect). Apple's own cleanup already owns that set by design.
• Xcode tools that agent-device runs outside XCRUN_TOOL_NAMES (xcrun --sdk … --show-sdk-version in runner-cache-metadata.ts:286-288, xcrun --find in core/debug-symbols/symbolication.ts:105). They are not shims with a hook on Xcode 26.2. A future Xcode that adds the hook to another tool that agent-device calls through xcrun must add that tool to the declaration.
• Splitting runner-session.ts (1,353 lines, over the 1,000-line debt line). This change adds no behavior there. The only new thing that reaches its existing startup catch is the refusal error.

Constraints

• Changes stay in packages/platform-apple/src/runner/*, packages/platform-apple/src/core/* (the tool-name declaration and the probe) and help/docs. No new package subpath, so there is no ADR 0019 eager-closure or ADR 0027 budget effect. No cross-package import, so R9 layering is unchanged. The runner reaches the core probe only through runner/host.ts. ADR 0005 (runner lifecycle) is unchanged: the session redirect lifetime, the handoff refusal (simulator_set_redirect) and the adoption refusal stay as they are.
• Error behavior is keyed on details.reason (AGENTS.md: typed reasons, never message sniffing). The reason and the static hint come from the one rule row.
• The probe is a required precondition, not an optional probe: a timeout or an unreadable value refuses, and never lets the redirect through.

Estimated cost

One PR, about 1 agent-day plus the live legs. Gross diff about 300-400 lines, about 65 % of it tests. Net production about +90 to +120 lines:

• tool-name declaration + per-tool probe + host delegate: +60-80
• gate and refusal builder in runner-device-set.ts: +15-20
• rule row + reason: +10
• help/docs: +4

This fits the 1,000-line budget in one PR. No split is needed.

Dependencies / related

• Found during #2917 validation (brand the simctl builders; runXcrun requires the brand). Recorded in the #2878 evidence ("Live: scoped simulator set"). Pre-existing: not a #2878, #2898 or #2917 regression.
• Separate proposal, filed on its own: build the simulator runner with its family destination (generic/platform=<P> Simulator plus a pinned ARCHS), so build-for-testing needs no redirect (draft design-runner-build-generic-destination.md). Its reasons do not depend on this issue: the build argv would agree with the cache key (runner-cache-metadata.ts:238), the host-wide redirect lock would no longer be held through a multi-minute build while other scoped-set starts give up after 30 s (runner-device-set.ts:18), and the redirect window gets shorter. It also changes every cold simulator build, including default-set users. It stays out of this issue for that reason. That proposal is blocked by this issue: once the build no longer calls acquireXcodebuildSimulatorSetRedirect, it must keep the pre-build refusal that completion condition 2 pins.
• #2878 / #2898 / #2917: simctl set scope is enforced for every agent-device simctl argv. This issue is about the one place where agent-device changes the global XCTest set outside that scope.
• #2784 (AX bridge scoped set via simctl spawn) and #2871 (devicectl display inventory for scoped sets): the in-session simctl and devicectl calls that trigger the cleanup on an armed host. #2871's devicectl device info displays is why the gate reads the devicectl shim as well as the simctl shim.
• #353 / #354: origin of the redirect. #1672 and #2598: redirect heal and lock ownership (unchanged here). #2803: audit umbrella.
• Blocked by: nothing.