[Bug]: iPhone Duo (foldable) simulator: snapshot_ui works, tap/touch/key input never lands, screenshot cannot target the lit panel

Start at the simulator UI-automation entry points for snapshot_ui, tap, and screenshot, then trace how element coordinates and simctl screenshot arguments are passed. Reproduce with the iPhone Duo in folded and unfolded postures and compare it with the iPhone 18 Pro control. Done means the supported behavior is verified or the limitation is narrowed to XcodeBuildMCP versus CoreSimulator or AXe.

Bug Description

On the iPhone Duo foldable simulator that ships with Xcode 27.1 beta, snapshot_ui returns a correct accessibility tree and screenshot returns a correct image, but no input tool has any effect: tap by elementRef reports success and nothing changes on screen. This holds in both postures (folded and unfolded, set by hand in Device Hub). I could not find any input path that works, including calling the bundled AXe 1.8.0 directly, so this may be a CoreSimulator or AXe limitation rather than something XcodeBuildMCP does wrong. Filing it here because XcodeBuildMCP owns tap/screenshot and bundles AXe, and because two related gaps are in XcodeBuildMCP's control:

1. screenshot (and xcrun simctl io <UDID> screenshot, which it wraps) always captures the inner panel. Folded, that panel is off, so every screenshot is black while snapshot_ui reports a correct tree from the cover panel. There is no way to pass --display=1 through the tool to capture the lit cover panel.
2. snapshot_ui frames come back in a portrait coordinate space (root 466 x 678 pt, which is the cover panel's point size) even when the lit inner panel is landscape 951 x 669 pt, so the tool taps at a rotated point. After a SpringBoard restart inside the simulator the frames flip to landscape. Either way the tap still does not land, so this looks like a symptom rather than the cause.

Control: on an iPhone 18 Pro simulator (same Xcode 27.1, same iOS 27.1 runtime, same XcodeBuildMCP 2.7.0 session), snapshot_ui then tap on the Accessibility elementRef navigates into the Accessibility pane as expected (new screenHash, back button and VoiceOver rows in the next snapshot, screenshot changed). So the failure is specific to the two-display Duo.

Device: com.apple.CoreSimulator.SimDeviceType.iPhone-Duo, runtime iOS 27.1 (24A94401). Inner panel 2853 x 2007 px unfolded, cover panel 1398 x 2034 px.

Debug Output

⚙️ XcodeBuildMCP Doctor

   Generated: 2026-09-21T01:31:28.923Z
   Server Version: 2.7.0
   Output Mode: Redacted (default)

System Information
  platform: darwin
  release: 25.6.0
  arch: arm64
  cpus: 8 x Apple M3
  memory: 16 GB
  hostname: <redacted>
  username: <redacted>
  homedir: /Users/<redacted>
  tmpdir: /var/folders/b6/g7g_k9tx7x5ffmw91f5ycp000000gn/T

Node.js Information
  version: v24.20.0
  execPath: /nix/store/v2x67wfihfy1366kmi6ccp78k657afya-nodejs-slim-24.20.0/bin/node
  pid: 71435
  ppid: 71430
  platform: darwin
  arch: arm64
  cwd: <repo>
  argv: /nix/store/v2x67wfihfy1366kmi6ccp78k657afya-nodejs-slim-24.20.0/bin/node /Users/<redacted>/.npm/_npx/6389024b78b4e852/node_modules/xcodebuildmcp/build/doctor-cli.js

Process Tree
  (omitted: contains local session and credential material)

Xcode Information
  version: Xcode 27.1 - Build version 27A9269
  path: /Applications/Xcode.app/Contents/Developer
  selectedXcode: /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild
  xcrunVersion: xcrun version 72.

Dependencies
  axe: 1.8.0
  mise: Not found

Environment Variables
  INCREMENTAL_BUILDS_ENABLED: (not set)
  DEVELOPER_DIR: (not set)
  HOME: /Users/<redacted>
  USER: <redacted>
  TMPDIR: /var/folders/b6/g7g_k9tx7x5ffmw91f5ycp000000gn/T/
  NODE_ENV: (not set)
  SENTRY_DISABLED: (not set)
  AXE_PATH: (not set)
  XBMCP_LAUNCH_JSON_WAIT_MS: (not set)
  XCODEBUILDMCP_DEBUGGER_BACKEND: (not set)
  XCODEBUILDMCP_UI_DEBUGGER_GUARD_MODE: (not set)
  XCODEBUILDMCP_RUNTIME: cli
  XCODEBUILDMCP_SILENCE_LOGS: true

PATH
  (omitted)

UI Automation (axe)
  Available: Yes
  UI Automation Supported: Yes
  Simulator Video Capture Supported (AXe >= 1.1.0): Yes
  UI-Debugger Guard Mode: error

Incremental Builds
  Enabled: No
  xcodemake Binary Available: No
  Makefile exists (cwd): (not checked: incremental builds disabled)

Mise Integration
  Running under mise: No
  Mise available: No

Debugger Backend (DAP)
  lldb-dap available: Yes
  Selected backend: dap

Manifest Tool Inventory
  Total Unique Tools: 82
  Workflow Count: 15
  coverage: 2 tools
  debugging: 8 tools
  device: 15 tools
  doctor: 1 tools
  macos: 13 tools
  project-discovery: 5 tools
  project-scaffolding: 2 tools
  session-management: 5 tools
  simulator-management: 10 tools
  simulator: 20 tools
  swift-package: 8 tools
  ui-automation: 14 tools
  utilities: 1 tools
  workflow-discovery: 1 tools
  xcode-ide: 5 tools

Runtime Tool Registration
  Enabled Workflows: 0
  Registered Tools: 0
  Note: Runtime registry unavailable.

Xcode IDE Bridge (mcpbridge)
  Workflow enabled: No
  mcpbridge path: /Applications/Xcode.app/Contents/Developer/usr/bin/mcpbridge
  Xcode running: true
  Connected: No
  Bridge PID: (none)
  Proxied tools: 0
  Last error: (none)
  Note: Bridge debug tools (status/sync/disconnect) are only registered when debug: true

Tool Availability Summary
  Build Tools: Available
  UI Automation Tools: Available
  Incremental Build Support: Not available

Sentry
  Sentry enabled: Yes

Troubleshooting Tips
  If UI automation tools are not available, install axe: brew tap cameroncooke/axe && brew install axe
  If incremental build support is not available, install xcodemake (https://github.com/cameroncooke/xcodemake) and ensure it is executable and available in your PATH
  To enable xcodemake, set environment variable: export INCREMENTAL_BUILDS_ENABLED=1
  For mise integration, follow instructions in the README.md file

✅ Doctor diagnostics complete


Doctor run complete. Please include this output when reporting issues.

Process Tree and PATH sections omitted: on this host the doctor's process tree prints the parent MCP client's command line, which includes a bearer token.

Editor/Client

Claude Code 2.1.278 (project .mcp.json entry) and Codex CLI 0.155.1 ([mcp_servers.xcodebuildmcp], npx --yes xcodebuildmcp@2.7.0 mcp). Same result from both clients.

MCP Server Version

2.7.0 (AXe 1.8.0 bundled). 2.6.2 cannot start AXe 1.7.1 on Xcode 27 at all (SimulatorKit.framework does not exist, since Xcode 27 moved it to Contents/SharedFrameworks), so 2.7.0 is the first version that can be tested here.

LLM

Claude Fable 5.1 (Claude Code) and GPT-6 Astra (Codex CLI). The same taps were also issued by hand with the bundled axe binary, outside any model, with the same result.

MCP Configuration

{
  "mcpServers": {
    "xcodebuildmcp": {
      "command": "npx",
      "args": ["--yes", "xcodebuildmcp@2.7.0", "mcp"],
      "env": {
        "XCODEBUILDMCP_ENABLED_WORKFLOWS": "simulator,ui-automation,debugging,logging"
      }
    }
  }
}

Steps to Reproduce

1. Xcode 27.1 beta (27A9269). xcrun simctl create duo com.apple.CoreSimulator.SimDeviceType.iPhone-Duo com.apple.CoreSimulator.SimRuntime.iOS-27-1, boot it. It boots folded.
2. session_set_defaults with that UDID, launch_app_sim with com.apple.Preferences.
3. snapshot_ui: correct tree (Settings sidebar rows). screenshot: all black (inner panel is off). xcrun simctl io <UDID> screenshot --display=1 shows Settings on the cover panel, but the tool cannot request that display.
4. Unfold the device in Device Hub (there is no simctl posture subcommand). screenshot now shows a landscape split view. snapshot_ui still correct.
5. tap the elementRef of a sidebar row such as Accessibility, or of the Open button on an Open in "App"? alert. The tool reports SUCCEEDED with a point such as (354, 401). wait_for_ui with a text predicate times out, the next snapshot_ui returns the same screenHash, and the screenshot is pixel-identical.
6. Bypass XcodeBuildMCP and call the bundled AXe directly with DYLD_FRAMEWORK_PATH=<pkg>/bundled/Frameworks <pkg>/bundled/axe .... All of these report success and change nothing:
   • tap -x 354 -y 401 (portrait frame from the AX tree)
   • tap -x 549 -y 354 (the same button in the landscape frame the screenshot shows)
   • tap --label Open, tap --label "Enter URL manually"
   • touch --down / touch --up at the same point
   • key 40 (Return), key 44 (Space), key 41 (Escape), key-sequence 43,44
   • button home (the system alert stays up)
7. describe-ui shows the Application root frame as {x:0, y:0, width:466, height:678} with the app's group at {width:669, height:951} while the lit panel is 951 x 669 pt landscape. After xcrun simctl spawn <UDID> launchctl kickstart -k system/com.apple.SpringBoard the frames come back landscape (for example Enter URL manually at {x:32, y:175.7, width:803, height:17.7}); taps at those points also do nothing.

Expected Behavior

tap on an elementRef from snapshot_ui activates that element on whichever panel is lit, and screenshot captures the lit panel (or exposes a display selector).

Actual Behavior

Every input tool reports success and has no effect on the Duo, in both postures. screenshot targets the inner panel only, so it is black while folded. snapshot_ui is correct in both postures.

Error Messages

None. Every tool call returns SUCCEEDED. Example tap result:

{"summary":{"status":"SUCCEEDED"},"action":{"type":"tap","elementRef":"e219","x":354,"y":401}}

followed by wait_for_ui WAIT_TIMEOUT and an unchanged screenHash.

---

Written with the help of Claude Fable 5.1 in Claude Code; every observation above was reproduced by hand on the machine that produced the doctor output.