iOS keyboard capture can race dismissal and omit the input assistant bar

Read RunnerTests+Keyboard.swift at the supplied source revision, focusing on captureKeyboardBandFact and keyboardBandFact, then follow the caller in RunnerTests+SnapshotAcquisition.swift. Use the upstream keyboard fixtures and tests to check that one captured tree determines keyboard presence and coverage, including the assistant bar and normalized frames. Done means the dismissal race is removed while absent results, invalid geometry, budgets, and wire shape remain preserved; the payload reports 14 native keyboard tests and iOS/macOS runner builds passing.

Problem

The iOS XCTest capture path reads its accessibility tree, then independently queries app.keyboards.firstMatch, keyboard.exists, and keyboard.frame to publish the keyboard band. These observations can describe different moments. If the keyboard disappears between the existence and frame reads, XCTest can fail the live element lookup instead of returning a usable capture. A keyboard appearing or disappearing after the tree capture can also make its nodes and published band disagree.

The measured band additionally omits SystemInputAssistantView when the prediction/input-assistant bar is outside the keyboard element. An app control covered by that bar can consequently be treated as clear of the keyboard.

Source at 45125d717b207736dd47bc7969edbff7807a6e6d: RunnerTests+Keyboard.swift, captureKeyboardBandFact/keyboardBandFact; caller: RunnerTests+SnapshotAcquisition.swift.

Generic reproduction and expected behavior

• Capture a standard text field with the software keyboard visible, including its input assistant. Compare the published keyboard frame with both captured rectangles.
• Example captured keyboard (0,600,390,244) plus assistant (0,556,390,44) must cover (0,556,390,288).
• Capture while ending editing: measurement must use that capture's immutable tree, rather than re-querying a live keyboard that may have disappeared. The source-level race does not establish a deterministic crash reproduction on every simulator.

Completion conditions

Use the same captured tree for keyboard presence and coverage. Include the native assistant only when a keyboard is present. Normalize each contributing frame through the existing coordinate-space policy before combining it, including landscape and placed windows. Preserve absent/unmeasurable results, invalid-geometry handling, budgets and wire shape. Verify through upstream fixtures only; no application-specific fixture or new dependency is required.

Verification at 81266c2598b162d373703a921fdfedaee97ea00f

Dedicated iPhone 15 Pro Simulator, iOS 27.0; only the upstream com.callstack.agentdevice.runner fixture. Each checkout had separate DerivedData and daemon state. The same live tree-backend capture and band-containment assertion ran against base 45125d717b207736dd47bc7969edbff7807a6e6d and the proposed head:

Relaunching with no software keyboard also reports keyboard: {kind: "absent"}. Fourteen native keyboard tests pass with zero retries, including mixed-window coordinates and invalid native geometry. iOS and macOS runner builds pass. The source-level dismissal race was not deterministically reproduced as an XCTest failure in this public fixture; the independent live-query sequence is removed by construction. No native CI or physical-device result is claimed.

Reproduction commands

In each checkout, first run pnpm install --frozen-lockfile and pnpm build. Select a dedicated simulator through $UDID and use a unique $SESSION and $AGENT_DEVICE_STATE_DIR for that build:

export UDID SESSION
export AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH="$PWD/.tmp/keyboard-ios"
export AGENT_DEVICE_STATE_DIR="$(mktemp -d)"
AGENT_DEVICE_XCUITEST_ARCHS=arm64 AGENT_DEVICE_XCUITEST_DESTINATION='generic/platform=iOS Simulator' pnpm build:xcuitest:ios
pnpm clean:daemon
xcrun simctl install "$UDID" .tmp/keyboard-ios/Build/Products/Debug-iphonesimulator/AgentDeviceRunner.app
node bin/agent-device.mjs open com.callstack.agentdevice.runner --foreground --relaunch --launch-args=--agent-device-text-entry-regression --launch-args=--agent-device-text-entry-soft-keyboard --session "$SESSION" --platform ios --udid "$UDID" --json
node bin/agent-device.mjs press 'id="agent-device-hardware-keyboard-input"' --session "$SESSION" --platform ios --udid "$UDID" --json
node --experimental-strip-types capture-tree.mjs

The snapshot probe uses the built SDK with the existing-daemon transport and existing forced-backend conformance helper. This deliberately exercises the changed XCTest tree route; ordinary simulator CLI snapshots can use the independent AX bridge instead. capture-tree.mjs:

import assert from 'node:assert/strict';
import { pathToFileURL } from 'node:url';
const root = process.cwd();
const load = (path) => import(pathToFileURL(`${root}/${path}`).href);
const { createAgentDeviceClient } = await load('dist/src/index.js');
const { readDaemonInfo } = await load('src/daemon-client/daemon-client-metadata.ts');
const { sendRequest } = await load('src/daemon-client/daemon-client-transport.ts');
const { resolveDaemonPaths } = await load('packages/daemon-contracts/src/daemon-resolution.ts');
const { createSnapshotBackendConformanceTransport } = await load('test/integration/ios-simulator-e2e/snapshot-backend-conformance.ts');
const { AGENT_DEVICE_STATE_DIR: stateDir, SESSION: session, UDID: udid } = process.env;
const paths = resolveDaemonPaths(stateDir, { env: process.env, cwd: root });
const info = readDaemonInfo(paths.infoPath);
assert.ok(info, 'CLI daemon must already be ready');
const transport = (request, context) => sendRequest(
  info, { ...request, session: request.session, token: info.token }, 'auto', paths, 90000,
  { host: { env: process.env, cwd: root }, signal: context?.signal, timeoutRecovery: 'none' }
);
const client = createAgentDeviceClient(
  { session, stateDir, daemonLifecycle: 'attach' },
  { transport: createSnapshotBackendConformanceTransport('tree', transport) }
);
const snapshot = await client.capture.snapshot({ platform: 'ios', udid, raw: true });
const assistant = snapshot.nodes.find((n) => n.identifier === 'SystemInputAssistantView').rect;
const band = snapshot.keyboard.frame;
console.log(JSON.stringify({ backend: snapshot.snapshotQuality.backend, keyboard: snapshot.keyboard, assistant }));
assert.equal(snapshot.snapshotQuality.backend, 'tree');
assert.ok(band.y <= assistant.y && band.y + band.height >= assistant.y + assistant.height);

For landscape, use node bin/agent-device.mjs orientation landscape-right --session "$SESSION" --platform ios --udid "$UDID" --json, reopen/focus the same fixture, and repeat the probe. Finish with node bin/agent-device.mjs close --session "$SESSION" --platform ios --udid "$UDID" --json and node bin/agent-device.mjs daemon stop --clean --state-dir "$AGENT_DEVICE_STATE_DIR".