macOS: touchscreen support (MacBook Pro PROLED XDR, macOS 27 gesture-recognizer touch)

Start by tracing the existing Windows touch path through NativeTaoBridge.EventCallback.onTouchInput and TaoComposeSceneHostWindows.onTouchInput, then compare it with the macOS trackpad path in touchpad_gestures.m. The issue proposes a native macOS gesture recognizer, shared Compose touch handling, and plugin configuration, but leaves hardware behavior and defaults unresolved. Done means direct touch reaches Compose without duplicate emulated input, trackpad behavior still works, and the documented hardware or Sidecar probe and regression checks pass.

Context

On 2026-10-28 Apple announced the MacBook Pro with PROLED XDR Display, the first Mac with a built-in multi-touch screen (available 2026-11-06). Apple says it supports scroll, pinch to zoom, rotate, drawing, double-tap and drag & drop on the screen, "as intuitive as they do on a trackpad" (product page).

The input model for AppKit has been documented since WWDC26 (June 2026, macOS 27), originally for Sidecar touch input:

• TN3212: Adopting gesture recognizers for Sidecar touch support
• NSIsTouchNative (Info.plist key, macOS 27+)
• AppKit updates, June 2026
• WWDC26 289: Modernize your AppKit app

Today Nucleus has a real touchscreen path on Windows (tao WindowEvent::Touch → NativeTaoBridge.EventCallback.onTouchInput → TaoComposeSceneHostWindows) and Linux (NativeTaoLinuxTouchBridge, platform/linux/touch.rs). On macOS there is no direct-touch path: the only "touch" is synthetic, from the trackpad pipeline. Nothing in the repo references NSIsTouchNative, NSGestureRecognizer or touchesBegan.

How macOS 27 delivers touch (summary of TN3212)

• Gesture recognizers are the only way to receive direct touches. Touches don't go through the responder chain, NSApp / NSWindow.sendEvent(_:), or local/global NSEvent monitors.
• AppKit gathers the recognizers at touch-began by walking up the view hierarchy from the hit-tested top-most view. A view covered by another view, even a transparent one, gets nothing.
• Exclusivity: by default, gestures are active in only one view hierarchy at a time (scrolling excepted). You can lift this with NSView.exclusiveGestureBehavior (per view, inherited by children) or the NSViewGestureRecognizerIsExclusive Info.plist key.
• Mouse emulation for third-party UI frameworks, which covers us, since Compose draws into the tao NSView. By default AppKit injects recognizers that translate:
  • tap → mouseDown immediately followed by mouseUp
  • touch and immediate pan → trackpad scroll events
  • long press without movement → rightMouseDown (+ rightMouseDragged) and rightMouseUp on lift
  • two-finger pinch or rotation → trackpad magnify and rotate at the same time
• NSIsTouchNative = YES turns that emulation off, for frameworks that handle touch themselves.
• Custom NSGestureRecognizer subclasses get touchesBegan/Moved/Ended/Cancelled(with:) with NSTouch objects. With the macOS 27 SDK they must override location(in:), or it returns NSZeroPoint and logs an error. They must also handle cancellation (touchesCancelled, mouseCancelled) and always reach ended / failed / cancelled, or recognizers stay blocked.
• isCancellableByScrollGesture lets a touch recognizer yield to scrolling.
• Capability detection: NSScreen.TouchCapabilities.multiTouch. On macOS 27 it returns true for all displays as soon as a touch-capable Sidecar display is connected.
• NSApplication.currentEvent no longer returns the event that triggered a control action. A touch is not part of NSEvent.pressedMouseButtons.

What a Nucleus app gets today, with no change (expected, unverified)

Open questions to check on hardware

1. Do the emulated magnify/rotate events reach our local NSEvent monitor (touchpad_gestures.m)? TN3212 says real touches bypass monitors. It says nothing about the emulated trackpad events. If they bypass it too, pinch-to-zoom is silently dead in every Nucleus app.
2. Does the emulated scroll carry phase / momentumPhase like a real trackpad, so TaoScrollGesturePhase and fling keep working?
3. How is a one-finger drag emulated? Is it always scroll, or is there a press-then-move that becomes mouseDragged? This decides whether sliders, text selection, WindowDragArea, tab drag (workspace/CrossWindowDrag) and DnD work by finger.
4. Do the tao NSView and our overlays (NSPanel popups, NativeView / TextureViewMac, decoration views in decoration.m) take part in hit testing in a way that lets AppKit's injected recognizers fire? A transparent view on top blocks every recognizer.
5. Do the title bar / traffic lights behave, given custom decoration and Liquid Glass (macOsSdkVersion = "26.0")? Is double-tap to fill the screen wired?
6. Does any of this need an app linked against the macOS 27 SDK (vtool macOsSdkVersion) to switch on?

Proposal: native touch path on macOS

1. Native recognizer (ObjC, next to touchpad_gestures.m): a NSGestureRecognizer subclass attached to the tao content NSView, implementing touchesBegan/Moved/Ended/Cancelled(with:) and location(in:). Restricted to direct touches (NSTouchTypeDirect), so trackpad touches keep the current path. It never "recognizes" in a way that cancels other input: it only forwards.
2. Forward through the existing wire: NativeTaoBridge.EventCallback.onTouchInput(handle, phase, id, xFixed, yFixed, forceFixed), already used on Windows, with TaoTouchEvent.PRESS/MOVE/RELEASE/CANCEL. The finger id comes from NSTouch.identity, and the position is physical px × 1024, like CURSOR_MOVED (view coords → flip Y → × backing scale). Force is -1 unless the panel reports it.
3. Compose side: share or move the Windows aggregation in TaoComposeSceneHostWindows.onTouchInput into the common host (AbstractTaoComposeSceneHost), so macOS emits real PointerType.Touch pointers. This also covers TaoTouchEventTime, the deadband (TaoPointerDeadband) and NSPanel popups (TaoPopupSceneLayer).
4. Exclusivity: set exclusiveGestureBehavior on the tao view so that multi-touch and several simultaneous interactions (faders, two-hand gestures) work.
5. Turn off the emulation only once the native path exists: the plugin's InfoPlistBuilder adds NSIsTouchNative = YES. Otherwise every touch arrives twice, once as a touch and once as emulated mouse / scroll / magnify. Maybe behind a DSL switch (macOS { touchNative = true }, default to be decided) so an app can fall back to the emulation.
6. Dedupe with the trackpad pipeline: with native touches on, pinch and rotate come from Compose's own detectTransformGestures / transformable over real pointers. The synthetic two-contact rotate and the magnify Scale path must stay trackpad-only.
7. Capability API (optional): expose NSScreen.TouchCapabilities.multiTouch, keeping in mind it is wrong (true for every display) when a Sidecar touch display is connected.
8. Sidecar: the same recognizer gives iPad Sidecar touch support (iPadOS 27) for free, so we can test without the new MacBook.
9. GraalVM: any new JNI entry point needs reachability-metadata.json. onTouchInput already exists, so ideally there are none.

Testing

• Hardware: MacBook Pro with PROLED XDR Display (available 2026-11-06), or Sidecar with an iPad on iPadOS 27 + macOS 27, which goes through the same gesture-recognizer pipeline.
• Lab probe: a touch probe that shows the live pointer list (id, type, position, pressure), plus a slider, a detectDragGestures canvas, transformable and a text field. Run it with and without NSIsTouchNative.
• Regression: trackpad scroll / Pan / magnify / rotate unchanged (TaoScrollWireDriftTest, the headful trackpad coverage in docs/internals/tao-trackpad.md).

Docs

Add a docs/internals/tao-macos-touch.md (emulation vs native, NSIsTouchNative, exclusivity, answers to the open questions), and a link from CLAUDE.md next to "Trackpad on Tao".