Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Research: single-codebase Swift rewrite without Peekaboo or Node

Abierto
  • ¿Algo no está bien?
#143 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
15/100
Tipo de issue
Refactorización
Claridad
Bien especificado
Estado de actividad
Estancado
Stack tecnológico
macos, swift
Área
desktop

Línea de trabajo

This issue is research and planning only and is closed as not planned for now; it names no implementation files to change. Read the proposed layout and steps, then assess whether the rewrite should proceed and where it should live. Done means resolving the listed open questions and establishing a go/no-go decision; the issue estimates 2–4 days for the rewrite itself.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Research and planning only. Closed as not planned for now; it may be picked up later.

Question

Should we rewrite desktop-tools as one codebase with no Peekaboo, no patch stack and no Node?

Sizing (as of v0.14.1)

Layer Size
TypeScript (MCP server, client, protocol, wait, tables, notifications) 5.7k lines, plus 1.3k of tests and 1.5k of scripts
Our Swift (native/sources) 8.4k lines
Patch stack on Peekaboo 4.8.0 63 patches, 15.3k diff lines, about 7.9k added lines
Peekaboo repo 380k lines
Peekaboo modules we link AutomationKit 94k (41k UI services), Bridge 42k, AXorcist 23k, Foundation 5k

The 48 MCP tools make about 66 distinct Bridge client calls. Much of Bridge is code we never call: agent execution, browser sessions, certification. An estimated 30–50k lines of Peekaboo are actually used. A purpose-built codebase would likely be 25–40k lines in total. Both figures are estimates; call coverage wasn't traced.

Rust vs Swift: decision is Swift

  • Speed: language makes no difference. Time goes to (1) one client process per call (execFile in src/client.ts), which pays for spawn, handshake and socket connect each time, and (2) Accessibility IPC round trips per attribute read. The fixes are a persistent connection and batched AX reads (AXUIElementCopyMultipleAttributeValues), in either language.
  • Testing and safety: Rust only helps the pure logic: protocol, snapshots, stale-target refusal, approval, the wait/expect engine. That's about a third of the code. The bugs that have hurt us come from live macOS behaviour, which only live-check catches.
  • macOS surface: AX, CGEvent, CGS spaces, TIS and Security are C APIs and would be easy from Rust. ScreenCaptureKit, Vision and AppKit/SwiftUI are where Rust gets painful. We interface with macOS too deeply for Rust to pay off, so we'd skip it.
  • Swift gives one language and one toolchain, native frameworks with no FFI, Swift 6 data-race safety (already on), and a Swift MCP server, so Node goes away too.

Proposed layout (single SwiftPM package)

Sources/
  DesktopProtocol/   wire types, the 48 tool schemas, error codes, receipts (pure)
  DesktopCore/       snapshots, targets/stale refusal, wait/expect, result bounding,
                     tables/cells, password-fill rules. Pure; the OS sits behind protocols
  DesktopMac/        the only module that touches macOS: AX, CGEvent post-to-pid, CGS spaces,
                     ScreenCaptureKit, Vision OCR, TIS, Keychain, pasteboard, notifications
  DesktopHost/       socket server, caller identity and approval, pause, activity, embed C ABI
  DesktopOverlay/    agent cursor and window outline
  desktop-tools-mcp/ MCP over stdio, one open connection to the host
  DesktopToolsApp/   menu bar shell that links DesktopHost
Tests/  CoreTests (fakes), ContractTests (golden schemas), MacTests (opt-in, live)
  • Operations run in the app process, because that's what the macOS grants attach to. The MCP binary only translates, and ships signed inside the app bundle: claude mcp add desktop -- "/Applications/Desktop Tools.app/Contents/MacOS/desktop-tools-mcp".
  • Write MCP over stdio ourselves (a few hundred lines) rather than pull in the SDK and its dependencies. Aim for no third-party dependencies; AXorcist (unpatched) could be a temporary dependency while bootstrapping.
  • A menu bar shell built on GPUI would have to live outside this repo because of its dependency stack.

Steps

  1. Baseline bench of today's build (below).
  2. Freeze the contract. Golden tool schemas, error codes and result shapes. Port the TS logic tests to Swift Testing as the spec for DesktopCore.
  3. Skeleton. Package, socket protocol, caller identity port, and the MCP binary end to end on desktop_apps. Use a separate socket and state folder so it runs next to the installed app.
  4. Core tools: apps, windows, inspect, click, type, key, scroll, drag, hover, wait. Bench against the baseline; this is the go/no-go point.
  5. The remaining ~33 tools by family; each family is done when its live-check items pass.
  6. Cutover. Bundle the MCP binary, update the docs, keep an npx shim for one release, then delete Peekaboo, native/patches, build.ts and the TS client.

Rough size: about 2–4 days in total, mostly step 4.

Benchmark plan

Pull live-check's MCP client out into a bench script that takes --server <command> and runs the same scenarios against node dist/mcp.js and the new binary, writing JSON results.

  • Latency per tool, p50 and p95 over 20+ runs: apps, windows, inspect on TextEdit, Finder and a heavy Safari page, click with expect, typing 200 characters, a wait that succeeds after 1s.
  • Scenario wall time for the live-check flows.
  • Correctness: live-check pass rate, flakiness over N repeats, field-level diffs of results.
  • Tokens returned per inspect (text and image bytes).
  • AX IPC calls per inspect.
  • Memory and CPU of the MCP server and the app, and MCP startup time.
  • Dev loop: build and test times, lines of code, dependency count, bundle size.
  • Add a no-op ping op to separate handshake cost from the work itself.

First numbers, taken against the installed v0.14.1 app, calling the client directly without Node:

  • Client spawn ending in a usage error: about 8 ms (a bare /usr/bin/true is about 4 ms)
  • apps end to end: about 49 ms p50
  • windows with a non-existent pid, which fails: about 72 ms

Open questions

  1. Embedders: does anything besides Desktop Tools use the C ABI in docs/embed.md or the Node createDesktop API in docs/node.md? If not, drop both. If so, keep the C ABI exactly and make the Node API a thin wrapper.
  2. Where it lives: native2/ on a branch in this repo, or a fresh repo.
Lenguaje dominante
Swift
Estrellas
0
Forks
0
Merge medio
39 min
PR fusionados (30 d)
144

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de githubnext/desktop-tools

Todos los issues de githubnext/desktop-tools

Issues similares

Más issues de Swift

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.