Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

Research: single-codebase Swift rewrite without Peekaboo or Node

Offen
  • Stimmt etwas nicht?
#143 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
15/100
Issue-Typ
Refactoring
Klarheit
Klar beschrieben
Aktivitätsstatus
Veraltet
Tech-Stack
macos, swift
Bereich
desktop

Rechercherichtung

This issue is research and planning only and is closed as not planned for now; it names no implementation files to change. Read the proposed layout and steps, then assess whether the rewrite should proceed and where it should live. Done means resolving the listed open questions and establishing a go/no-go decision; the issue estimates 2–4 days for the rewrite itself.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Research and planning only. Closed as not planned for now; it may be picked up later.

Question

Should we rewrite desktop-tools as one codebase with no Peekaboo, no patch stack and no Node?

Sizing (as of v0.14.1)

Layer Size
TypeScript (MCP server, client, protocol, wait, tables, notifications) 5.7k lines, plus 1.3k of tests and 1.5k of scripts
Our Swift (native/sources) 8.4k lines
Patch stack on Peekaboo 4.8.0 63 patches, 15.3k diff lines, about 7.9k added lines
Peekaboo repo 380k lines
Peekaboo modules we link AutomationKit 94k (41k UI services), Bridge 42k, AXorcist 23k, Foundation 5k

The 48 MCP tools make about 66 distinct Bridge client calls. Much of Bridge is code we never call: agent execution, browser sessions, certification. An estimated 30–50k lines of Peekaboo are actually used. A purpose-built codebase would likely be 25–40k lines in total. Both figures are estimates; call coverage wasn't traced.

Rust vs Swift: decision is Swift

  • Speed: language makes no difference. Time goes to (1) one client process per call (execFile in src/client.ts), which pays for spawn, handshake and socket connect each time, and (2) Accessibility IPC round trips per attribute read. The fixes are a persistent connection and batched AX reads (AXUIElementCopyMultipleAttributeValues), in either language.
  • Testing and safety: Rust only helps the pure logic: protocol, snapshots, stale-target refusal, approval, the wait/expect engine. That's about a third of the code. The bugs that have hurt us come from live macOS behaviour, which only live-check catches.
  • macOS surface: AX, CGEvent, CGS spaces, TIS and Security are C APIs and would be easy from Rust. ScreenCaptureKit, Vision and AppKit/SwiftUI are where Rust gets painful. We interface with macOS too deeply for Rust to pay off, so we'd skip it.
  • Swift gives one language and one toolchain, native frameworks with no FFI, Swift 6 data-race safety (already on), and a Swift MCP server, so Node goes away too.

Proposed layout (single SwiftPM package)

Sources/
  DesktopProtocol/   wire types, the 48 tool schemas, error codes, receipts (pure)
  DesktopCore/       snapshots, targets/stale refusal, wait/expect, result bounding,
                     tables/cells, password-fill rules. Pure; the OS sits behind protocols
  DesktopMac/        the only module that touches macOS: AX, CGEvent post-to-pid, CGS spaces,
                     ScreenCaptureKit, Vision OCR, TIS, Keychain, pasteboard, notifications
  DesktopHost/       socket server, caller identity and approval, pause, activity, embed C ABI
  DesktopOverlay/    agent cursor and window outline
  desktop-tools-mcp/ MCP over stdio, one open connection to the host
  DesktopToolsApp/   menu bar shell that links DesktopHost
Tests/  CoreTests (fakes), ContractTests (golden schemas), MacTests (opt-in, live)
  • Operations run in the app process, because that's what the macOS grants attach to. The MCP binary only translates, and ships signed inside the app bundle: claude mcp add desktop -- "/Applications/Desktop Tools.app/Contents/MacOS/desktop-tools-mcp".
  • Write MCP over stdio ourselves (a few hundred lines) rather than pull in the SDK and its dependencies. Aim for no third-party dependencies; AXorcist (unpatched) could be a temporary dependency while bootstrapping.
  • A menu bar shell built on GPUI would have to live outside this repo because of its dependency stack.

Steps

  1. Baseline bench of today's build (below).
  2. Freeze the contract. Golden tool schemas, error codes and result shapes. Port the TS logic tests to Swift Testing as the spec for DesktopCore.
  3. Skeleton. Package, socket protocol, caller identity port, and the MCP binary end to end on desktop_apps. Use a separate socket and state folder so it runs next to the installed app.
  4. Core tools: apps, windows, inspect, click, type, key, scroll, drag, hover, wait. Bench against the baseline; this is the go/no-go point.
  5. The remaining ~33 tools by family; each family is done when its live-check items pass.
  6. Cutover. Bundle the MCP binary, update the docs, keep an npx shim for one release, then delete Peekaboo, native/patches, build.ts and the TS client.

Rough size: about 2–4 days in total, mostly step 4.

Benchmark plan

Pull live-check's MCP client out into a bench script that takes --server <command> and runs the same scenarios against node dist/mcp.js and the new binary, writing JSON results.

  • Latency per tool, p50 and p95 over 20+ runs: apps, windows, inspect on TextEdit, Finder and a heavy Safari page, click with expect, typing 200 characters, a wait that succeeds after 1s.
  • Scenario wall time for the live-check flows.
  • Correctness: live-check pass rate, flakiness over N repeats, field-level diffs of results.
  • Tokens returned per inspect (text and image bytes).
  • AX IPC calls per inspect.
  • Memory and CPU of the MCP server and the app, and MCP startup time.
  • Dev loop: build and test times, lines of code, dependency count, bundle size.
  • Add a no-op ping op to separate handshake cost from the work itself.

First numbers, taken against the installed v0.14.1 app, calling the client directly without Node:

  • Client spawn ending in a usage error: about 8 ms (a bare /usr/bin/true is about 4 ms)
  • apps end to end: about 49 ms p50
  • windows with a non-existent pid, which fails: about 72 ms

Open questions

  1. Embedders: does anything besides Desktop Tools use the C ABI in docs/embed.md or the Node createDesktop API in docs/node.md? If not, drop both. If so, keep the C ABI exactly and make the Node API a thin wrapper.
  2. Where it lives: native2/ on a branch in this repo, or a fresh repo.
Vorherrschende Sprache
Swift
Sterne
0
Forks
0
Ø Merge
39 Min.
Gemergte PRs (30 T.)
144

Entwicklungsumgebung

Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus githubnext/desktop-tools

Alle Issues in githubnext/desktop-tools

Ähnliche Issues

Weitere Issues zu Swift

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.