Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Research: single-codebase Swift rewrite without Peekaboo or Node

Đang mở
  • Có gì không ổn?
#143 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
15/100
Loại issue
Tái cấu trúc
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Đình trệ
Công nghệ
macos, swift
Lĩnh vực
desktop

Hướng nghiên cứu

This issue is research and planning only and is closed as not planned for now; it names no implementation files to change. Read the proposed layout and steps, then assess whether the rewrite should proceed and where it should live. Done means resolving the listed open questions and establishing a go/no-go decision; the issue estimates 2–4 days for the rewrite itself.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Research and planning only. Closed as not planned for now; it may be picked up later.

Question

Should we rewrite desktop-tools as one codebase with no Peekaboo, no patch stack and no Node?

Sizing (as of v0.14.1)

Layer Size
TypeScript (MCP server, client, protocol, wait, tables, notifications) 5.7k lines, plus 1.3k of tests and 1.5k of scripts
Our Swift (native/sources) 8.4k lines
Patch stack on Peekaboo 4.8.0 63 patches, 15.3k diff lines, about 7.9k added lines
Peekaboo repo 380k lines
Peekaboo modules we link AutomationKit 94k (41k UI services), Bridge 42k, AXorcist 23k, Foundation 5k

The 48 MCP tools make about 66 distinct Bridge client calls. Much of Bridge is code we never call: agent execution, browser sessions, certification. An estimated 30–50k lines of Peekaboo are actually used. A purpose-built codebase would likely be 25–40k lines in total. Both figures are estimates; call coverage wasn't traced.

Rust vs Swift: decision is Swift

  • Speed: language makes no difference. Time goes to (1) one client process per call (execFile in src/client.ts), which pays for spawn, handshake and socket connect each time, and (2) Accessibility IPC round trips per attribute read. The fixes are a persistent connection and batched AX reads (AXUIElementCopyMultipleAttributeValues), in either language.
  • Testing and safety: Rust only helps the pure logic: protocol, snapshots, stale-target refusal, approval, the wait/expect engine. That's about a third of the code. The bugs that have hurt us come from live macOS behaviour, which only live-check catches.
  • macOS surface: AX, CGEvent, CGS spaces, TIS and Security are C APIs and would be easy from Rust. ScreenCaptureKit, Vision and AppKit/SwiftUI are where Rust gets painful. We interface with macOS too deeply for Rust to pay off, so we'd skip it.
  • Swift gives one language and one toolchain, native frameworks with no FFI, Swift 6 data-race safety (already on), and a Swift MCP server, so Node goes away too.

Proposed layout (single SwiftPM package)

Sources/
  DesktopProtocol/   wire types, the 48 tool schemas, error codes, receipts (pure)
  DesktopCore/       snapshots, targets/stale refusal, wait/expect, result bounding,
                     tables/cells, password-fill rules. Pure; the OS sits behind protocols
  DesktopMac/        the only module that touches macOS: AX, CGEvent post-to-pid, CGS spaces,
                     ScreenCaptureKit, Vision OCR, TIS, Keychain, pasteboard, notifications
  DesktopHost/       socket server, caller identity and approval, pause, activity, embed C ABI
  DesktopOverlay/    agent cursor and window outline
  desktop-tools-mcp/ MCP over stdio, one open connection to the host
  DesktopToolsApp/   menu bar shell that links DesktopHost
Tests/  CoreTests (fakes), ContractTests (golden schemas), MacTests (opt-in, live)
  • Operations run in the app process, because that's what the macOS grants attach to. The MCP binary only translates, and ships signed inside the app bundle: claude mcp add desktop -- "/Applications/Desktop Tools.app/Contents/MacOS/desktop-tools-mcp".
  • Write MCP over stdio ourselves (a few hundred lines) rather than pull in the SDK and its dependencies. Aim for no third-party dependencies; AXorcist (unpatched) could be a temporary dependency while bootstrapping.
  • A menu bar shell built on GPUI would have to live outside this repo because of its dependency stack.

Steps

  1. Baseline bench of today's build (below).
  2. Freeze the contract. Golden tool schemas, error codes and result shapes. Port the TS logic tests to Swift Testing as the spec for DesktopCore.
  3. Skeleton. Package, socket protocol, caller identity port, and the MCP binary end to end on desktop_apps. Use a separate socket and state folder so it runs next to the installed app.
  4. Core tools: apps, windows, inspect, click, type, key, scroll, drag, hover, wait. Bench against the baseline; this is the go/no-go point.
  5. The remaining ~33 tools by family; each family is done when its live-check items pass.
  6. Cutover. Bundle the MCP binary, update the docs, keep an npx shim for one release, then delete Peekaboo, native/patches, build.ts and the TS client.

Rough size: about 2–4 days in total, mostly step 4.

Benchmark plan

Pull live-check's MCP client out into a bench script that takes --server <command> and runs the same scenarios against node dist/mcp.js and the new binary, writing JSON results.

  • Latency per tool, p50 and p95 over 20+ runs: apps, windows, inspect on TextEdit, Finder and a heavy Safari page, click with expect, typing 200 characters, a wait that succeeds after 1s.
  • Scenario wall time for the live-check flows.
  • Correctness: live-check pass rate, flakiness over N repeats, field-level diffs of results.
  • Tokens returned per inspect (text and image bytes).
  • AX IPC calls per inspect.
  • Memory and CPU of the MCP server and the app, and MCP startup time.
  • Dev loop: build and test times, lines of code, dependency count, bundle size.
  • Add a no-op ping op to separate handshake cost from the work itself.

First numbers, taken against the installed v0.14.1 app, calling the client directly without Node:

  • Client spawn ending in a usage error: about 8 ms (a bare /usr/bin/true is about 4 ms)
  • apps end to end: about 49 ms p50
  • windows with a non-existent pid, which fails: about 72 ms

Open questions

  1. Embedders: does anything besides Desktop Tools use the C ABI in docs/embed.md or the Node createDesktop API in docs/node.md? If not, drop both. If so, keep the C ABI exactly and make the Node API a thin wrapper.
  2. Where it lives: native2/ on a branch in this repo, or a fresh repo.
Ngôn ngữ chính
Swift
Star
0
Fork
0
Merge trung bình
42 phút
Pull request đã merge (30 ngày)
123

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của githubnext/desktop-tools

Tất cả issue của githubnext/desktop-tools

Issue tương tự

Thêm issue về Swift

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.