RFC — Phase 5: Remote Agent Backend (over SSH / WebSocket)

This is a design RFC, not a bounded change. Read the candidate paths and the eight open questions, then check connectHeadless.ts and createDirectConnectSession.ts, which the RFC names as the existing H5 entry points. Done means the open questions have recorded decisions from the maintainers, so a newcomer's best first step is to comment with answers rather than write code.

RFC — Phase 5: Remote Agent Backend (over SSH / WebSocket)

Status: Draft — open for discussion
Champion: @GoDiao
Related: Original feature request #3 — Remote Development Support (mirrored from openai/codex#10450)

---

Why this RFC

DreamCoder v0.4.0 shipped H5 Remote Access (Phase 3): you sit in front of a desktop running DreamCoder and reach into it from your phone. That solves "I'm away from my desk."

It does not solve a different problem that #3 (and many real-world workflows) raise: "my code lives on a remote machine and I don't want to clone it locally." Typical setups:

• VS Code Remote-SSH style daily workflow
• GPU / specialized hardware boxes (training, inference, simulation)
• Cloud / production-like dev VMs
• Security/compliance contexts where code may not leave the server

What's needed is the opposite topology of H5:

	H5 Remote Access (v0.4.0)	Remote Agent Backend (this RFC)
Agent runtime	local desktop	remote host
Codebase location	local	remote
Remote endpoint	phone / browser (thin)	server / GPU box (heavy)
Problem solved	"I'm away from my desk"	"Code isn't on my desk"

The two are complementary — if we land both, DreamCoder works wherever you are and wherever your code is.

---

Goals

• Run the DreamCoder agent (Claude Code CLI + tools + PTY + MCP) on a remote machine, against a remote codebase.
• Use the local DreamCoder desktop GUI as the client UI — chat panel, sessions, settings, terminal pass-through.
• Authenticate cleanly. Idle remote sessions should not become an attack surface.
• Be opt-in and reversible — local-only mode stays the default, the remote backend is an additional connection mode.
• Reuse as much of the existing sidecar / H5 protocol as possible. Avoid forking the codebase.

Non-goals (for v1)

• Full IDE features (LSP indexing, language servers, refactor across files via remote). Out of scope — Claude Code already calls remote tools fine via PTY.
• Multi-user collaborative editing on the same remote session.
• Reinventing SSH / VPN. We assume the user already has SSH access to the host.
• File sync / mirroring like VS Code Remote-SSH's vfs. The agent operates server-side; the client renders chat + file diffs.

---

Candidate paths

Three architectural shapes are on the table. They are not mutually exclusive — Path C is arguably a superset.

Path A — SSH reverse tunnel to a remote sidecar

[ Local DreamCoder GUI ] ──(localhost http/ws)──► [ ssh -R tunnel ] ──► [ dreamcoder-sidecar on remote ]
                                                                              │
                                                                              ▼
                                                                       remote codebase + claude CLI

How: User configures a remote host. DreamCoder spawns ssh -R 0:127.0.0.1:<port> user@host 'dreamcoder-sidecar --headless --bind 127.0.0.1:<port>'. The local GUI talks to localhost as if it were the local sidecar.

• ✅ Auth is "free" — SSH does it.
• ✅ No new ports exposed on the public internet.
• ✅ Works through corporate firewalls if SSH already does.
• ❌ Need to bootstrap dreamcoder-sidecar onto the remote (one-time install or auto-upload binary).
• ❌ Tunnel lifetime tied to a process — reconnect logic needed.
• ❌ Windows SSH server story is messier than Linux.

Path B — Remote-installed sidecar, local GUI connects via WebSocket

[ remote host ]  dreamcoder-sidecar --serve --auth=token  ◄──(wss + token)── [ Local DreamCoder GUI ]

How: User installs dreamcoder-sidecar on the remote (binary or cargo install or bun install -g), starts it as a service, and pairs the local GUI to it (host + token, like the H5 pairing flow but in reverse direction).

• ✅ Survives across multiple local clients (laptop + work desktop pairing to the same remote box).
• ✅ Reuses H5's token rotation, QR pairing UX, request classifier.
• ❌ Now there's a port to expose on the remote — needs TLS, ideally behind reverse-proxy or only on a private network.
• ❌ Operational story: who keeps the sidecar daemon alive? (systemd template? launchd? pm2?)

Path C — Reuse H5 protocol, run headless DreamCoder on remote, local GUI as the "phone"

[ remote host ]  dreamcoder --headless (full server, H5 endpoint)  ◄──(wss, H5 protocol)── [ local GUI in "client" mode ]

How: We already have connectHeadless.ts and createDirectConnectSession.ts from v0.4.0. The remote box runs the full DreamCoder server in headless mode (no Tauri shell, just the bun server). The local desktop GUI gets a "Connect to remote DreamCoder" mode and talks the same H5 protocol the phone speaks today.

• ✅ Maximum code reuse — protocol, auth, token rotation, request classifier are already live.
• ✅ One mental model for both directions: the desktop is just a richer H5 client.
• ✅ Symmetrical — same binary works as host or client.
• ❌ Headless-mode server needs hardening: process lifecycle, log rotation, multi-session isolation, resource limits.
• ❌ The desktop GUI's existing assumption that the sidecar is 127.0.0.1 needs auditing — anywhere we trust loopback today must be re-checked under remote topology.

Tentative recommendation

Default: Path C with Path A as a 5-minute fallback.

• C maximizes leverage from the v0.4.0 H5 stack and gives users a single coherent story.
• A is trivially achievable for power users today as a documented recipe ("run DreamCoder over SSH tunnel") even before C ships — could be the v0.5 stop-gap while C bakes.
• B is what C looks like in production when paired with auto-start service templates; it's a deployment shape, not a separate architecture.

---

Open questions

1. Auth model: Token + TLS cert pinning? mTLS? Reuse H5's rotating token? OAuth-device-code? What's the minimum bar that won't get someone fired by their security team?
2. Discovery: How does the local GUI find the remote? Manual host:port + token? mDNS on private networks? A small "DreamCoder Hub" relay (opt-in)?
3. File operations: Do we surface remote file diffs in the GUI (currently we read locally)? Stream them over the wire? Cache?
4. Terminal: portable-pty lives on the remote in this design — does the existing xterm.js client need protocol changes for remote PTY resize / signal forwarding?
5. MCP servers: Remote-side or local-side? Probably remote (they need filesystem access), but some MCP servers are local-only by design (e.g. Apple Notes). Do we support both ends simultaneously?
6. Updater: How does dreamcoder-sidecar (or headless DreamCoder) get updated on the remote? Pinned version + manual update? Auto-update from a configured endpoint?
7. Computer Use: On a headless Linux box without a display server, screenshot mode is moot. UIA tree mode is Windows-only. Does Computer Use just disable itself in remote mode, or do we ship something like xvfb-based headless screenshotting?
8. Resource limits: One remote box, multiple DreamCoder sessions, multiple users — what's the isolation story? Per-session subprocess? cgroup? Just trust the OS user account?

---

Proposed work breakdown (subject to scope)

1. Spike: prove Path C in a branch — local desktop GUI connects to a headless DreamCoder over WebSocket on a LAN box. (1-2 days)
2. Auth hardening: extend H5 token model to remote topology; remove any lingering "trust loopback" assumptions. (~1 week)
3. Headless lifecycle: systemd unit + launchd plist + Windows service templates; log rotation; PID file. (~3 days)
4. GUI side: "Add remote connection" flow — host + paste token, save profile, switch profile. Mirror the existing H5 setup UX. (~1 week)
5. Path A docs: write the SSH-tunnel recipe for users who want it today. (~half a day)
6. Computer Use mode: gate / disable on headless Linux for v1. (~1 day)
7. MCP: decide local-vs-remote per server in settings; default = remote. (~3 days)
8. Docs + sample setup: "Install DreamCoder on a GPU server" walkthrough. (~2 days)

---

How to participate

• 💬 Comment on this issue with answers to any of the open questions, or push back on the candidate paths.
• 🔬 Try the SSH-tunnel recipe once we publish it (Path A) and report back what breaks — that data informs the auth/lifecycle hardening for Path C.
• 🛠 Pick up a sub-task from the work breakdown if you want to ship something. Mentor available — @GoDiao.

cc @v2less (original requester in #3) — you're the source for this RFC. Would love your take on the candidate paths and which workflow shape best matches your day-to-day.

cc @pocca2048 — credit to your codex#10450 issue, which #3 mirrored. Cross-pollination welcome if you're around.

Tagging interested area: rfc, enhancement, area:remote, phase-5.