feat(cli): run engram serve in the background as a daemon PR #2229

Open
  • Is something wrong?
#2,228 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@carolitascl is already working on this.

Since Nov 1, 2026.

  • #2229 by @carolitascl — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
docker, go, linux, macos, shell, sqlite
Domain
cli, devtools

Research direction

Start with the serve command entry point and the code path that logs HTTP server did not bind and exits 0, since the exit-code fix is the smallest self-contained piece. The daemon, pidfile and service install work touches process lifecycle and the docs templates added in #279, so read those before designing anything. Done means a bind failure returns non-zero and the daemon verbs are agreed with maintainers first.

Written by the indexing model from the issue text.

Description

status:needs-review type:feature
Existing Issue Search
  • I searched existing issues and will add evidence to a canonical tracker when applicable.
🔍 Observed Consequence

Running engram serve as a persistent background daemon currently requires hand-authoring OS-specific supervisor units from the docs (launchd plist on macOS, systemd unit on Linux, Task Scheduler on Windows). Three concrete pain points, all reproduced today on a real macOS setup:

  1. Silent staleness of manual units. The launchd template needs literal absolute paths (launchd does not expand $HOME). After the install method changed (Homebrew /opt/homebrew/bin/engram → a plain ~/.local/bin/engram install), the previously installed plist kept pointing at the old binary path. The service was disabled and unloaded — the daemon silently stopped surviving reboots, with no diagnostic pointing at the cause.

  2. No CLI verbs for daemon control. Start/stop/status means remembering launchctl/systemctl incantations and plist paths. There is no engram serve --status, no --stop, no way to ask engram itself whether a daemon is running.

  3. Bind failure exits 0. When the port is already taken, serve logs HTTP server did not bind on 127.0.0.1:7437: an existing instance owns the port and exits with code 0. Under a KeepAlive supervisor this produces a silent restart loop every throttle interval — the stderr log fills with bind failures while the supervisor's run counter just climbs. A supervisor cannot distinguish healthy from crash-looping, and users see nothing.

A related collision: MCP clients (e.g. the Pi adapter) spawn their own transient engram serve per session, so a supervised daemon and a session-owned instance race for the port with no defined ownership or reuse contract.

💡 Desired Outcome

CLI-native daemon support that works regardless of how the binary was installed:

engram serve --daemon        # fork to background, write pidfile, print PID/port, exit 0
engram serve --status        # running (pid N) / not running / unreachable
engram serve --stop          # stop the daemon instance tracked by the pidfile
# or, as a superset:
engram service install       # render + install the platform unit (launchd/systemd/schtasks) with the CURRENT binary path

Expected behavior:

  • --daemon backgrounds the process (double-fork or equivalent), records a pidfile under the data dir (e.g. ~/.engram/serve.pid), prints the PID and port, exits 0.
  • A second serve against a live instance detects it (pidfile + port probe) and reports already running (pid N) with a non-zero exit code by default, so callers can decide to reuse or replace.
  • Port-in-use returns a non-zero exit code (today it is 0), keeping the clear message. Any supervisor (launchd KeepAlive, systemd Restart) can then react correctly instead of hot-looping.
  • service install renders the existing docs templates with the real binary path and home directory, removing the literal-path fragility that made manual units go stale.
📦 Affected Area

CLI (commands, flags)

Release Context

not release-specific

🔄 Workarounds and Constraints
  • Manual docs templates (added via #279): they work, but are copy-paste-heavy, go silently stale when the binary path changes (hit today: a launchd agent pointing at a removed Homebrew path), and every control verb is OS-specific shell knowledge.
  • Homebrew service stanza (#1357): right direction for brew installs, but does not cover other install methods (e.g. a binary at ~/.local/bin), and the CLI itself still has no daemon verbs.
  • nohup engram serve &: no supervision, no restart on crash or reboot, dies with the session.
📎 Additional Context

Repro notes from today (macOS, arm64):

  • A launchd agent with KeepAlive=true running the current serve: while another process held port 7437, engram serve exited 0 every ~10 seconds; stderr showed only repeated HTTP server did not bind on 127.0.0.1:7437: an existing instance owns the port lines. launchctl kept respawning — an invisible crash-loop.
  • The stale-unit case: plist referencing /opt/homebrew/bin/engram after migrating to ~/.local/bin/engram — service disabled and unloaded, nothing surfaces the mismatch.

Related: #279 (closed — docs templates + Local daemon: status line), #1357 (closed — Homebrew formula service integration). This request is complementary: CLI-level and install-agnostic, and it also gives programmatic clients a defined attach/reuse contract instead of racing the port.

Dominant language
Go
Stars
7.1k
Forks
729
Avg merge
10h 46m
Merged PRs (30d)
268

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Gentleman-Programming/engram

All issues in Gentleman-Programming/engram

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.