Inbox stream scaling: per-Workspace event routing and a single cross-process poller

Start with src/lib/events.ts, src/lib/inbox-stream-client.ts, and src/app/api/inbox/stream/route.ts, then read ADR-0003 and the existing src/lib/__tests__/inbox-stream.test.ts and events.test.ts. Trace how streams subscribe, seed their Workspace baseline, and poll for cross-process updates before changing the shared routing and poller. Run the named Vitest suites and required verification; done means one poll per tick, Workspace-scoped delivery, clean disconnects, and the documented isolation and scaling checks pass.

Background

The desk inbox stream (GET /api/inbox/stream, src/app/api/inbox/stream/route.ts, ADR-0003) is correct for multi-tenancy (events are filtered by Workspace, covered by a two-Workspace test) but its cost scales with the number of open connections, not with the number of changes. That is fine for a handful of tenants and a poor fit for hundreds of tenants with several agent tabs each. This issue fixes the scaling shape without changing observable behaviour or the ADR-0003 delivery guarantees.

Follow-up to the earlier inbox-stream hardening (cross-process refresh, isolation tests); it builds on connectInboxStream in src/lib/inbox-stream-client.ts and on the named heartbeat event.

Current mechanics (what exists today)

1. One shared in-process bus (src/lib/events.ts): a single EventEmitter on globalThis, one "conversation" channel, setMaxListeners(200). notifyConversationEvent emits for every Workspace.
2. Each stream connection registers its own bus listener that discards events where event.workspaceId !== workspaceId -> every event costs O(open connections) listener invocations. Two connections per browser tab (list pane + conversation view) double the count. Past 200 connections Node prints MaxListenersExceededWarning.
3. Each stream connection runs its own DB poll every DB_TICK_MS = 10 s: conversation.findMany({ where: { workspaceId }, orderBy: updatedAt desc, take: 20 }) to compute a signature that wakes cross-process writers. Cost is connections / 10 s queries regardless of activity (e.g. 300 tenants x 2 connections = ~60 queries/s idle).

Scope

A. Per-Workspace event routing (replaces the global bus fan-out)

• Key the bus by Workspace: maintain Map<workspaceId, Set<listener>> (or per-Workspace emitters) in src/lib/events.ts. notifyConversationEvent delivers only to that Workspace's listeners; delivery cost becomes O(connections in that Workspace).
• Keep the public API (notifyConversationEvent, subscribeToConversationEvents) compatible, or migrate all callers in the same PR. subscribeToConversationEvents takes the workspaceId to subscribe to; entries are removed when their set empties (no leak for departed tenants).
• Remove the now-unneeded setMaxListeners(200) workaround, or justify keeping it.
• The globalThis singleton must still work across Next.js route-bundle duplication (the reason it is on globalThis today).

B. One cross-process poll per process, fanned out per Workspace

• Replace the per-connection setInterval + findMany with a single process-level poller that runs only while at least one stream is connected, and only for Workspaces that have at least one connected stream.
• Preferred shape: one query per tick across all active Workspaces (e.g. group by workspaceId, max(updatedAt) plus the existing top-N signature if the top-N semantic must be kept), compared to the previous per-Workspace signature; a change emits one {conversationId:"*", kind:"refresh"} to that Workspace's listeners only.
• Preserve the ADR-0003 spike finding: the baseline for a Workspace is seeded when its first stream connects (a null baseline must never swallow a write landing between connect and the first tick). Also preserve behaviour for a Workspace whose first stream connects while the poller is already running.
• Poller stops when the last stream disconnects (no timers left running; important for tests and graceful shutdown).
• Failure handling: a transient DB error skips the tick and retries; do not swallow it silently - log it once per failure streak with the Workspace-agnostic error message.
• Keep the 10 s bound for cross-process wake (ADR-0003). PostgreSQL LISTEN/NOTIFY is explicitly out of scope (separate decision, see ADR-0003 "upgrade path").

Out of scope

• Changing the SSE wire format, event names, the 15 s heartbeat event, or the client (connectInboxStream); the client must need no changes.
• Sharing one EventSource between list pane and conversation view (tracked separately).
• The widget visitor stream (/api/widget/stream) and its 2 s poll.
• Redis/NATS/LISTEN-NOTIFY or any new infrastructure/dependency (needs a separate maintainer decision).
• Per-event dedupe/coalescing on the client, rate limiting, or auth changes.
• Any schema change or new migration. If one looks necessary, stop and raise it.

Guardrails

• Workspace isolation is a security boundary (ADR-0001). The workspaceId used for routing must come from the authenticated membership server-side (as today), never from the request. A listener must be unable to receive another Workspace's event under any code path, including the poller's fan-out and a poller tick that errors partway.
• Self-hosted mode (one Workspace) must behave identically and must not require extra services.
• No behavioural regression: heartbeats, ready frame, cross-process refresh, resync-on-reconnect.
• Do not introduce unbounded per-tenant state: all maps/sets must shrink on disconnect.
• Matches the repo's rule: one coherent PR, <600 meaningful changed lines.

Verification requirements

• Unit/integration (vitest, PGlite):
  • Existing src/lib/__tests__/inbox-stream.test.ts and events.test.ts still pass (update events.test.ts for the new subscribe signature).
  • The two-Workspace isolation test stays and passes unchanged (bus path and poller path). Add: Workspace A connects while Workspace B's stream already exists; a write in A wakes only A.
  • Listener bookkeeping: subscribe/unsubscribe leaves the map empty; N connections in one Workspace and M in others -> an event invokes exactly that Workspace's N listeners (assert via listener spies, not timing).
  • Poller: exactly one DB query per tick regardless of connection count (spy on prisma), poller timer is cleared when the last stream disconnects, baseline seeded per Workspace at first connect (port the 'baseline fix' and 'does not fire for pre-connection writes' tests).
• Scale evidence (in the PR description): a small throwaway script or test run showing idle query rate and per-event listener invocations before vs after for e.g. 200 simulated connections across 100 Workspaces. Do not commit the script.
• npm run verify (lint, typecheck, tests, build) plus the desk-live-tickets and widget-liveness Playwright specs (both must pass unchanged).
• PR description answers: root cause or band-aid? what does it not fix? what risk does it add?

Documentation

• Update ADR-0003 "Scaling notes" section to reflect the new routing/poller shape and remove the "known limits" it lists once resolved.
• Update docs/architecture.md if it describes the event bus.

Acceptance

Idle DB load from the inbox stream is O(active Workspaces) per 10 s (one query), event delivery is O(connections in the originating Workspace), no cross-tenant delivery, all verification above green.