Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

sandbox-relay: no SIGTERM teardown, so a relay restart leaves stale presence records and turns in the gap get an in-turn tool error instead of a 503

Open
  • Is something wrong?
#453 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
node.js, typescript
Domain
backend

Research direction

Start in packages/sandbox-relay/src/main.ts, where the servers are started with no signal handling, then read packages/sandbox-relay/src/relay.ts to see what an attach stream's teardown does (remove from sh:sandbox:records, write sh:sandbox:detached:<id>, awaited). Wire SIGTERM to stop accepting new streams, run teardown for every parked session, and exit within a short deadline; add the integration test from the Acceptance section (SIGTERM with two attached workers → both records gone, both marks written before exit). Optionally trace routeExec in the harness for the no live worker for sandbox path and check spec P6.3 §5's relay-restart row against the new behavior.

Written by the indexing model from the issue text.

Description

bug

Found during the P6.3 acceptance (#425, Task 18 step 4) on the shared KVM rig.

What happens

sandbox-relay has no SIGTERM handler. packages/sandbox-relay/src/main.ts starts the servers and installs no signal handling. So systemctl restart sh-relay (and a pod deletion or rollout on K8s) ends the process on the default signal action, and no attach stream's teardown runs (packages/sandbox-relay/src/relay.ts). As a result:

  • the presence records in sh:sandbox:records are not removed;
  • no detach mark (sh:sandbox:detached:<id>) is written.

Spec P6.3 §5, row "Relay restart", says: "On a graceful restart every record disappears (teardown), then the workers re-Hello … Turns in the gap get 503: SandboxAffinityPendingError …". Every real restart is instead the row's other case, "A hard-killed relay can leave stale records, which turns may lease until the worker reattaches".

Measured (rig, VM path, 4 sandboxes)

The relay was stopped at 17:34:50.107 and listening again at 17:34:51.779. Workers reattached 1.8–4.7 s after the stop. A poller on the rig never saw a record disappear or a detach mark appear.

A microVM session's turn sent into that gap was leased to its affine sandbox: the record was still present, but its worker had not reattached. The exec failed with:

2 UNKNOWN: no live worker for sandbox 'moca_microvm_0'

The failure reached the model as the bash tool's output. The model retried and succeeded, so the workspace was intact and the session stayed on its sandbox. The turn never got a 503 with Retry-After.

Why it matters
  • In the reconnect gap a turn gets an error inside the turn, not a retryable 503. A model may report it as a failure.
  • The affinity grace clock (detach marks) never starts on a relay restart. This is harmless for a few seconds, but it means the grace path in spec §4 is not what protects sessions on a restart; the stale records are.
Proposed fix
  1. Relay: on SIGTERM, stop accepting new streams and run teardown for every parked session. That removes its record and writes its detach mark, with the remove awaited. Then end the streams and exit, bounded by a short deadline under systemd's TimeoutStopSec and the pod's terminationGracePeriodSeconds.
  2. Optionally, harness: classify no live worker for sandbox from routeExec as saturated, so the turn answers 503 rather than passing the text to the model. This also covers a genuinely hard-killed relay.
Acceptance
  • A relay integration test: SIGTERM with two attached workers removes both records and writes both marks before exit.
  • On a rig or Kind: a turn sent during a relay restart gets 503 (pending) and then succeeds on its own sandbox.
  • Spec §5's relay-restart row matches the result.
Dominant language
TypeScript
Stars
5
Forks
11
Avg merge
8h 32m
Merged PRs (30d)
166

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rossoctl/moca

All issues in rossoctl/moca

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.