Hacktoberfest 2026:メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

feat(gateway): relayed bodies travel on byte channels

クローズ
#558 コメント 0 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@V3RON がすでに取り組んでいます。

2026年10月14日 から。

  • #570 @V3RON による — マージ済み

評価

この issue はまだ評価されていません。

説明

task:ready

Part of #552.

Scope

After this PR, both daemons wrap the uplink with the link (#556), and a
relayed HTTP exchange carries its bodies on a byte channel instead of
base64 relay.body calls and relay-chunk pushes (ADR 0028 §4). Small
calls no longer queue behind relayed bodies, and a slow reader slows only
its own transfer. The relay's own slot limits go: a relay waits only when
the gateway already has 64 channels open to its worker. #552's measured
promises (bytes, timings, memory, 2 GiB) are proven by the task after
this one.

Values used here are in #552's Shared values table.

In short

sequenceDiagram
  participant G as gateway relay *
  participant W as worker relay *
  participant T as relay target (agent-device host)
  G->>G: openChannel() (waits if 64 are open)
  G->>W: relay.request {.., channel}
  W->>W: acceptChannel when the line is read, before startup parking
  G->>W: request body on the channel *
  W->>T: Request (body = the channel's readable)
  T-->>W: Response
  W->>W: STOP the request direction if the target did not read it all
  W-->>G: relay-head
  W-->>G: response body on the channel *, then END
  W-->>G: relay.request answer (may come before END)
  Note over G: the caller's body ends only at END

(* = changed.) If this goes wrong, an upload or download through a
gateway arrives corrupt or short, or hangs, and agents on that worker see
commands time out.

Technical spec

Facts
# Fact Source
F1 relay.request, relay.body, relay.cancel are contract operations src/contract/operations.ts:986, :1010, :1024
F2 relay-head and relay-chunk are request-scoped pushes src/contract/pushes.ts:141-142
F3 Chunk and push bounds live in the contract src/contract/relay.ts:7-11
F4 The gateway relay limits 8 relays per worker, 4 unanswered relay.body calls, 256 KiB pieces, 64 MiB unread; a 330 s deadline runs from start over the whole exchange src/gateway/relay.ts:22-27, :56-105, :149-157
F5 The gateway owns the caller's body stream: it ends it when relay.request answers {}, and a caller's cancel sends relay.cancel src/gateway/relay.ts:264-274, :323-337, :353-357
F6 The gateway reports bad relay data from a worker as INTERNAL worker <id> sent an invalid relay response, and an answer with no head as INTERNAL worker <id> ended the relay without a response src/gateway/relay.ts:317-321, :323-333
F7 The worker limits 16 active relays per connection; RelayBodyFeed feeds the target's body and is aborted when the call settles so a sender never waits on a target that stopped reading; #stream sends 256 KiB relay-chunk pushes src/daemon/relay.ts:36, :48-107, :150, :267-304
F8 The worker's server routes relay.request with head and chunk push callbacks, and relay.body / relay.cancel to the dispatcher src/daemon/server.ts:1108-1113, :1314-1336
F9 Dispatch parses input, checks access, parks until startup converges, then runs the handler; the relay handlers refuse FORBIDDEN off the gateway uplink src/daemon/dispatch.ts:314-325; src/daemon/dispatcher.ts:581-588
F10 DaemonServer#acceptUplink takes the worker's uplink; every connection gets a RelayConnection src/daemon/server.ts:651, :673
F11 The dispatch session carries onRelayHead and onRelayChunk; the gateway's own dispatcher refuses the three relay operations src/daemon/dispatch.ts:121-126; src/gateway/dispatcher.ts:201-203
F12 The worker hands its dialled uplink to accept; the gateway builds a WorkerLink on the accepted uplink, whose WorkerClientFactory takes an IpcConnection; defaultConnect makes the GatewayWorkerClient src/daemon/gateway-uplink.ts:140; src/daemon/main.ts:406; src/gateway/service.ts:217, :233; src/gateway/worker-link.ts:80-86, :622-624
F13 The gateway reaches a worker's client through liveClient src/gateway/fleet-ports.ts:32-35, :58-62
F14 The client's relay methods and hooks; simlock/admin exports none src/simlock-client/client.ts:312-317, :579-593; src/simlock-client/wire.ts:58-60, :334-350; src/admin/index.test.ts:9-12
F15 The protocol range is { min: 33, max: 33 }, one range for every client and daemon src/contract/protocol.ts:13
F16 Relay test harnesses: startPair (in-memory), startFleet (real WebSocket), openUplink, patternStream, patternDigest; the startup-parking test src/daemon/relay-gateway.e2e.test.ts:153, :178; src/daemon/test-relay-support.ts:102, :110, :184; src/daemon/server.test.ts:1241
F17 Tests of behaviour this PR removes or renames src/daemon/relay-gateway.e2e.test.ts:379, :405, :496, :740; src/daemon/relay-worker.e2e.test.ts:56, :71, :165, :182, :210, :254, :370, :391, :437, :462; src/gateway/relay.test.ts:100, :166; src/contract/operations.test.ts:94, :184; src/admin/index.test.ts:11
F19 relay.request's current fields and bounds src/contract/operations.ts:986-1008
F20 The worker's relay fails a duplicate relayId, a request it cannot build, target headers out of bounds and a response body that fails after the head; a cancel for an unknown id does nothing src/daemon/relay.ts:122-124, :180-182, :204-214, :242-252, :292-302
F21 The server answers a protocol mismatch and DAEMON_STOPPING before dispatch src/daemon/server.ts:775, :786
F22 The gateway clears its deadline and caller-abort listener when the exchange finishes src/gateway/relay.ts:382-390
Modules touched
flowchart LR
  GU["daemon/gateway-uplink.ts *"] --> LK["link (#556)"]
  GS["gateway/service.ts *"] --> LK
  GS --> WL["gateway/worker-link.ts *"]
  WL --> WC["gateway/worker-client.ts *"]
  GR["gateway/relay.ts *"] --> WC
  WC --> CL["simlock-client: client.ts *, wire.ts *"]
  SV["daemon/server.ts *"] --> RL["daemon/relay.ts *"]
  DS["daemon/dispatch.ts *, dispatcher.ts *"] --> RL
  GD["gateway/dispatcher.ts *"]
  CT["contract: operations, pushes, relay, protocol *"]

(* = changed; src/daemon/main.ts passes the new types through.)

Wiring (F12). GatewayUplink#dial wraps the dialled connection with
createLink(…, { end: "worker" }) before accept; GatewayService#accept
wraps the accepted one with end: "gateway" and hands the WorkerLink an
uplink whose connection is the LinkConnection. GatewayUplinkOptions.accept,
DaemonServer#acceptUplink, WorkerClientFactory and WorkerLink's
uplink take a LinkConnection.

Contract (F1–F3, F19). relay.request gains channel, 1–32
characters of [A-Za-z0-9-], as an exported relayChannelSchema that
the operation's schema uses. relay.body, the relay-chunk push, the
chunk bounds and onRelayChunk (F11) go; the gateway dispatcher's
relay.body refusal goes. This PR bumps the protocol by one.

Gateway (src/gateway/relay.ts, worker-client.ts, worker-link.ts).

  • GatewayWorkerClient gains openChannel(options: { signal: AbortSignal }),
    which defaultConnect takes from the LinkConnection it is given. The
    relay methods lose relayBody; the hooks lose onRelayChunk.
  • An exchange opens a channel with a signal that aborts on its deadline
    or its caller's abort, then sends relay.request naming it. The channel
    wait counts against the deadline (F4). WorkerSlots goes.
  • It pipes the caller's request body into the channel's writable, or
    closes the writable at once when there is none. When that writable
    errors with stopped, it stops reading the caller's body.
  • The exchange keeps owning the caller's body stream (F5): it reads the
    channel's readable into its own stream, so errors map as below and a
    caller's cancel sends relay.cancel and aborts the channel. The body
    ends at the channel's end, never at the answer; a {} answer only
    settles the call. The deadline and the caller-abort listener (F22) stay
    live until the channel's end, so a worker that answers and never ends
    the body hits the deadline.
  • On every end of the exchange (answer, failure, cancel, deadline) the
    channel ends or is aborted, so it leaves the 64.

Worker (src/daemon/server.ts, relay.ts, dispatch.ts,
dispatcher.ts).

  • Only on the gateway uplink (isGatewayUplink): when the server reads a
    relay.request line, it parses channel with relayChannelSchema and,
    if it passes, calls acceptChannel before dispatching, so a request
    that parks for startup (F9) holds its channel. If either fails, the
    call later fails BAD_REQUEST. Off the uplink nothing is accepted and
    the handler answers FORBIDDEN as today.
  • RelayConnection owns an accepted channel from that moment. Every way
    relay.request fails aborts the channel: schema, access, protocol
    mismatch and DAEMON_STOPPING (F21), startup failure, duplicate
    relayId, missing target, closed connection, a request it cannot
    build, target error, target headers out of bounds, a response body that
    fails after the head (F20), and cancel. A successful {} answer is
    written only after the response direction's END has been written, and
    does not abort.
  • The target's Request body is a stream the relay owns, fed from the
    channel's readable as the target pulls (null with hasBody: false).
    Once the target has answered, the relay cancels its own reader on the
    channel, which sends STOP for any unread rest; with hasBody: false
    it does so at once. The response body is piped into the channel's
    writable, which then closes. relay-head stays a push. RelayBodyFeed, the
    relay.body handler and MAX_ACTIVE_RELAYS (F7) go.

Failure mapping. How each side reports a channel failure
(LinkChannelError.reason, #556):

Reason Gateway, before relay-head Gateway, after relay-head Worker
invalid INTERNAL worker <id> sent an invalid relay response body errors with that message relay.request fails BAD_REQUEST; target's request aborted
closed WORKER_UNREACHABLE, as an uplink drop today body errors WORKER_UNREACHABLE target's request aborted (as a connection close today)
stalled INTERNAL worker <id> did not read the relayed request body for 60 s body errors with a message saying the reader fell behind for 60 s target's request aborted; relay.request answers { cancelled: true }
not-accepted WORKER_UNREACHABLE worker <id> did not take the relayed request within 30 s — —
aborted by the peer the relay.request answer decides, as today; with no answer by the deadline, the deadline error the relay.request answer decides: its error's message; if it settles without one, INTERNAL worker <id> ended the relayed response early target's request aborted; relay.request answers { cancelled: true }
aborted after this side's cancel as a cancel today as a cancel today —

A request parked for startup whose channel has already failed answers by
this table without running its target.

Other tasks. #531 and #533 (task:ready) and #348 also bump the
protocol and change src/daemon/server.ts, dispatcher.ts and main.ts.
As for those tasks, they run at the same time: whichever merges second
takes the base branch and bumps again.

Contract and event changes
  • relay.request gains channel; relay.body and the relay-chunk push
    go; the chunk bounds go. This PR bumps the protocol by one.
  • No events.
Other code on the same state

The state is a relay's channel and the calls in flight on the uplink.

  • The link (#556): owns the channel limit, credit, lanes, STOP and
    the 30 s and 60 s waits; the relay reads its failure reasons (table).
  • The agent-device front on the gateway (src/daemon/main.ts:1058):
    unchanged; it gets a Response as before.
  • The agent-device host's 2 GiB check (#557): acts on the target's
    response on the worker before the relay pipes it; an errored body is a
    worker-side aborted channel (table). Either order of merge works.
  • WorkerLink refresh calls (10 s timeout,
    src/gateway/worker-link.ts:43) and forwarded lease calls: now go
    in the link's priority lane.
  • Worker startup parking (F9): the channel is accepted before
    parking; an upload parked over 60 s fails stalled.
  • Connection close (src/daemon/server.ts:1576): still aborts every
    relay; the link fails every channel with closed.
Rules in play
  • architecture.md: rule 10 (the channel limit lives only in the link;
    the channel bound only in relayChannelSchema), rule 11 (the deadline covers the
    channel wait).
  • safety.md: rule 10, channel is wire input: its shape is checked
    with relayChannelSchema before it is used to accept a channel.
Tests

Seam: startFleet (real gateway and worker over the real WebSocket, fake
relay target) and startPair (in-memory) in
src/daemon/relay-gateway.e2e.test.ts, and openUplink in
src/daemon/relay-worker.e2e.test.ts, whose test end now wraps its pair
end with the link. The startup tests are modelled on
src/daemon/server.test.ts:1241. The tests in F17 are deleted or
rewritten as below.

  • through startFleet, the worker's server and the gateway's worker link hold link connections, and a relayed GET crosses on a channel
  • a 20 MiB PUT body arrives at the target byte for byte, and a 20 MiB response body comes back byte for byte (as today)
  • an empty upload (0 bytes) and a GET relay as today
  • with 63 relays to one worker held in their targets, a 64th starts at once; a 65th starts once one of them ends; a 65th whose 200 ms deadline passes first fails WORKER_UNREACHABLE with the deadline message; a 65th whose caller aborts while it waits fails at once with the abort reason
  • after 64 relays to one worker that were cancelled, timed out, failed with a missing target, or whose target answered without reading its upload, a 65th starts at once
  • through openUplink, a relay.request that fails on its schema, a duplicate relayId, a missing target, a target error, target headers out of bounds, a response body failing after the head, or startup failure leaves the test end's channel errored aborted
  • a duplicate active relayId fails BAD_REQUEST, and a relay.cancel for an unknown id answers {} (as today)
  • a worker that answers {} and never ends the response body: the caller's body errors with the deadline message
  • with the worker's transport held so the {} answer overtakes a 20 MiB response, the caller still gets every byte, and its body ends only at the channel's end
  • a target that answers 401 without reading its 20 MiB upload: the caller gets the 401 and its body, and the gateway stops reading the caller's body
  • a hasBody: false relay whose channel still receives bytes: the worker stops that direction and the gateway's writer errors stopped
  • each failure reason maps as the table says, on the gateway before and after the head and on the worker
  • a caller's cancel of the body stream sends relay.cancel, aborts the channel and aborts the target's request
  • a relay.request on the uplink naming a channel the gateway never opened or one already accepted fails BAD_REQUEST; with channel missing or out of bounds it fails BAD_REQUEST
  • relay.request and relay.cancel from a session that is not the gateway uplink answer FORBIDDEN (as today)
  • a relay sent while the worker is still starting holds its channel, and its 2 MiB upload arrives in full once startup converges
  • a relay whose 8 MiB upload stays parked by worker startup for 60 s fails INTERNAL on the gateway with the 60 s message
  • a parked relay whose channel failed while it waited answers by the table without running its target
  • a GET, a HEAD, and 204, 205 and 304 answers relay with a null body (as today)
  • caller abort, request-body error, deadline, uplink drop, no uplink, missing target, header filtering, 101 headers, target throws, body error after head, and relay plus device.exec isolation behave as today (relay-gateway.e2e.test.ts:332, :527–:706; relay-worker.e2e.test.ts:101–:147, :409–:427; :437 rewritten to read the response from the channel)
  • OPERATIONS has no relay.body, PUSH_SCHEMAS no relay-chunk, and SimlockAdminClient as simlock/admin exports it has no relay methods
  • a worker one protocol version behind the gateway is listed as incompatible (as today, with the bumped range)

Done when

  • Every Tests line passes.

Out of scope

  • The 2 GiB download cap (#557).
  • Any change to the link itself (#556).
  • Live streaming.
  • The measured promises and the slow-lane timing (#559).

Depends on

  • #556

Written by an agent.

主要言語
TypeScript
スター
19
フォーク
1
平均マージ
7時間 57分
マージ済み PR(30日)
213

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

callstackincubator/simlock のほかの issue

callstackincubator/simlock の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。