Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

`HttpApiSchema`: stream bodies in a custom framing

Aperta
  • Qualcosa non va?
#9,910 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@effect-bot ci sta già lavorando.

Dal 25/10/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

docker-made-me-do-it enhancement

Problem

HttpApi streams are raw bytes (StreamUint8Array), line-delimited JSON (StreamNdjson) or Server-Sent Events (StreamSse). Many APIs stream in a framing of their own:

  • Docker's application/vnd.docker.multiplexed-stream (logs, attach, exec): an 8-byte header [stream, 0, 0, 0, size (uint32, big-endian)], then size bytes of stdout, stderr or an error message.
  • Connect and gRPC-Web streaming: a 5-byte envelope (flags, length) around each message.
  • AWS event streams, and length-prefixed protobuf in general.

Such an endpoint is declared as StreamUint8Array. Every client and every server then frames and unframes the bytes by hand, outside the declaration:

success: HttpApiSchema.StreamUint8Array({ contentType: "application/vnd.docker.multiplexed-stream" })

// client, at every call site
client.containers.logs(request).pipe(demultiplex) // Stream<Uint8Array> → Stream<LogFrame>

The OpenAPI document and the types only say "bytes", while the API's elements are frames.

Proposal

A stream schema built from a framing: functions between the body's bytes and the encoded elements, and an element schema.

const Multiplexed = HttpApiSchema.StreamFramed({
  contentType: "application/vnd.docker.multiplexed-stream",
  schema: LogFrame, // { stream: "stdout" | "stderr", data: Uint8Array }
  decode: (bytes) => demultiplex(bytes), // Stream<Uint8Array, E> → Stream<LogFrame.Encoded, E | FramingError>
  encode: (frames) => multiplex(frames), // Stream<LogFrame.Encoded, E> → Stream<Uint8Array, E>
  maxBytes: "1 GiB"
})

HttpApiEndpoint.get("logs", "/containers/:id/logs", { params: { id: Schema.String }, success: Multiplexed })
// client: Stream<LogFrame, …>; server handler: returns Stream<LogFrame>
  • Client: response bytes → decode → each element decoded with schema. A framing error fails the stream like a malformed NDJSON line does today.
  • Server: handler elements encoded with schema → encode → response bytes.
  • Payloads: the same schema works as a streaming request payload (for example Connect client streaming), with decode on the server and encode on the client.
  • OpenAPI: the media type with type: string, format: binary, as for StreamUint8Array, plus the element schema in an extension for generators.

StreamNdjson is then this schema with a line framing, which keeps the two consistent. With several media types per stream (#9909), one schema could carry a framing per media type, so Docker's logs decode to LogFrame whatever the Content-Type:

const Logs = HttpApiSchema.StreamFramed({
  contentType: {
    "application/vnd.docker.raw-stream": rawFraming, // { decode, encode }
    "application/vnd.docker.multiplexed-stream": multiplexedFraming
  },
  schema: LogFrame
})

Scope

In scope: request and response streams over a plain HTTP body.

Out of scope: upgraded connections (HttpApiSchema.Upgrade), whose two directions would need a typed duplex. That can follow once this exists.

Cost

  • A new kind of stream for the client, the server, payloads, limits and OpenAPI, with functions in the declaration rather than data.
  • OpenAPI can only describe the bytes (format: binary). Even 3.2's itemSchema covers only the line- and event-based media types it defines, so the framing needs an extension.
  • Protocols with a framing of their own (Connect, gRPC-Web) usually come with their own RPC layer rather than HttpApi, so there may be few users. Without this, a client applies a plain function to a StreamUint8Array. Docker's client does that in three places.

Open questions

  • Are decode and encode Stream → Stream functions, or Channels? Stream functions are simpler to write; Channels compose without wrapping.
  • Should a framing be able to emit typed failures, such as Docker's error frame (type 3) or a Connect end-of-stream with an error? That would match the error option proposed for StreamNdjson (#9906).
Lingua principale
TypeScript
Stelle
16.7k
Fork
808
Merge medio
10h 13m
PR unite (30g)
800

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Effect-TS/effect

Tutte le issue di Effect-TS/effect

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.