Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Converting a FetchHttpClient response stream drops workerd's known-length capability

Chiusa
  • Qualcosa non va?
#8,205 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Una pull request collegata è già stata integrata.

  • #9673 di @effect-bot — integrata

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
typescript
Ambito
backend

Direzione di ricerca

Start by reproducing the failure with FetchHttpClient, HttpClientResponse.fromWeb(), Stream.fromReadableStream, and Stream.toReadableStreamEffect() in the issue's Vitest example. Inspect how the response body is wrapped and converted, then verify that direct conversion preserves the source body's known-length capability without changing arbitrary transformed streams. Done means the shown R2 write succeeds and the regression case passes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What version of Effect is running?

  • Effect 4.0.0-rc.112
  • @cloudflare/vitest-pool-workers 0.22.0
  • Miniflare 5.20260815.0-alpha / workerd 1.20260815.1

What steps can reproduce the bug?

import { env } from "cloudflare:workers"
import { Effect, Stream } from "effect"
import {
  FetchHttpClient,
  HttpClient,
  HttpClientRequest
} from "effect/unstable/http"
import { expect, it, vi } from "vitest"

it("drops the native body's known length after Effect conversion", async () => {
  const bytes = new Uint8Array([1, 2, 3])
  const url = "https://example.com/archive.tgz"
  vi.stubGlobal("fetch", () => Promise.resolve(new Response(bytes)))

  const nativeResponse = await fetch(url)
  if (nativeResponse.body === null) throw new Error("Missing response body")
  await expect(
    env.BACKUP.put("native.tgz", nativeResponse.body)
  ).resolves.not.toBeNull()

  const program = Effect.gen(function*() {
    const request = HttpClientRequest.get(url)
    const response = yield* HttpClient.execute(request)
    const body = yield* Stream.toReadableStreamEffect(response.stream)

    yield* Effect.tryPromise(() => env.BACKUP.put("effect.tgz", body))
  }).pipe(Effect.provide(FetchHttpClient.layer))

  await expect(Effect.runPromise(program)).rejects.toThrow(
    "Provided readable stream must have a known length"
  )
})

The R2 operation rejects with:

Provided readable stream must have a known length
(request/response body or readable half of FixedLengthStream)

In the same runtime, the native response-body write succeeds. FixedLengthStream.readable also succeeds.

What is the expected behavior?

For the direct conversion shown above, Stream.toReadableStreamEffect(response.stream) should produce a Web stream that retains the source body's runtime-known length and is accepted by R2.

What do you see instead?

The public HttpClientResponse body API is an Effect Stream. Converting it with Stream.toReadableStreamEffect() creates a generic Web ReadableStream, which does not retain the runtime-internal identity-length capability that R2 queries.

The body bytes are unchanged, but R2 rejects the converted stream. This prevents direct streaming through HttpClientResponse.stream without reconstructing the length or buffering the body.

Additional information

HttpClientResponse.fromWeb() keeps the original Response in a private field. HttpClientResponse.stream wraps Response.body with Stream.fromReadableStream. The HTTP client also adds interruption and finalization wrappers. Stream.toReadableStreamEffect() then creates a new Web stream, so workerd can no longer query the source body's identity length.

Cloudflare documents ReadableStream as a valid R2Bucket.put() value, but workerd's R2 implementation requires a stream to report an identity length before it starts the write. This check is in workerd itself, not only in Miniflare.

This request concerns direct conversion of response.stream. An arbitrary transformed, filtered, concatenated, or generated Effect stream does not necessarily have a known length.

A caller can read Content-Length, create a Cloudflare FixedLengthStream, and pump the Effect stream into it with Sink.fromWritableStream. This is a platform-specific manual bridge, and it works only when the header is present and equals the number of bytes emitted by the stream. Buffering through response.arrayBuffer is another workaround when the body fits safely in memory.

Our workaround fetches archives with the platform fetch() and passes the original Response.body to R2. Effect still owns retries, errors, and timeouts around that operation. Packument requests continue to use FetchHttpClient.

Lingua principale
TypeScript
Stelle
16.7k
Fork
808
Merge medio
10h 13m
PR unite (30g)
800

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Effect-TS/effect

Tutte le issue di Effect-TS/effect

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.