bug: Streams that are not SSE (Bedrock, Ollama NDJSON) reach the client only when they end

Aperta
#1,880 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
68/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
go

Direzione di ricerca

Start in core/listener/forwardproxy/server.go around lines 972 and 1015, then read the existing fix for #1879. Reproduce with a Go test that sends five flushed event-stream lines at 400 ms intervals, with and without a body-reading plugin. Done means supported non-SSE streams reach the client incrementally and are not buffered until completion.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug laptop

Problem

The proxy streams a response only when its type is text/event-stream. It buffers all other streaming responses until they are complete. These streams arrive at the client only at the end:

  • Bedrock invoke-with-response-stream (application/vnd.amazon.eventstream)
  • Ollama /api/chat (application/x-ndjson)
  • Gemini streamGenerateContent without alt=sse

These streams also get a 502 when they are larger than 10 MB (see #1879).

Reproduction (main at 676557d35)

A Go test that builds forwardproxy.NewServer with one ReadsBody plugin. The upstream sends Content-Type: application/vnd.amazon.eventstream and writes one line every 400 ms, five times, with a flush after each line.

Result: the client gets the first line after 2.007 s, which is when the upstream ends the stream. Without the proxy, the first line arrives at about 0 s.

Cause

core/listener/forwardproxy/server.go:972: httpx.IsEventStream is the only test for a stream. All other responses go to the buffered branch at server.go:1015 when any plugin reads bodies. inference-parser always reads bodies.

Fix

Use the same fix as #1879: buffer only parsed traffic. Also send ContentLength == -1 responses through flushWriter when no plugin must read them.

Impact if not fixed

Claude Code on Bedrock through Cortex shows no streamed output. Local Ollama and Gemini clients that use these formats have the same problem.

Lingua principale
Go
Stelle
26
Fork
46
Merge medio
9h 55m
PR unite (30g)
257

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di rossoctl/cortex

Tutte le issue di rossoctl/cortex

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.