Stream write fails with HTTP 409 "inconsistent sequencing state" with concurrent writers (5.5.0, HTTP transport)

Abierto
  • ¿Algo no está bien?
#5,418 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
15/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
nodejs, typescript
Área
backend, distributed-systems

Línea de trabajo

The failure comes from the HTTP stream write path: a PUT to /api/v2/runs/<runId>/stream/<streamName> that returns 409 and is wrapped as a StreamError. Start there, then reproduce two concurrent getWritable() writers on the HTTP transport to see whether the conflict is reachable from client code. Done means either the concurrent-write conflict no longer occurs, or the 409 is classified as a non-retryable dropped write. The issue suggests the server side may be at fault, so this may end in a server change.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

On Vercel, a step that writes to a run's default stream intermittently fails with HTTP 409 and the server message Redis stream "strm_<runId>_user" has inconsistent sequencing state. The failure is deterministic once it starts: every retry of the step gets the same 409, so the step exhausts its retries and the run fails, even though the user-facing work in a parallel step had already succeeded.

Environment

  • workflow / @workflow/core / @workflow/world-vercel 5.5.0
  • Vercel world, Fluid compute, Node.js runtime
  • Streams pinned to the HTTP transport: WORKFLOW_STREAMS_TRANSPORT=http and WORKFLOW_EVENTS_TRANSPORT=http

What happens

  • Two steps of the same run write to the run's default stream at the same time: one writes model output chunks (getWritable() in a "use step" function, lock released in finally), the other writes small progress parts. The streaming docs say concurrent writers are allowed.
  • One write gets:
StreamError: Stream write failed: HTTP 409 (PUT https://vercel-workflow.com/api/v2/runs/<runId>/stream/strm_<runId>_user)
{"success":false,"error":"conflict","message":"Redis stream \"strm_<runId>_user\" has inconsistent sequencing state"}
  • Both writer steps then fail the same way on each retry (Max retries reached, bubbling error to parent workflow), and the run ends with FatalError / USER_ERROR wrapping the StreamError (STREAM_ERROR).
  • In our case the run was still executing on an older deployment while a newer one had been promoted; we have not confirmed whether that matters.

Expected

  • Concurrent writers to the same stream should not put the stream into an inconsistent state, or
  • if the state is unrecoverable, the error should be classified so callers can treat it as a dropped write rather than retrying an append that can never succeed.

Notes

  • On the HTTP transport the client sends no sequence or writer id, so the inconsistent state does not seem reachable from client behaviour; it looks server-side.
  • Possibly related: #4306 (acknowledged chunks missing under concurrent load), #5383 (stream socket closed mid-run on 5.4.0). Separately, #4414 means a step that catches this error still fails after its body returns, which turns a dropped progress write into a failed run.
  • Happy to provide run ids privately if useful.
Lenguaje dominante
TypeScript
Estrellas
2.5k
Forks
401
Merge medio
2 d 36 min
PR fusionados (30 d)
150

Preparar el entorno

  • Sin Dockerfile ni archivo de Docker Compose
  • Tiene una plantilla de pull request
  • Sin guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de vercel/workflow

Todos los issues de vercel/workflow

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.