Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

RFC: Proactive Server-Side Cancellation via `Request-Timeout-Ms`

Abierto
  • ¿Algo no está bien?
#3,277 3 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

@nightcityblade ya está trabajando en esto.

Desde el 1/6/2026.

  • #3351 de @nightcityblade — cerrado sin fusionar

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
62/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
python
Área
api

Línea de trabajo

Comienza en _base_client.py, en _build_headers(), y sigue cómo se representan los valores de timeout del cliente y de cada solicitud. El cambio estará terminado cuando un timeout de reloj de pared representado por un float simple produzca Request-Timeout-Ms, los valores httpx.Timeout no lo hagan y se conserve un encabezado personalizado existente; añade o actualiza pruebas específicas para estos casos si se identifica la ubicación correspondiente entre las pruebas cercanas.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Confirm this is a feature request for the Python library and not the underlying OpenAI API.
  • This is a feature request for the Python library
Describe the feature or improvement you're requesting

Hey, I'm writing on behalf on Baseten,

When a client times out, the server has no idea, and chaining the cancellation in services, e.g. http1 has canvats. It keeps doing expensive work (inference, token generation) that nobody will ever receive. The only signal today is a TCP disconnect — reactive, not proactive. In theory/future, this could also be adopted to e.g. issue a timeout on server side if work is unrealistic to be completed within that time.

Proposal

Send a Request-Timeout-Ms header on every request so nginx, Go contexts, and load balancers can cancel in-flight work proactively when the deadline elapses — no client disconnect needed, or cancellation chain can work proactivley. Internal services can also convert this into Request-Deadline-Ms a ms since unix epoch time, which allows server side verification in distributed systems.

Why not just use x-stainless-read-timeout?

timeout.read is a per-chunk silence threshold, not a wall-clock budget. It resets on every received chunk, so it's the wrong value to drive server-side cancellation — a healthy long running stream would get killed incorrectly.

What's a valid value?
Only a plain float timeout (e.g. OpenAI(timeout=20.0)) is a true wall-clock budget for e2e time. httpx.Timeout objects have no equivalent field. We should not send the header for those — worse than no header, as we could cancel the work on server side for this..

Proposed Implementation

# _build_headers(), _base_client.py
if "request-timeout-ms" not in lower_custom_headers:
    timeout = self.timeout if isinstance(options.timeout, NotGiven) else options.timeout
    if not isinstance(timeout, Timeout) and timeout is not None:
        headers["request-timeout-ms"] = str(int(timeout * 1000))

Prior Art

  • gRPC: grpc-timeout propagates deadlines e2e across all services — the canonical example of this pattern. Middleware can decreatse that
  • Envoy: x-envoy-upstream-rq-timeout-ms — exact same semantics, widely adopted in service meshes. Unfortunately not very cross-vendor agnostic.
  • Google Maps/Cloud API: X-Server-Timeout used for deadline propagation - unfortunately in seconds, not milliseconds.
  • Stainless SDKs: Already send x-stainless-read-timeout for observability — this builds on that foundation with correct cancellation semantics.

It would be great to have a vendor agnostic name, that could be adopted from a range of LLM projects. The stainless OpenAI API is IMO the best proxy. I think having a header we can rely on would help us save a ton of compute - i believe. Please don't make the header contain openai or stainless.

Additional context
Lenguaje dominante
Python
Estrellas
31.8k
Forks
7.3k
Merge medio
1 d 3 h
PR fusionados (30 d)
131

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de openai/openai-python

Todos los issues de openai/openai-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.