Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

SIGSEGV in continuous batching when a streaming client disconnects mid-generation (block_manager.hpp:633 assertion, GPU, 2026.2.1)

Abierto
  • ¿Algo no está bien?
#4,428 19 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
cpp
Área
ai, backend

Línea de trabajo

Comienza con openvino.genai/src/cpp/src/continuous_batching/cache/block_manager.hpp alrededor de la línea 633 y llm_executor.hpp alrededor de la línea 132, siguiendo la cancelación después de que un cliente de streaming se desconecte. Reproduce el problema con la solicitud de streaming proporcionada y un cierre forzado; se considera completado cuando el proceso permanece activo y las demás generaciones en curso continúan cuando un cliente se desconecta a mitad de la generación.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

A client that hard-closes its HTTP connection during a streaming
/v3/chat/completions generation reliably crashes the whole OVMS process within
seconds on Intel Arc GPUs. All other in-flight generations in the batch die with
it. Reproduced on demand; also observed 43× in 24 h under an agent fleet whose
callers time out and disconnect.

Environment

  • Image: docker.io/openvino/model_server:2026.2.1-gpu (stock, unmodified)
  • GPU: Intel Arc Pro B60 (xe driver) and Arc B580 — both affected, VM passthrough
  • Model: Qwen3.6-27B INT4 OpenVINO IR (also seen with a 9B on the second card)
  • Serving args:
    --task=text_generation --target_device=GPU --plugin_config='{"ENABLE_CPU_PINNING":false}' --tool_parser qwen3coder --enable_tool_guided_generation true --reasoning_parser qwen3 --kv_cache_precision u8 --cache_size 4 --enable_prefix_caching true --cache_interval_multiplier 64 --max_num_seqs 16 --max_num_batched_tokens 4096

Reproducer (deterministic)

  1. POST a streaming chat completion ("stream": true, long generation).
  2. Read a few SSE chunks.
  3. Hard-close the client socket mid-stream (socket.close(), no graceful shutdown).
  4. Within ~90 s the server logs the assertion below and the process exits.

Observed on the very first attempt; kernel gained exactly one segfault entry.

Logs

[llm_executor][error][llm_executor.hpp:132] Error occurred in LLM executor:
Check 'm_block_table.count(seq_id) > 0' failed at
../../../../../repos/openvino.genai/src/cpp/src/continuous_batching/cache/block_manager.hpp:633

Kernel (55+ occurrences over days, all error 4 read faults):

ovms[…]: segfault at … ip …6f3 … error 4 in ovms
libstdc++.so.6.0.33[…] from mediapipe/NNNNN threads

Symbolized against the shipped binary (BuildID a1b35868566ed20139449c9674ff41b1104ec73c):
the faulting IP lands in spdlog::logger::sink_it_+0x43 — i.e. the process dies
while logging, consistent with the executor thread's error path racing the
request-drop path that has already freed the sequence's block table entry.

Impact

  • --max_num_seqs 16 means one disconnected client kills up to 15 innocent
    concurrent generations (All requests: 3; Scheduled requests: 3 logged at death).
  • On a GPU deployment the recompile/reload takes 1–6 minutes per crash.
  • Any latency spike becomes self-amplifying: slow generations → client timeouts →
    disconnects → crash → colder cache → slower generations.

Notes

  • The xe "CAT error → engine reset" events sometimes seen after the crash FOLLOW
    the segfault by 300–460 ms and are absent for many crashes — downstream cleanup,
    not the cause.
  • Not OOM (memory.events oom_kill 0), not concurrency-proportional (a second
    card at 2.3× the concurrent load crashes 51× less per stream-second — the
    discriminating variable is the mid-stream disconnect rate).
  • 2026.1 release notes mention cancellation of requests from disconnected clients
    (NPU); this looks like the GPU continuous-batching cancellation path freeing
    seq_id from m_block_table while the executor still steps the sequence.
Lenguaje dominante
C++
Estrellas
932
Forks
278
Merge medio
2 d 23 h
PR fusionados (30 d)
68

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de openvinotoolkit/model_server

Todos los issues de openvinotoolkit/model_server

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.