Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Async topic writer wedges under reconnect: 'cannot schedule new futures after shutdown' in _encode_data_inplace (compressing codec)

Abierto
  • ¿Algo no está bien?
#852 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
grpc, python
Área
api, backend

Línea de trabajo

Empieza en ydb/_topic_writer/topic_writer_asyncio.py, especialmente en write_with_ack, _encode_loop y _encode_data_inplace, y sigue cómo se crea y se cierra self._encode_executor alrededor de la ruta de reconexión de gRPC. Reproduce el escenario de pérdida de conexión descrito en el issue o la carga de trabajo de caos de PR #851. Se considera terminado cuando un write_with_ack con un codec de compresión se completa o informa de un error después de la reconexión, en lugar de quedarse bloqueado o dejar una excepción del executor sin recuperar.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

Under connection loss + reconnect (e.g. a node killed during chaos testing), the async topic writer using a compressing codec (GZIP) can get permanently wedged: write_with_ack never returns because the encode ThreadPoolExecutor was shut down while the writer is still live and re-encoding, so submitting the encode job raises RuntimeError('cannot schedule new futures after shutdown'). The error surfaces only as an unretrieved background future, and the awaiting write_with_ack hangs indefinitely (no timeout of its own).

Traceback (observed)
grpc/aio/_call.py ... raise asyncio.CancelledError()   # stream cancelled by the node kill
asyncio.exceptions.CancelledError

ERROR  Future exception was never retrieved
future: <Future finished exception=RuntimeError('cannot schedule new futures after shutdown')>
Traceback (most recent call last):
  File ".../ydb/_topic_writer/topic_writer_asyncio.py", line 128, in write_with_ack
    results = [f.result() for f in futures]
  File ".../ydb/_topic_writer/topic_writer_asyncio.py", line 578, in _encode_loop
    await self._encode_data_inplace(batch_codec, messages)
  File ".../ydb/_topic_writer/topic_writer_asyncio.py", line 599, in _encode_data_inplace
    encoded_data_futures = eventloop.run_in_executor(self._encode_executor, encoder_function, ...)
  File ".../concurrent/futures/thread.py", line 167, in submit
    raise RuntimeError('cannot schedule new futures after shutdown')
RuntimeError: cannot schedule new futures after shutdown
Conditions
  • ydb.aio topic writer (topic_client.writer(...)), compressing codec (TopicCodec.GZIP).
  • Connection drop + reconnect (chaos: a CancelledError propagates from the gRPC stream).
  • After that, _encode_data_inplace submits to self._encode_executor, which has already been shut down → RuntimeError.
Impact

write_with_ack never completes → the writer silently hangs. In an application that awaits write_with_ack without its own timeout, the whole writer path stalls. Same class of async-topic-writer reconnect-lifecycle issue as the earlier WriterAsyncIOStream.create() thread leak (PR #845).

Root cause (likely)

self._encode_executor is shut down while the writer is still alive and re-encoding after a reconnect (executor lifecycle not tied correctly to the writer's live/reconnect state).

Workaround

Use TopicCodec.RAW — _encode_data_inplace returns early for RAW and never touches the executor.

How it was found

YDB Python SLO chaos testing (PR #851): the async topic workload silently stalled at ~half the run; the baseline container's delivery metrics went N/A mid-run. The SLO harness was hardened (timeouts + writer/reader recreate + RAW) to not hang, but the underlying SDK writer bug remains.

Lenguaje dominante
Python
Estrellas
102
Forks
80
Merge medio
1 d 4 h
PR fusionados (30 d)
8

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de ydb-platform/ydb-python-sdk

Todos los issues de ydb-platform/ydb-python-sdk

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.