ASGI app SIGTERM handler silently replaces the hosting server's handler
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 52/100
Línea de trabajo
Read python/restate/server.py:254-270 and inspect how asgi_app registers the SIGTERM handler for request and lifespan scopes. Reproduce the displacement with the provided minimal repro, then check behavior with both loop.add_signal_handler and signal.signal. Done means the host server's shutdown handler still runs after restate.app(...) is called, while active receive channels are notified.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
On the first call into restate.app(...) of any scope, asgi_app registers loop.add_signal_handler(signal.SIGTERM, _on_sigterm) (python/restate/server.py:254-267). Because loop.add_signal_handler replaces the existing handler without chaining, SIGTERM no longer reaches the hosting ASGI server's shutdown path after the first request.
Observed behavior
Measured on restate-sdk 1.0.5 (main 63d2fe5), Python 3.13:
- Hypercorn 0.18: serving a plain ASGI app exits in ~0.1 s on SIGTERM. Serving
restate.app(...), hitting/healthonce, and sending SIGTERM leaves the process running past a 15 s grace period until SIGKILL. - Uvicorn 0.54.0: serving a plain ASGI app exits in ~0.1 s on SIGTERM. Serving
restate.app(...)after one request hangs the same way until SIGKILL. - Lifespan scope: a single
lifespan.startupcall is enough to displace the host handler, because the signal handler is installed before thescope["type"] == "lifespan"check (python/restate/server.py:269-270).
_on_sigterm only calls notify_shutdown() on active ReceiveChannel instances. Nothing notifies the host server to stop accepting connections or exit.
Why both loop.add_signal_handler and signal.signal matter
ASGI servers register SIGTERM in two ways on Unix:
- Uvicorn uses
signal.signal(signal.SIGTERM, ...). - Hypercorn uses
loop.add_signal_handler(signal.SIGTERM, ...)(hypercorn/asyncio/run.py). Onceloop.add_signal_handleris active, Python'ssignal.getsignal(signal.SIGTERM)returnsasyncio.unix_events._sighandler_noop, and the actual host callback lives inloop._signal_handlers[signal.SIGTERM](anasyncio.Handlestoring the callback and its arguments).
PR #226 adds re-dispatch using signal.getsignal(signal.SIGTERM) before loop.add_signal_handler. Tested on top of 1.0.5, #226 fixes Uvicorn, but Hypercorn still hangs for 15 s because signal.getsignal(signal.SIGTERM) only captures _sighandler_noop rather than the previous loop handler.
Suggested fix
Before calling loop.add_signal_handler(signal.SIGTERM, _on_sigterm):
- Check
getattr(loop, "_signal_handlers", {}).get(signal.SIGTERM)first. If present, save thatasyncio.Handleand invokeprev_handle._run()from_on_sigtermafter notifying the channels. - Otherwise, fall back to
signal.getsignal(signal.SIGTERM)and invoke it asprev(signal.SIGTERM, None)when callable and notasyncio.unix_events._sighandler_noop. - Optionally move the registration after the
scope["type"] == "lifespan"check so a rejected lifespan call does not touch process signals.
Minimal repro (no external server required)
# uv run --no-project --python 3.13 --with restate-sdk==1.0.5 python repro.py
import asyncio
import signal
import restate
calls: list[str] = []
def build_app():
svc = restate.Service("probe")
@svc.handler()
async def noop(ctx: restate.Context) -> str:
return "ok"
return restate.app(services=[svc])
async def one_call(app):
async def receive():
return {"type": "lifespan.startup"}
async def send(msg):
pass
try:
await app({"type": "lifespan", "asgi": {"version": "3.0"}}, receive, send)
except Exception:
pass
def run_case(style: str) -> bool:
calls.clear()
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
if style == "loop.add_signal_handler":
loop.add_signal_handler(signal.SIGTERM, lambda: calls.append("host_ran"))
elif style == "signal.signal":
signal.signal(signal.SIGTERM, lambda *_: calls.append("host_ran"))
app = build_app()
loop.run_until_complete(one_call(app))
signal.raise_signal(signal.SIGTERM)
loop.run_until_complete(asyncio.sleep(0.05))
print(f"{style:<24} -> {calls or ['displaced (nothing ran)']}")
return "host_ran" in calls
for mode in ("loop.add_signal_handler", "signal.signal"):
run_case(mode)
Output on restate-sdk 1.0.5:
loop.add_signal_handler -> ['displaced (nothing ran)']
signal.signal -> ['displaced (nothing ran)']
- Lenguaje dominante
- Python
- Estrellas
- 83
- Forks
- 26
- Merge medio
- 7 d 4 h
- PR fusionados (30 d)
- 3
Preparar el entorno
Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de restatedev/sdk-python
-
pydantic ext: request_stream() never initializes the turnstile — KeyError('call_...') on first tool call of a streamed runPosiblemente ocupada @QuocThuanTruong la tomó hace 88 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
restatedev/sdk-python#223 · 1 comentario ·
-
Dificultad 2/5 1-2 días Aptitud para principiantes 68/100
restatedev/sdk-python#204 · 1 comentario ·
-
pydantic ext: tool results lose ToolReturn and deferred metadata, turnstile breaks on deferred and nested calls, iter() is not durablePosiblemente ocupada @sharosoo la tomó hace 5 días. Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
restatedev/sdk-python#237 ·
-
RestateModelWrapper should not wrap `SdkInternalBaseException` as generic Exception; classify non-retryable provider errors as terminalPosiblemente ocupada Un pull request vinculado a esta issue está abierto o ya se fusionó. Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
restatedev/sdk-python#225 ·
-
pydantic ext: auto_wrap_tools + async event_stream_handler crashes with 'Object of type coroutine is not JSON serializable' on first tool callPosiblemente ocupada @QuocThuanTruong la tomó hace 88 días. Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 68/100
restatedev/sdk-python#222 · 1 comentario ·
Todos los issues de restatedev/sdk-python
Issues similares
-
type-bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
push_gh-pages fails on master branch on forksPosiblemente ocupada Un pull request vinculado a esta issue está abierto o ya se fusionó. Abiertogood first issue infrastructure
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
kuznia-rdzeni/coreblocks#1135 ·
-
good first issue server
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
Los mantenedores suelen responder en 1 día
-
documentation priority: high
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
a-lex-s/mealie-telegram-bot#54 ·
Los mantenedores suelen responder en 1 día