ASGI app SIGTERM handler silently replaces the hosting server's handler
评估
调研方向
Read python/restate/server.py:254-270 and inspect how asgi_app registers the SIGTERM handler for request and lifespan scopes. Reproduce the displacement with the provided minimal repro, then check behavior with both loop.add_signal_handler and signal.signal. Done means the host server's shutdown handler still runs after restate.app(...) is called, while active receive channels are notified.
由索引模型根据 Issue 内容生成。
描述
On the first call into restate.app(...) of any scope, asgi_app registers loop.add_signal_handler(signal.SIGTERM, _on_sigterm) (python/restate/server.py:254-267). Because loop.add_signal_handler replaces the existing handler without chaining, SIGTERM no longer reaches the hosting ASGI server's shutdown path after the first request.
Observed behavior
Measured on restate-sdk 1.0.5 (main 63d2fe5), Python 3.13:
- Hypercorn 0.18: serving a plain ASGI app exits in ~0.1 s on SIGTERM. Serving
restate.app(...), hitting/healthonce, and sending SIGTERM leaves the process running past a 15 s grace period until SIGKILL. - Uvicorn 0.54.0: serving a plain ASGI app exits in ~0.1 s on SIGTERM. Serving
restate.app(...)after one request hangs the same way until SIGKILL. - Lifespan scope: a single
lifespan.startupcall is enough to displace the host handler, because the signal handler is installed before thescope["type"] == "lifespan"check (python/restate/server.py:269-270).
_on_sigterm only calls notify_shutdown() on active ReceiveChannel instances. Nothing notifies the host server to stop accepting connections or exit.
Why both loop.add_signal_handler and signal.signal matter
ASGI servers register SIGTERM in two ways on Unix:
- Uvicorn uses
signal.signal(signal.SIGTERM, ...). - Hypercorn uses
loop.add_signal_handler(signal.SIGTERM, ...)(hypercorn/asyncio/run.py). Onceloop.add_signal_handleris active, Python'ssignal.getsignal(signal.SIGTERM)returnsasyncio.unix_events._sighandler_noop, and the actual host callback lives inloop._signal_handlers[signal.SIGTERM](anasyncio.Handlestoring the callback and its arguments).
PR #226 adds re-dispatch using signal.getsignal(signal.SIGTERM) before loop.add_signal_handler. Tested on top of 1.0.5, #226 fixes Uvicorn, but Hypercorn still hangs for 15 s because signal.getsignal(signal.SIGTERM) only captures _sighandler_noop rather than the previous loop handler.
Suggested fix
Before calling loop.add_signal_handler(signal.SIGTERM, _on_sigterm):
- Check
getattr(loop, "_signal_handlers", {}).get(signal.SIGTERM)first. If present, save thatasyncio.Handleand invokeprev_handle._run()from_on_sigtermafter notifying the channels. - Otherwise, fall back to
signal.getsignal(signal.SIGTERM)and invoke it asprev(signal.SIGTERM, None)when callable and notasyncio.unix_events._sighandler_noop. - Optionally move the registration after the
scope["type"] == "lifespan"check so a rejected lifespan call does not touch process signals.
Minimal repro (no external server required)
# uv run --no-project --python 3.13 --with restate-sdk==1.0.5 python repro.py
import asyncio
import signal
import restate
calls: list[str] = []
def build_app():
svc = restate.Service("probe")
@svc.handler()
async def noop(ctx: restate.Context) -> str:
return "ok"
return restate.app(services=[svc])
async def one_call(app):
async def receive():
return {"type": "lifespan.startup"}
async def send(msg):
pass
try:
await app({"type": "lifespan", "asgi": {"version": "3.0"}}, receive, send)
except Exception:
pass
def run_case(style: str) -> bool:
calls.clear()
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
if style == "loop.add_signal_handler":
loop.add_signal_handler(signal.SIGTERM, lambda: calls.append("host_ran"))
elif style == "signal.signal":
signal.signal(signal.SIGTERM, lambda *_: calls.append("host_ran"))
app = build_app()
loop.run_until_complete(one_call(app))
signal.raise_signal(signal.SIGTERM)
loop.run_until_complete(asyncio.sleep(0.05))
print(f"{style:<24} -> {calls or ['displaced (nothing ran)']}")
return "host_ran" in calls
for mode in ("loop.add_signal_handler", "signal.signal"):
run_case(mode)
Output on restate-sdk 1.0.5:
loop.add_signal_handler -> ['displaced (nothing ran)']
signal.signal -> ['displaced (nothing ran)']
- 主要语言
- Python
- 星标
- 83
- 派生
- 26
- 平均合并
- 7 天 4 小时
- 30 天内合并 PR
- 3
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
restatedev/sdk-python 的其他 Issue
-
pydantic ext: request_stream() never initializes the turnstile — KeyError('call_...') on first tool call of a streamed run可能已有人在做 @QuocThuanTruong 于 90 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
restatedev/sdk-python#223 · 1 条评论 ·
-
难度 2/5 1-2 天 新手友好度 68/100
restatedev/sdk-python#204 · 1 条评论 ·
-
pydantic ext: tool results lose ToolReturn and deferred metadata, turnstile breaks on deferred and nested calls, iter() is not durable可能已有人在做 @sharosoo 于 7 天前认领。 未关闭
难度 4/5 3-5 天 新手友好度 48/100
restatedev/sdk-python#237 ·
-
RestateModelWrapper should not wrap `SdkInternalBaseException` as generic Exception; classify non-retryable provider errors as terminal可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 4/5 3-5 天 新手友好度 48/100
restatedev/sdk-python#225 ·
-
pydantic ext: auto_wrap_tools + async event_stream_handler crashes with 'Object of type coroutine is not JSON serializable' on first tool call可能已有人在做 @QuocThuanTruong 于 90 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 68/100
restatedev/sdk-python#222 · 1 条评论 ·
查看 restatedev/sdk-python 的全部 Issue
相似的 Issue
-
The changelog workflow excludes otelbot but not dependabot, so every Dependabot pull request fails the changelog check可能已有人在做 @ocelotl 今天认领。 未关闭
难度 1/5 1 小时以内 新手友好度 82/100
open-telemetry/opentelemetry-python#5825 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
cuga-project/cuga-agent#1020 ·
维护者通常 2 天内回复
-
难度 1/5 1 小时以内 新手友好度 70/100
microsoft/playwright#43640 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
stac-utils/stac-fastapi-elasticsearch-opensearch#965 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 1 天内回复