how does `default_response_timeout` work? · pytorch/serve#2452

(6 评论) (0 反应) (0 负责人)Java (3,844 star) (790 fork)batch import

documentationgood first issuetriaged

描述

📚 The doc issue

I set the value of default_response_timeout to 4 i.e. 4 seconds. At the start of the model load, this happens after 4 (ish) seconds:

org.pytorch.serve.wlm.WorkerInitializationException: Backend worker did not respond in given time

My guess is because the model takes a while to load (more than 4 seconds), the worker gets killed. Is there a way to set a larger initial delay i.e. differentiate these two scenarios:

account for the initial model load with a number different from default_response_timeout
if model doesn't response in default_response_timeout after the initial load, then kill the worker

Suggest a potential alternative/fix

No response

贡献者指南

技术栈: pythonpytorchjava
领域: documentationbackend
议题类型: documentation
难度: 2
预计时间: 1-3 hours
活动状态: stale
清晰度: clear
前置要求: PyTorch Serveconfiguration
新手友好度: 80
研究方向: Examine the issue comments for any maintainer suggestions or workarounds. Look at the PyTorch Serve source code, particularly the worker initialization files (e.g., `wlm.py` for Python backend or relevant Java classes) to understand how `default response timeout` is currently implemented. Propose adding a separate configuration parameter like `initial load timeout` to differentiate between model loading and subsequent timeouts. Update the documentation to clarify this behavior.