没有vllm时使用fun-asr-nano每次都重新加载模型
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Start with _server_app.py lines 159-172 and trace how _load_vllm_engine and the fallback_models entry are used across requests. Review lines 273-280 as well, then make the fallback state prevent repeated loading when vLLM is unavailable; done means repeated fun-asr-nano requests reuse the loaded fallback model.
Written by the indexing model from the issue text.
Description
如题，在环境中没安装vllm时，若使用fun-asr-nano，则每次都会重新加载模型。
问题出现在_server_app.py的159~172行。
问题在于：
- vLLM 加载失败时 →
app.state.engine仍然是None - 降级到
AutoModel时 → 虽然模型被存入app.state.fallback_models["fun-asr-nano"]，但没有设置任何标志阻止下次重新加载 - 下次请求时 → 因为
app.state.engine is None，条件不成立，再次进入_load_vllm_engine() - 再次失败 → 再次加载
AutoModel（重复步骤 2-3）
另外，第273~280行这个逻辑也不是很合理，运行起来的python应该不能动态地就多了vllm库，所以server跑起来检测vllm不存在后，使用一个状态变量记一下就行了，至少server重启之前这个变量的值应该都不会变了。然后再所有模型都可以走_load_fallback。对于有vllm，则判断模型是不是fun-asr-nano，再决定走不走_load_vllm_engine()。这样我觉得更合理一些。
以现有代码，最小改变是：
第160行之后加两行：
if app.state.fallback_models["fun-asr-nano"]:
return app.state.fallback_models["fun-asr-nano"]
供参考。
感谢您的付出。
- Dominant language
- Python
- Stars
- 20.4k
- Forks
- 2k
- Avg merge
- 4h 55m
- Merged PRs (30d)
- 169
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from modelscope/FunASR
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
modelscope/FunASR#3704 · 1 comment ·
-
needs triage question
Difficulty 4/5 3-5 days Newbie friendliness 30/100
modelscope/FunASR#3718 · 3 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 48/100
modelscope/FunASR#3717 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
modelscope/FunASR#3715 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
modelscope/FunASR#3707 · 3 comments ·
All issues in modelscope/FunASR
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100