vllm-omni backend (cuda13) fails to load: editable install points at build path, and unpinned vllm-omni main no longer matches backend.py
维护者通常 1 天内回复
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 52/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 技术栈
- docker, python
- 领域
- backend, build-system
调研方向
读取 backend/python/vllm-omni/install.sh、backend/python/common/libbackend.sh 和 backend/python/vllm-omni/backend.py;将克隆的 vllm-omni 修订版本与导入项和 Omni 参数进行比较。重现 issue 中的 gallery model 请求,然后验证 backend 能够加载并返回音频,且没有 editable-path 或 API-version 错误。
由索引模型根据 Issue 内容生成。
描述
LocalAI version:
quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-13 (sha256:02e823f5f25ddef3f4edb94a8960e506daa4f71595d7c75b041d52b6eaaf425b, LocalAI bcf196d)
Backend cuda13-vllm-omni from quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-13-vllm-omni (sha256:d97f6feee8effc5b71201e5d77eb934f746c4f2197fbf10736860d1df558eee3), installed 2026-09-11. Inside it: vllm 0.27.1, vllm_omni 0.27.0rc2.dev102+gc3f80502e.
Environment, CPU architecture, OS, and Version:
Docker Desktop 29.7.2 (Compose 5.5.1) on Windows 11, WSL2 kernel 6.18.33.2-microsoft-standard-WSL2, x86_64. AMD Ryzen 7 7800X3D, NVIDIA RTX 5070 Ti 16 GB, driver 616.92.
Describe the bug
vllm-omni-qwen3-tts-custom-voice from the gallery can't be loaded with the prebuilt cuda13 vllm-omni backend. There are three separate problems, and each one only shows up after the previous one is worked around:
-
ModuleNotFoundError: No module named 'vllm_omni'. The editable install points at the build path.
backend/python/vllm-omni/install.shinstalls vllm-omni withuv pip install -e .. The generated findervenv/lib/python3.12/site-packages/__editable___vllm_omni_*_finder.pyhardcodes the build-time location:MAPPING = {'vllm_omni': '/vllm-omni/vllm-omni/vllm_omni'}, plus severalNAMESPACESentries with the same prefix.direct_url.jsonisfile:///vllm-omni/vllm-omni. At runtime the backend lives in/backends/cuda13-vllm-omni/, so the source is at/backends/cuda13-vllm-omni/vllm-omni/vllm_omniand the import resolves nothing._makeVenvPortableinbackend/python/common/libbackend.shfixes the python symlinks,pyvenv.cfgand shebangs, but not editable finders. Rewriting the paths in the finder makes the import work.
This is probably the root cause of theNo module named 'vllm_omni'part of #9162, reported there for cuda12.backend/python/fish-speech/install.shalso usespip install -e, so it likely has the same problem. -
ModuleNotFoundError: No module named 'vllm_omni.utils.platform_utils'.
backend.pyline 43 doesfrom vllm_omni.utils.platform_utils import detect_device_type, is_npu. The vllm-omni source bundled in the image doesn't have that module anymore:vllm_omni/utils/containsaudio.py,custom_voice_io.py,forced_aligner.py,mm_outputs.py, … and there is a newvllm_omni/platforms/package. Neither function is defined anywhere in the bundled source. -
ValueError: `stage_configs_path` is no longer supported; use `deploy_config` instead.
After shimming the two helpers above, loading fails here.backend.py(around line 225) always passesstage_configs_path(defaultNone) toOmni(...)forllm/ttsmodel types.
Problems 2 and 3 have the same underlying cause: install.sh clones https://github.com/vllm-project/vllm-omni.git without pinning a tag or commit. Each backend build therefore picks up whatever main is at that moment, and main has moved past the API that backend.py targets.
To Reproduce
- Run
quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-13with an NVIDIA GPU. - Install
vllm-omni-qwen3-tts-custom-voicefrom the model gallery. This pulls thecuda13-vllm-omnibackend. curl http://localhost:8080/v1/audio/speech -H "Content-Type: application/json" -d '{"model":"vllm-omni-qwen3-tts-custom-voice","input":"Hello"}'
Expected behavior
The model loads and returns audio.
Logs
ERROR Failed to load model modelID="vllm-omni-qwen3-tts-custom-voice" error=failed to load model with internal loader: grpc service not ready: backend process exited with code 1: ModuleNotFoundError: No module named 'vllm_omni' backend="vllm-omni"
after fixing the finder paths:
WARN Backend process exited unexpectedly id="vllm-omni-qwen3-tts-custom-voice" process="run.sh" exitCode="1" stderr="ModuleNotFoundError: No module named 'vllm_omni.utils.platform_utils'"
after shimming platform_utils:
ERROR Failed to load model modelID="vllm-omni-qwen3-tts-custom-voice" error=failed to load model with internal loader: could not load model (no success): Unexpected err=ValueError('`stage_configs_path` is no longer supported; use `deploy_config` instead.'), type(err)=<class 'ValueError'> backend="vllm-omni"
Additional context
Possible fixes:
- Pin vllm-omni in
install.shto a tag or commit thatbackend.pyis tested against, and bump it deliberately. - Install it non-editable (
uv pip install .), or make_makeVenvPortablerewrite editable finder paths to the runtime backend dir. Check otherbackend/python/*/install.shfor-einstalls (at least fish-speech). - Update
backend.pyfor the current vllm-omni API (vllm_omni.platforms,deploy_config) if moving forward instead of pinning back.
Related: #9162, #9293
- 主要语言
- Go
- 星标
- 50k
- 派生
- 4.6k
- 平均合并
- 18 小时 32 分钟
- 30 天内合并 PR
- 328
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 提供 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
mudler/LocalAI 的其他 Issue
-
[Bug] Qwen3.8-27b Uncensored Fails to Download due to 404可能重新可做 关联的 PR 已关闭且未合并。 未关闭bug unconfirmed
难度 1/5 1 小时以内 新手友好度 70/100
维护者通常 1 天内回复
-
Realtime: ASR EOU language tags leak into the LLM conversation (unintended reply-language switching)可能已有人在做 @nexxtmobile 于 17 天前认领。 未关闭area/api bug unconfirmed
难度 2/5 1-3 小时 新手友好度 70/100
维护者通常 1 天内回复
-
Open Responses API doesn't accept "output_text" as an input Item可能重新可做 关联的 PR 已关闭且未合并。 未关闭bug unconfirmed
难度 2/5 1-3 小时 新手友好度 78/100
mudler/LocalAI#12337 · 1 条评论 ·
维护者通常 1 天内回复
-
Gallery model audio-cpp-sortformer-diarization fails to install: SHA mismatch, reproducible across retries可能重新可做 关联的 PR 已关闭且未合并。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
mudler/LocalAI#11995 · 1 条评论 ·
维护者通常 1 天内回复
-
Agent pool Postgres DSN is logged in cleartext, including the password, while the auth DSN is redacted可能已有人在做 @localai-org-maint-bot 于 52 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
mudler/LocalAI#11991 · 1 条评论 ·
维护者通常 1 天内回复
相似的 Issue
-
Routing rule or circuit breaker trip expression with an unknown CEL function hangs daprd可能已有人在做 @RaphaelFakhri 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复
-
[falco-talon] Setting `podAnnotations` makes the chart fail to render可能已有人在做 @RaphaelFakhri 今天认领。 未关闭needs-kind
难度 1/5 1 小时以内 新手友好度 88/100
falcosecurity/charts#1084 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 76/100
DataDog/terraform-provider-datadog#4479 ·
维护者通常 2 天内回复
-
component/tests needs-triage
难度 2/5 1-3 小时 新手友好度 62/100
NVIDIA/nodewright#967 ·
维护者通常 1 天内回复