docker-compose.yaml GPU example: capabilities [gpu, utility] omits compute, so libcuda.so.1 is not injected

已关闭 适合新手
  • 哪里不对？
#11,973 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
84/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
docker, docker-compose
领域
devops, documentation

调研方向

从 docker-compose.yaml 中已注释的 NVIDIA 示例开始，并在文档页面中搜索相同的 capabilities 片段。如果可能，复现 legacy driver 变体，然后更新示例，使 CUDA 后端接收 libcuda.so.1；确认 compose 配置包含 compute，以及 gpu 和 utility。

由索引模型根据 Issue 内容生成。

描述

LocalAI version:
quay.io/go-skynet/local-ai:master-gpu-nvidia-cuda-13 (sha256:02e823f5f25ddef3f4edb94a8960e506daa4f71595d7c75b041d52b6eaaf425b, LocalAI bcf196d), docker-compose.yaml at the same commit.

Environment, CPU architecture, OS, and Version:
Docker Desktop 29.7.2 (Compose 5.5.1) on Windows 11, WSL2 kernel 6.18.33.2-microsoft-standard-WSL2, x86_64. NVIDIA RTX 5070 Ti, driver 616.92.

Describe the bug
The commented NVIDIA example in docker-compose.yaml recommends:

    # environment:
    #   NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
    # init: true
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: 1
    #           capabilities: [gpu, utility]

With capabilities: [gpu, utility], the container gets only the utility driver libraries. docker inspect shows "Capabilities":[["gpu","utility"]], and /usr/lib/x86_64-linux-gnu contains libnvidia-ml.so.1 and libdxcore.so but no libcuda.so.1. nvidia-smi works inside the container, so the setup looks fine, but every CUDA backend fails to start with ImportError: libcuda.so.1: cannot open shared object file. In my setup, also setting NVIDIA_DRIVER_CAPABILITIES=compute,utility in environment: (as the example suggests) did not change this.

Changing it to capabilities: [gpu, compute, utility] fixes it. libcuda.so.1 then shows up in /usr/lib/x86_64-linux-gnu and in ldconfig -p, and CUDA backends load.

To Reproduce

  1. Enable the legacy driver: nvidia example from docker-compose.yaml as written, with a CUDA image such as master-gpu-nvidia-cuda-13.
  2. docker compose up -d
  3. docker exec <container> sh -c 'ldconfig -p | grep libcuda.so.1' returns nothing. Loading a model on a Python CUDA backend fails with the libcuda.so.1 error above.

Expected behavior
Following the compose example gives a container where CUDA backends work.

Logs

ERROR Failed to load model ... error=failed to load model with internal loader: grpc service not ready: backend process exited with code 1: ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

Additional context
Suggested fix: use capabilities: [gpu, compute, utility] in the examples, and mention that compute is what brings in libcuda. I only tested the legacy driver: nvidia variant, on Docker Desktop/WSL2. I did not test the CDI (nvidia.com/gpu) variant, which may treat capabilities differently. It's worth checking the docs pages that show the same snippet as well.

主要语言
Go
星标
50k
派生
4.6k
平均合并
18 小时 40 分钟
30 天内合并 PR
325

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

mudler/LocalAI 的其他 Issue

查看 mudler/LocalAI 的全部 Issue

相似的 Issue

更多 Go Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。