Misc. bug: Llama delivered via one-line install script provided the Vulkan version instead of the CUDA version

Open
  • Is something wrong?
#31,041 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
cpp, linux
Domain
backend, devops

Research direction

Start by locating the one-line install script and its logic for choosing the Linux build; the issue does not name a script path or test. Compare the detection result with the available CUDA and Vulkan builds, then reproduce the install on a system with the reported NVIDIA GPUs. Done means the script installs the CUDA version when CUDA is available and the installed llama server --list-devices reports CUDA devices.

Written by the indexing model from the issue text.

Description

bug-unconfirmed
Name and Version
$ ~/.local/bin/llama --version
version: 0.7.0-dev (build 11706, commit f74e2fa6a)
built with Clang 22.1.8 for Linux x86_64
Operating systems

Linux

Which llama.cpp modules do you know to be affected?

Other (Please specify in the next section)

Command line
`~/.local/bin/llama serve -hf ggml-org/Qwen3.8-27B-GGUF`
Problem description & steps to reproduce

After running ~/.local/bin/llama serve -hf ggml-org/Qwen3.8-27B-GGUF I noticed very slow tokens/sec out performance. When I ran ~/.local/bin/llama server --list-device it reported back Vulkan instead of CUDA.

$ ~/.local/bin/llama server --list-devices

Available devices:
  Vulkan0: NVIDIA GeForce RTX 2080 Ti (11510 MiB, 10976 MiB free)
  Vulkan1: NVIDIA GeForce RTX 2080 Ti (11510 MiB, 11242 MiB free)
  Vulkan2: NVIDIA GeForce RTX 2080 Ti (11510 MiB, 11242 MiB free)
  Vulkan3: NVIDIA GeForce RTX 2080 Ti (11510 MiB, 11242 MiB free)
  Vulkan4: NVIDIA GeForce RTX 2080 Ti (11510 MiB, 11243 MiB free)
  Vulkan5: NVIDIA GeForce RTX 2080 Ti (11510 MiB, 11242 MiB free)

When I tried the same from a manually downloaded CUDA version I got the following:

$ ~/Downloads/llama.cpp/llama-b11160/llama-server --list-devices
0.00.000.699 I srv  llama_server: initializing ...
Available devices:
  CUDA0: NVIDIA GeForce RTX 2080 Ti (11011 MiB, 10593 MiB free)
  CUDA1: NVIDIA GeForce RTX 2080 Ti (11011 MiB, 10851 MiB free)
  CUDA2: NVIDIA GeForce RTX 2080 Ti (11011 MiB, 10851 MiB free)
  CUDA3: NVIDIA GeForce RTX 2080 Ti (11011 MiB, 10851 MiB free)
  CUDA4: NVIDIA GeForce RTX 2080 Ti (11011 MiB, 10851 MiB free)
  CUDA5: NVIDIA GeForce RTX 2080 Ti (11011 MiB, 10851 MiB free)

The one click install script didn't detect the right CUDA version and instead deployed the Vulkan version.

Dominant language
C++
Stars
133k
Forks
24.9k
Avg merge
1d 22h
Merged PRs (30d)
473

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ggml-org/llama.cpp

All issues in ggml-org/llama.cpp

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.