Docs: consolidate build-from-source and GPU backend guide
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 75/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- cmake, python
- Domain
- build-system, documentation
Research direction
The issue points to consolidating build documentation, likely in docs/build.md or a new docs/gpu.md. Start by reviewing the existing README, CONTRIBUTING.md, and the linked GitHub issues (#2013, #695, #721, #756) to understand the current scattered information. Then, create a clear table of CMAKE_ARGS for each GPU backend and add troubleshooting steps for common errors. Verify the changes by building the project with a GPU backend to ensure the instructions work.
Written by the indexing model from the issue text.
Description
Problem
The build-from-source documentation is spread across README, CONTRIBUTING.md, and many GitHub issues (e.g. #2013, #695, #721, #756). Users with newer GPUs (RTX 50-series, sm_120) or uncommon CUDA versions repeatedly hit the same build failures because the docs lack:
- A CMAKE_ARGS mapping table showing which flag to set for each GPU backend
- CUDA version compatibility notes (e.g., CUDA 12.9+ requires specific cmake workarounds)
- The minimum/maximum supported CUDA toolkit versions
- How to verify the build actually used the GPU (simple Python snippet to check backend)
Suggested improvements
In docs/build.md (or a new docs/gpu.md):
| Backend | CMAKE_ARGS | GPU Requirements |
|---------|-----------------------------------------------|-------------------------------|
| CUDA | -DGGML_CUDA=ON | NVIDIA GPU, CUDA >= 11.3 |
| Metal | -DGGML_METAL=ON (default on macOS) | Apple Silicon or AMD GPU |
| Vulkan | -DGGML_VULKAN=ON | Vulkan 1.2+ capable GPU |
| SYCL | -DGGML_SYCL=ON -DGGML_SYCL_TARGET=INTEL | Intel Arc / Iris Xe |
| HIP | -DGGML_HIPBLAS=ON | AMD ROCm-compatible GPU |
And a troubleshooting section for the most common build errors (CMake not finding CUDA, nvcc version mismatch, sm_120 not recognized).
Value
Consolidating this reduces the "can't install with GPU" issues that currently fill the tracker.
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.5k
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from abetlen/llama-cpp-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
abetlen/llama-cpp-python#2371 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2352 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2211 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2210 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2145 ·
All issues in abetlen/llama-cpp-python
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
stephrobert/dsoxlab#238 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
sublimehq/package_control#1780 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
nwg-piotr/nwg-displays#145 ·