Add support for experimental wheel variants (i.e., wheelnext)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- build-system, tooling
Research direction
The issue is about modifying the wheel building and publishing process. Start by examining the project's build scripts, likely in setup.py or pyproject.toml, and CI/CD workflows. Research the WheelNext specification and how projects like PyTorch implement variant metadata. The goal is to produce backend-specific wheels (CUDA, ROCm, Metal) with correct metadata, ensuring CPU wheels remain as a fallback. Testing will involve building wheels locally and verifying the metadata.
Written by the indexing model from the issue text.
Description
Is your feature request related to a problem? Please describe.
Today, installing llama-cpp-python on machines with different GPU backends (CUDA, ROCm, Metal, etc.) requires separate package names, custom extra indexes, or installer-level logic to select the correct wheel. This creates friction for downstream tooling (CLIs, orchestrators, and packaging systems) that want to provide a “just works” experience, especially when users don’t know which backend they need. Even a simple developer-driven install might require picking precisely the correct wheel.
Describe the solution you'd like
Add support for WheelNext-compatible experimental wheel variants when building and publishing wheels.
This would allow llama-cpp-python to produce a single package version that provides multiple backend-aware binary wheels, each annotated with variant metadata (e.g., GPU type, CUDA version, ROCm version).
Installers that understand the WheelNext spec (now used experimentally by PyTorch, uv, and others) can automatically select the correct backend wheel based on the system’s hardware/software configuration without a need for custom index URLs, separate packages, or manual backend flags.
Key pieces:
- Generate wheels with variant metadata following the experimental WheelNext (wheel variants) conventions.
- Publish per-backend wheels using the standardized naming + metadata fields.
- Ensure that CPU-only wheels remain available as fallback.
This would significantly simplify installation for all users and remove backend-selection logic from downstream tools. Wheel variants are fully backward-compatible so existing workflows won't be disrupted.
Describe alternatives you've considered
- Separate package names per backend (e.g., llama-cpp-python-cuda): fragments packaging and forces manual selection.
- Extras for backend variants (pip install llama-cpp-python[cuda]): still requires external detection and doesn’t integrate with hardware-aware installer selection.
- Custom index URLs for backend wheels: brittle and requires orchestration logic outside Python packaging.
- CLI-backed installation routing (what many downstream projects do currently): it’s reinventing the wheel and provides an inconsistent experience for end users.
All of these solutions put the burden on downstream tooling rather than on standardized wheel metadata.
Additional context
- https://wheelnext.dev
- https://pytorch.org/blog/pytorch-wheel-variants/
- https://astral.sh/blog/wheel-variants
- https://labs.quansight.org/blog/python-wheels-from-tags-to-variants
- https://developer.nvidia.com/blog/streamline-cuda-accelerated-python-install-and-packaging-workflows-with-wheel-variants
- https://lwn.net/Articles/1028299/
- Dominant language
- Python
- Stars
- 10.7k
- Forks
- 1.5k
- Avg merge
- 1h 40m
- Merged PRs (30d)
- 5
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from abetlen/llama-cpp-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
abetlen/llama-cpp-python#2424 ·
Maintainers usually reply within 1 day
-
Seven llama_sampler_init_* bindings admit keyword arguments that the ctypes function object silently dropsPossibly taken @Belal0066 claimed this 59 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
abetlen/llama-cpp-python#2371 ·
Maintainers usually reply within 1 day
-
uv add llama-cpp-python wheels fails for versions above 0.3.30Possibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2352 · 1 comment · 3 reactions ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
abetlen/llama-cpp-python#2314 ·
Maintainers usually reply within 1 day
-
Llama.embed() calls LlamaBatch.add_sequence with old 3-arg signature; missing logits_arrayMay be free again @lxcxjxhx claimed this 136 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2211 · 2 comments ·
Maintainers usually reply within 1 day
All issues in abetlen/llama-cpp-python
Similar issues
-
Bug IronPython Python 3
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pyrevitlabs/pyRevit#4184 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 74/100
jqueryscript/awesome-claude-code#1009 ·
-
tests
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
[Plugin] 群聊记忆 · 人格养成Open
Difficulty 1/5 1-3 hours Newbie friendliness 74/100