Add support for experimental wheel variants (i.e., wheelnext)
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 45/100
- Issue 类型
- 功能
- 描述清晰度
- 基本清楚
- 活跃度
- 停滞
- 技术栈
- python
- 领域
- build-system, tooling
调研方向
该 issue 涉及修改 wheel 的构建和发布流程。首先检查项目的构建脚本，可能位于 setup.py 或 pyproject.toml 中，以及 CI/CD 工作流。研究 WheelNext 规范，以及 PyTorch 等项目如何实现变体元数据。目标是生成具有正确元数据的特定后端 wheel（CUDA、ROCm、Metal），同时确保 CPU wheel 仍作为 fallback。测试将包括在本地构建 wheel 并验证元数据。
由索引模型根据 Issue 内容生成。
描述
Is your feature request related to a problem? Please describe.
Today, installing llama-cpp-python on machines with different GPU backends (CUDA, ROCm, Metal, etc.) requires separate package names, custom extra indexes, or installer-level logic to select the correct wheel. This creates friction for downstream tooling (CLIs, orchestrators, and packaging systems) that want to provide a “just works” experience, especially when users don’t know which backend they need. Even a simple developer-driven install might require picking precisely the correct wheel.
Describe the solution you'd like
Add support for WheelNext-compatible experimental wheel variants when building and publishing wheels.
This would allow llama-cpp-python to produce a single package version that provides multiple backend-aware binary wheels, each annotated with variant metadata (e.g., GPU type, CUDA version, ROCm version).
Installers that understand the WheelNext spec (now used experimentally by PyTorch, uv, and others) can automatically select the correct backend wheel based on the system’s hardware/software configuration without a need for custom index URLs, separate packages, or manual backend flags.
Key pieces:
- Generate wheels with variant metadata following the experimental WheelNext (wheel variants) conventions.
- Publish per-backend wheels using the standardized naming + metadata fields.
- Ensure that CPU-only wheels remain available as fallback.
This would significantly simplify installation for all users and remove backend-selection logic from downstream tools. Wheel variants are fully backward-compatible so existing workflows won't be disrupted.
Describe alternatives you've considered
- Separate package names per backend (e.g., llama-cpp-python-cuda): fragments packaging and forces manual selection.
- Extras for backend variants (pip install llama-cpp-python[cuda]): still requires external detection and doesn’t integrate with hardware-aware installer selection.
- Custom index URLs for backend wheels: brittle and requires orchestration logic outside Python packaging.
- CLI-backed installation routing (what many downstream projects do currently): it’s reinventing the wheel and provides an inconsistent experience for end users.
All of these solutions put the burden on downstream tooling rather than on standardized wheel metadata.
Additional context
- https://wheelnext.dev
- https://pytorch.org/blog/pytorch-wheel-variants/
- https://astral.sh/blog/wheel-variants
- https://labs.quansight.org/blog/python-wheels-from-tags-to-variants
- https://developer.nvidia.com/blog/streamline-cuda-accelerated-python-install-and-packaging-workflows-with-wheel-variants
- https://lwn.net/Articles/1028299/
- 主要语言
- Python
- 星标
- 10.6k
- 派生
- 1.5k
- 平均合并
- 3 小时 57 分钟
- 30 天内合并 PR
- 4
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
abetlen/llama-cpp-python 的其他 Issue
-
Seven llama_sampler_init_* bindings admit keyword arguments that the ctypes function object silently drops可能已有人在做 @Belal0066 于 19 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 88/100
abetlen/llama-cpp-python#2371 ·
维护者通常 1 天内回复
-
uv add llama-cpp-python wheels fails for versions above 0.3.30可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 65/100
abetlen/llama-cpp-python#2352 · 1 条评论 · 2 个 reaction ·
维护者通常 1 天内回复
-
Docs: consolidate build-from-source and GPU backend guide可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
abetlen/llama-cpp-python#2314 ·
维护者通常 1 天内回复
-
Llama.embed() calls LlamaBatch.add_sequence with old 3-arg signature; missing logits_array可能重新可做 @lxcxjxhx 于 96 天前认领，目前没有进行中的 PR。 未关闭
难度 2/5 1-3 小时 新手友好度 65/100
abetlen/llama-cpp-python#2211 · 2 条评论 ·
维护者通常 1 天内回复
-
Llama() silently accepts and discards `embedding` kwarg; .embed() then raises confusingly可能已有人在做 @Anai-Guo 于 37 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 65/100
abetlen/llama-cpp-python#2210 ·
维护者通常 1 天内回复
查看 abetlen/llama-cpp-python 的全部 Issue
相似的 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 76/100
lugassawan/swe-workbench#799 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 74/100
vllm-project/vllm#61131 ·
维护者通常 1 天内回复
-
Non-string form field names raise AttributeError when files are attached可能已有人在做 @nightcityblade 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
-
难度 2/5 1-3 小时 新手友好度 78/100
kubernetes-client/python#2735 ·
维护者通常 1 天内回复
-
desktop help wanted test
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复