Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Improve installation DX: prebuilt wheels for 3.13/3.14/3.14t + declarative backend selection

Aberta
  • Algo errado?
#2,136 0 comentários 1 reação 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
3/5
Tempo estimado
1-2 dias
Facilidade para iniciantes
45/100
Tipo de issue
Funcionalidade
Clareza
Claramente especificada
Status de atividade
Estagnada
Stack de tecnologia
cmake, python
Domínio
build-system, ci-cd, developer-experience, tooling

Direção de pesquisa

A issue aponta para três arquivos específicos de CI workflow: .github/workflows/build-wheels-metal.yaml, build-wheels-cuda.yaml e build-and-release.yaml. Comece examinando as versões atuais do cibuildwheel e as matrizes CIBW_BUILD nesses arquivos. A alteração envolve atualizar a versão e adicionar suporte a builds Python 3.13, 3.14 e free-threaded. Teste as alterações localmente ou em um fork para garantir que os builds das wheels sejam concluídos com sucesso. “Done” significa que os workflows produzem wheels para as novas versões do Python e que a documentação é atualizada para mencionar a opção config-settings -C cmake.args.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Problem

Installing llama-cpp-python with a GPU backend requires setting CMAKE_ARGS as an environment variable at build time:

CMAKE_ARGS="-DGGML_METAL=on" pip install llama-cpp-python

This creates pain across the ecosystem:

  1. Not declarable in pyproject.toml — Every downstream project needs custom Makefiles or install scripts with GPU auto-detection logic (macOS → Metal, nvidia-smi → CUDA, rocminfo → ROCm, fallback → OpenBLAS). This is duplicated across hundreds of projects.

  2. Cache invalidation is broken — pip and uv cache wheels by package version, not by CMAKE_ARGS. A cached OpenBLAS wheel silently gets reused when Metal or CUDA is requested. Workaround: --no-cache, which defeats caching entirely.

  3. GPU prebuilt wheels stop at Python 3.12 — The Metal wheel CI (build-wheels-metal.yaml) is hardcoded to CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-*". The CUDA wheel CI (build-wheels-cuda.yaml) has its matrix pinned to Python 3.9-3.12. CPU-only wheels include 3.13 (via default cibuildwheel config in build-and-release.yaml), but the arm64 job there also pins to cp38-cp312. No workflow produces 3.14 or free-threaded (3.13t/3.14t) wheels. Python 3.13 has been stable since Oct 2024, 3.14 since Oct 2025. Free-threaded builds are increasingly important — vLLM, llguidance, and the broader no-GIL ecosystem depend on them.

Current state of published wheel indexes:

Index cp313 cp314 Free-threaded
CPU (/whl/cpu/) ✅ ❌ ❌
Metal (/whl/metal/) ❌ ❌ ❌
CUDA (/whl/cu1xx/) ❌ ❌ ❌

Proposed changes

1. Expand prebuilt wheel matrix (highest impact, smallest change)

Update CIBW_BUILD in Metal/CUDA workflows and add free-threaded support. This is the single highest-impact change — it eliminates source builds for most users.

build-wheels-metal.yaml:

Upgrade cibuildwheel from v2.22.0 to v3.x (3.0 added cp314/cp314t support). In cibuildwheel 3.0, cp314t is built by default (free-threading is no longer experimental in 3.14), and cp313t requires CIBW_ENABLE: cpython-freethreading.

-        uses: pypa/[email protected]
+        uses: pypa/[email protected]
         env:
-          CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-*"
+          CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-* cp313-* cp314-*"
+          CIBW_ENABLE: cpython-freethreading

build-and-release.yaml — same cibuildwheel upgrade, and update the build_wheels_arm64 job:

-          CIBW_BUILD: "cp38-* cp39-* cp310-* cp311-* cp312-*"
+          CIBW_BUILD: "cp38-* cp39-* cp310-* cp311-* cp312-* cp313-* cp314-*"
+          CIBW_ENABLE: cpython-freethreading

build-wheels-cuda.yaml — uses a different build system (python -m build --wheel with a PowerShell matrix). The pyver matrix would need "3.13", "3.14" added.

With prebuilt wheels, any downstream project can use uv's declarative index support:

# pyproject.toml — zero Makefile, zero CMAKE_ARGS
[project]
dependencies = ["llama-cpp-python~=0.3"]

[tool.uv.sources]
llama-cpp-python = [
  { index = "llama-metal", marker = "sys_platform == 'darwin'" },
  { index = "llama-cpu",   marker = "sys_platform == 'linux'" },
]

[[tool.uv.index]]
name = "llama-metal"
url = "https://abetlen.github.io/llama-cpp-python/whl/metal"
explicit = true

[[tool.uv.index]]
name = "llama-cpu"
url = "https://abetlen.github.io/llama-cpp-python/whl/cpu"
explicit = true
2. Document --config-settings as the source-build path

Since the build backend is scikit-build-core, cmake args can be passed via the standard PEP 517 config-settings interface:

pip install llama-cpp-python -C cmake.args="-DGGML_METAL=on"
# or with uv:
uv pip install llama-cpp-python -C cmake.args="-DGGML_METAL=on"

This is cleaner than the CMAKE_ARGS env var — it's the standard PEP 517 mechanism, more explicit, and discoverable. It's already supported via scikit-build-core but not documented in the README or install docs.

3. (Future) Adopt PEP 817 Wheel Variants

PEP 817 (draft, Dec 2025) introduces a standard mechanism for GPU/accelerator wheel variants. PyTorch 2.9 already ships experimental variant-enabled wheels. Once PEP 817 is accepted and tool support lands, llama-cpp-python could publish variant wheels that are auto-selected by the installer:

# Future: just works, installer picks Metal/CUDA/CPU automatically
pip install llama-cpp-python

This is mentioned for context only — the actionable items are (1) and (2) above.

Ecosystem context

  • Quansight offered funded engineering help for free-threaded support in #2103 (via vLLM ecosystem work) — awaiting maintainer signal
  • ~470K monthly PyPI downloads (pypistats) — every project using this beyond toy scripts hits this install wall
  • How others solved it: PyTorch uses per-backend index URLs + PEP 817 variants; ONNX Runtime publishes separate PyPI packages per backend (onnxruntime-gpu, onnxruntime-silicon)

Related

Wheel matrix gaps (same root cause):

  • #2103 — Pre-built wheels for Python 3.14 and 3.14 free-threaded
  • #2130 — Pre-built CPU-only wheel for Windows (cp313)
  • #2068 — Where can I download wheel for CUDA 12.8?
  • #2091 — CUDA 12.8 wheel request

Wheel variants / long-term packaging:

  • #2092 — Add support for experimental wheel variants (wheelnext)
  • #1506 — Multi-arch support for pre-built CPU wheel (by @abetlen)
  • Discussion #1875 — Automating pre-building of wheels for all platforms

Downstream impact of missing wheels:

  • #2118 — Installation deadlock on Hugging Face Spaces (musl/glibc mismatch)
  • #2113 — No working wheels for Debian/Ubuntu

Happy to submit a PR for (1) and (2).

Linguagem predominante
Python
Estrelas
10.6k
Forks
1.5k
Merge médio
3h 57min
PRs com merge (30d)
4

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de abetlen/llama-cpp-python

Todas as issues de abetlen/llama-cpp-python

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.