Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Improve installation DX: prebuilt wheels for 3.13/3.14/3.14t + declarative backend selection

Aperta
  • Qualcosa non va?
#2,136 0 commenti 1 reazione 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
45/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
cmake, python
Ambito
build-system, ci-cd, developer-experience, tooling

Direzione di ricerca

La issue indica tre file specifici di CI workflow: .github/workflows/build-wheels-metal.yaml, build-wheels-cuda.yaml e build-and-release.yaml. Inizia esaminando le versioni correnti di cibuildwheel e le matrici CIBW_BUILD in questi file. La modifica consiste nell’aggiornare la versione e nell’aggiungere il supporto alle build Python 3.13, 3.14 e free-threaded. Prova le modifiche localmente o in un fork per assicurarti che le build delle wheel vadano a buon fine. «Done» significa che i workflow producono wheel per le nuove versioni di Python e che la documentazione viene aggiornata per menzionare l’opzione config-settings -C cmake.args.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

Installing llama-cpp-python with a GPU backend requires setting CMAKE_ARGS as an environment variable at build time:

CMAKE_ARGS="-DGGML_METAL=on" pip install llama-cpp-python

This creates pain across the ecosystem:

  1. Not declarable in pyproject.toml — Every downstream project needs custom Makefiles or install scripts with GPU auto-detection logic (macOS → Metal, nvidia-smi → CUDA, rocminfo → ROCm, fallback → OpenBLAS). This is duplicated across hundreds of projects.

  2. Cache invalidation is broken — pip and uv cache wheels by package version, not by CMAKE_ARGS. A cached OpenBLAS wheel silently gets reused when Metal or CUDA is requested. Workaround: --no-cache, which defeats caching entirely.

  3. GPU prebuilt wheels stop at Python 3.12 — The Metal wheel CI (build-wheels-metal.yaml) is hardcoded to CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-*". The CUDA wheel CI (build-wheels-cuda.yaml) has its matrix pinned to Python 3.9-3.12. CPU-only wheels include 3.13 (via default cibuildwheel config in build-and-release.yaml), but the arm64 job there also pins to cp38-cp312. No workflow produces 3.14 or free-threaded (3.13t/3.14t) wheels. Python 3.13 has been stable since Oct 2024, 3.14 since Oct 2025. Free-threaded builds are increasingly important — vLLM, llguidance, and the broader no-GIL ecosystem depend on them.

Current state of published wheel indexes:

Index cp313 cp314 Free-threaded
CPU (/whl/cpu/) ✅ ❌ ❌
Metal (/whl/metal/) ❌ ❌ ❌
CUDA (/whl/cu1xx/) ❌ ❌ ❌

Proposed changes

1. Expand prebuilt wheel matrix (highest impact, smallest change)

Update CIBW_BUILD in Metal/CUDA workflows and add free-threaded support. This is the single highest-impact change — it eliminates source builds for most users.

build-wheels-metal.yaml:

Upgrade cibuildwheel from v2.22.0 to v3.x (3.0 added cp314/cp314t support). In cibuildwheel 3.0, cp314t is built by default (free-threading is no longer experimental in 3.14), and cp313t requires CIBW_ENABLE: cpython-freethreading.

-        uses: pypa/[email protected]
+        uses: pypa/[email protected]
         env:
-          CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-*"
+          CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-* cp313-* cp314-*"
+          CIBW_ENABLE: cpython-freethreading

build-and-release.yaml — same cibuildwheel upgrade, and update the build_wheels_arm64 job:

-          CIBW_BUILD: "cp38-* cp39-* cp310-* cp311-* cp312-*"
+          CIBW_BUILD: "cp38-* cp39-* cp310-* cp311-* cp312-* cp313-* cp314-*"
+          CIBW_ENABLE: cpython-freethreading

build-wheels-cuda.yaml — uses a different build system (python -m build --wheel with a PowerShell matrix). The pyver matrix would need "3.13", "3.14" added.

With prebuilt wheels, any downstream project can use uv's declarative index support:

# pyproject.toml — zero Makefile, zero CMAKE_ARGS
[project]
dependencies = ["llama-cpp-python~=0.3"]

[tool.uv.sources]
llama-cpp-python = [
  { index = "llama-metal", marker = "sys_platform == 'darwin'" },
  { index = "llama-cpu",   marker = "sys_platform == 'linux'" },
]

[[tool.uv.index]]
name = "llama-metal"
url = "https://abetlen.github.io/llama-cpp-python/whl/metal"
explicit = true

[[tool.uv.index]]
name = "llama-cpu"
url = "https://abetlen.github.io/llama-cpp-python/whl/cpu"
explicit = true
2. Document --config-settings as the source-build path

Since the build backend is scikit-build-core, cmake args can be passed via the standard PEP 517 config-settings interface:

pip install llama-cpp-python -C cmake.args="-DGGML_METAL=on"
# or with uv:
uv pip install llama-cpp-python -C cmake.args="-DGGML_METAL=on"

This is cleaner than the CMAKE_ARGS env var — it's the standard PEP 517 mechanism, more explicit, and discoverable. It's already supported via scikit-build-core but not documented in the README or install docs.

3. (Future) Adopt PEP 817 Wheel Variants

PEP 817 (draft, Dec 2025) introduces a standard mechanism for GPU/accelerator wheel variants. PyTorch 2.9 already ships experimental variant-enabled wheels. Once PEP 817 is accepted and tool support lands, llama-cpp-python could publish variant wheels that are auto-selected by the installer:

# Future: just works, installer picks Metal/CUDA/CPU automatically
pip install llama-cpp-python

This is mentioned for context only — the actionable items are (1) and (2) above.

Ecosystem context

  • Quansight offered funded engineering help for free-threaded support in #2103 (via vLLM ecosystem work) — awaiting maintainer signal
  • ~470K monthly PyPI downloads (pypistats) — every project using this beyond toy scripts hits this install wall
  • How others solved it: PyTorch uses per-backend index URLs + PEP 817 variants; ONNX Runtime publishes separate PyPI packages per backend (onnxruntime-gpu, onnxruntime-silicon)

Related

Wheel matrix gaps (same root cause):

  • #2103 — Pre-built wheels for Python 3.14 and 3.14 free-threaded
  • #2130 — Pre-built CPU-only wheel for Windows (cp313)
  • #2068 — Where can I download wheel for CUDA 12.8?
  • #2091 — CUDA 12.8 wheel request

Wheel variants / long-term packaging:

  • #2092 — Add support for experimental wheel variants (wheelnext)
  • #1506 — Multi-arch support for pre-built CPU wheel (by @abetlen)
  • Discussion #1875 — Automating pre-building of wheels for all platforms

Downstream impact of missing wheels:

  • #2118 — Installation deadlock on Hugging Face Spaces (musl/glibc mismatch)
  • #2113 — No working wheels for Debian/Ubuntu

Happy to submit a PR for (1) and (2).

Lingua principale
Python
Stelle
10.6k
Fork
1.5k
Merge medio
3h 57m
PR unite (30g)
4

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di abetlen/llama-cpp-python

Tutte le issue di abetlen/llama-cpp-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.