[Deployment Issue] Installation deadlock on Hugging Face Spaces (CPU): Wheels fail (musl/glibc mismatch) & Source builds timeout

Open
#2,118 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
cmake, docker, python
Domain
backend, cloud, devops

Research direction

The issue is about deployment on Hugging Face Spaces. Examine the Dockerfile patterns in the repository, especially the official image at ghcr.io/abetlen/llama-cpp-python. Look at the build scripts and CI configurations to understand how wheels are built for different platforms. Check if there are existing Dockerfiles for CPU-only environments. The goal is to produce a working Dockerfile that avoids musl wheels and builds within time limits.

Written by the indexing model from the issue text.

Description

System Info

Platform: Hugging Face Spaces (Docker / CPU Basic Tier)

Base Image: python:3.10-slim (Debian Bookworm)

Goal: Deploy OpenAI-compatible server for a GGUF model.

Description I am unable to deploy llama-cpp-python on a CPU-only Docker environment (Hugging Face Spaces). I have attempted three distinct installation methods, all of which fail.

Describe the solution you'd like
Is there a recommended Docker pattern for Debian-based CPU-only deployments that avoids the musl wheel issue but does not require a full source compilation?

Describe alternatives you've considered

Attempt 1: Pre-built Wheels (Architecture Mismatch) I attempted to install using the CPU-specific extra index URL to avoid compilation time.

Dockerfile:

RUN apt-get update && apt-get install -y build-essential cmake gcc
RUN pip install llama-cpp-python \
    --prefer-binary \
    --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu

Error: The installer pulls a wheel that requires musl, causing a runtime crash on Debian.

OSError: libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory

Attempt 2: Official Docker Image I attempted to use the official image to avoid installation steps entirely. Dockerfile:

FROM ghcr.io/abetlen/llama-cpp-python:latest...

Error: The server fails to start, unable to load the shared library (likely due to GPU/CUDA dependencies in :latest or pathing issues).

FileNotFoundError: Shared library with base name 'llama' not found

Attempt 3: Build from Source I attempted to build from source to ensure correct architecture. Dockerfile:

ENV CMAKE_ARGS="-DGGML_CUDA=off"
RUN pip install llama-cpp-python --no-cache-dir

Error: The build process hits the Hugging Face Spaces timeout limit (hard limit) during the wheel building phase.

Building wheel for llama-cpp-python (pyproject.toml): started
... [Process limits out / Times out]```
Dominant language
Python
Stars
10.7k
Forks
1.5k
Avg merge
1h 40m
Merged PRs (30d)
5

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from abetlen/llama-cpp-python

All issues in abetlen/llama-cpp-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.