Windows: conflicts with llama-cpp-python 0.3.37 ggml DLLs in the same process

Open
#202 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
backend, devtools

Research direction

Start by reviewing the Windows DLL loading and bundled ggml library naming in ggml-python, then compare the bundled ggml version with llama-cpp-python 0.3.37. Use the linked Windows CI matrix to reproduce both import orders on Python 3.11. Done means both packages import and the reported ggml call works in either order on Windows.

Written by the indexing model from the issue text.

Description

On Windows, when ggml-python 0.0.45 and llama-cpp-python are used in the same process, both packages load a DLL named ggml and the second import reuses the one already loaded:

  • llama_cpp imported first, llama-cpp-python 0.3.37: ggml-python calls into llama-cpp's newer ggml build and crashes. For example ggml_gallocr_new(ggml_backend_get_default_buffer_type(backend)) raises OSError: exception: access violation writing 0x0000000000000000. With llama-cpp-python 0.3.36 the same code works.
  • ggml imported first (either version): llama_cpp fails to load with Failed to load shared library '...\llama_cpp\lib\llama.dll': [WinError 127] The specified procedure could not be found.

Linux and macOS are not affected. A Windows CI matrix that reproduces both cases is here: https://github.com/MaiAditya/txtai/actions/runs/44072580894 (Python 3.11, windows-latest). This came up in txtai, see neuml/txtai#1584.

Would updating the bundled ggml to the version used by current llama-cpp-python releases be possible? That would make the first case work again. Longer term, giving the bundled DLLs a package-specific name would avoid the clash entirely.

Dominant language
Python
Stars
152
Forks
17
PR merge metrics
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from abetlen/ggml-python

All issues in abetlen/ggml-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.