Windows: every failed `import llama_cpp` prepends to PATH and adds DLL directories again
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in llama_cpp/_ctypes_extensions.py and trace load_shared_library, especially the Windows os.add_dll_directory and PATH updates before the CDLL load. Reproduce with repeated failed imports on Windows and check how DLL directory handles are managed when loading raises. Done means failed attempts leave PATH and DLL search directories unchanged or avoid adding duplicate entries.
Written by the indexing model from the issue text.
Description
On Windows, load_shared_library in llama_cpp/_ctypes_extensions.py changes process state before it knows the library will load:
if sys.platform == "win32":
os.add_dll_directory(str(base_path))
os.environ["PATH"] = str(base_path) + os.pathsep + os.environ["PATH"]
if sys.platform == "win32" and sys.version_info >= (3, 8):
os.add_dll_directory(str(base_path))
if "CUDA_PATH" in os.environ:
os.add_dll_directory(os.path.join(os.environ["CUDA_PATH"], "bin"))
os.add_dll_directory(os.path.join(os.environ["CUDA_PATH"], "lib"))
If llama.dll then fails to load, the import raises and Python does not cache the module. The next import llama_cpp runs all of this again. Each failed attempt adds one more PATH entry and four more DLL directories, and none of them are removed.
That matters for long-running apps that check whether llama.cpp is available, for example on a status poll. In our case the app retried the import from a status check. After about three hours the process PATH held the same llama_cpp\lib entry 104 times (12,323 characters). Then an unrelated library failed at its own os.add_dll_directory call with [WinError 206] The filename or extension is too long.
Reproduce
Use any install where llama.dll cannot load. A CUDA 12 wheel on a machine with only CUDA 13 installed does it:
import os
for _ in range(3):
try:
import llama_cpp
except RuntimeError as error:
print(type(error).__name__)
print(os.environ["PATH"].count("llama_cpp")) # prints 3, expected 1
Expected
A failed load leaves PATH and the DLL search list as they were, or adds each entry at most once per process.
Possible fix
- Keep the handles that
os.add_dll_directoryreturns, and close them when theCDLLcall fails. - Prepend to PATH only if the entry is not already there.
base_pathis added twice on Python 3.8 and later. One call is enough.
Environment
- llama-cpp-python 0.3.37 (CUDA wheel, built against cublas64_12 and cudart64_12)
- Windows 11, Python 3.12
CUDA_PATHpoints to CUDA 13.1
- Dominant language
- Python
- Stars
- 10.7k
- Forks
- 1.5k
- Avg merge
- 1h 40m
- Merged PRs (30d)
- 5
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from abetlen/llama-cpp-python
-
Seven llama_sampler_init_* bindings admit keyword arguments that the ctypes function object silently dropsPossibly taken @Belal0066 claimed this 59 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
abetlen/llama-cpp-python#2371 ·
Maintainers usually reply within 1 day
-
uv add llama-cpp-python wheels fails for versions above 0.3.30Possibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2352 · 1 comment · 3 reactions ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
abetlen/llama-cpp-python#2314 ·
Maintainers usually reply within 1 day
-
Llama.embed() calls LlamaBatch.add_sequence with old 3-arg signature; missing logits_arrayMay be free again @lxcxjxhx claimed this 136 days ago, and no pull request is open. Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2211 · 2 comments ·
Maintainers usually reply within 1 day
-
Llama() silently accepts and discards `embedding` kwarg; .embed() then raises confusinglyPossibly taken @Anai-Guo claimed this 76 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
abetlen/llama-cpp-python#2210 ·
Maintainers usually reply within 1 day
All issues in abetlen/llama-cpp-python
Similar issues
-
Bug IronPython Python 3
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pyrevitlabs/pyRevit#4184 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 74/100
jqueryscript/awesome-claude-code#1009 ·
-
tests
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
[Plugin] 群聊记忆 · 人格养成Open
Difficulty 1/5 1-3 hours Newbie friendliness 74/100