llama-bench default verbosity swallows GGML_LOG_ERROR (Metal OOM reported only as "res = -3")
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- cpp
- Domain
- cli, observability
Research direction
Start at llama-bench's log callback and reproduce the reported failure with and without -v, comparing stderr output. Trace how log levels are filtered, then verify that backend error messages appear at default verbosity while verbose filtering remains unchanged.
Written by the indexing model from the issue text.
Description
When a backend fails during llama-bench, the only output at default verbosity is e.g.:
test_prompt: failed to decode prompt batch, res = -3
llama_bench: error: failed to run prompt
The backend's actual error - in our case:
ggml_metal_synchronize: error: command buffer 0 failed with status 5
error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
is filtered out unless -v is passed, because llama-bench's log callback drops non-verbose lines regardless of level. Errors should bypass the verbosity filter: we spent hours theorizing about kernels while the one line naming the real cause (GPU OOM) was being discarded.
Repro: bench any model slightly over the Metal working set on Apple Silicon (we hit it with a 112 GiB model against a 107.5 GiB recommendedMaxWorkingSetSize), compare stderr with and without -v.
- Dominant language
- C++
- Stars
- 129k
- Forks
- 23.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 411
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ggml-org/llama.cpp
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
bug-unconfirmed
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
/v1/responses: reasoning item with "summary": null rejected as "Cannot determine type of 'item'" Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
All issues in ggml-org/llama.cpp
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
infiniflow/infinity#3502 ·