ggml-org/llama.cpp

Eval bug: wrong default threads in llama-bench

Open

#17611 opened on Nov 30, 2025

View on GitHub
 (6 comments) (0 reactions) (0 assignees)C++ (110,169 stars) (18,202 forks)batch import
good first issuelow severityperformance

Description

Name and Version

latest commit: https://github.com/ggml-org/llama.cpp/commit/fa0465954faef9d7170b967ad89f8bc5303a32f3

llama-bench using by default wrong threads.

Linux server with NUMA nodes

Operating systems

Linux

GGML backends

CPU

Hardware

Supermicro X10dri Dual CPU

Contributor guide

Tech stack
ccpp
Domain
backendperformancetooling
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
C++ programmingfamiliarity with llama bench codeunderstanding of NUMA
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
Investigate the default thread selection in llama bench, specifically in the context of NUMA systems. The issue suggests that on a Linux server with NUMA nodes, the default thread count is incorrect. Look at the relevant source file (likely llama bench.cpp) to understand how threads are determined and what logic should be used for NUMA. See comments in the issue for any additional context.