ggml-org/llama.cpp

Eval bug: wrong default threads in llama-bench

Open

#17,611 创建于 2025年11月30日

在 GitHub 查看
 (6 评论) (0 反应) (0 负责人)C++ (110,169 star) (18,202 fork)batch import
good first issuelow severityperformance

描述

Name and Version

latest commit: https://github.com/ggml-org/llama.cpp/commit/fa0465954faef9d7170b967ad89f8bc5303a32f3

llama-bench using by default wrong threads.

Linux server with NUMA nodes

Operating systems

Linux

GGML backends

CPU

Hardware

Supermicro X10dri Dual CPU

贡献者指南

技术栈
ccpp
领域
backendperformancetooling
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-2 days
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
active
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
C++ programmingfamiliarity with llama bench codeunderstanding of NUMA
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
40
研究方向
Investigate the default thread selection in llama bench, specifically in the context of NUMA systems. The issue suggests that on a Linux server with NUMA nodes, the default thread count is incorrect. Look at the relevant source file (likely llama bench.cpp) to understand how threads are determined and what logic should be used for NUMA. See comments in the issue for any additional context.