ggml-org/llama.cpp

Eval bug: wrong default threads in llama-bench

Open

#17,611 建立於 2025年11月30日

在 GitHub 查看
 (6 留言) (0 反應) (0 負責人)C++ (110,169 star) (18,202 fork)batch import
good first issuelow severityperformance

描述

Name and Version

latest commit: https://github.com/ggml-org/llama.cpp/commit/fa0465954faef9d7170b967ad89f8bc5303a32f3

llama-bench using by default wrong threads.

Linux server with NUMA nodes

Operating systems

Linux

GGML backends

CPU

Hardware

Supermicro X10dri Dual CPU

貢獻者指南

技術棧
ccpp
領域
backendperformancetooling
議題類型
bug
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
1-2 days
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
active
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
clear
前置要求
C++ programmingfamiliarity with llama bench codeunderstanding of NUMA
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
40
研究方向
Investigate the default thread selection in llama bench, specifically in the context of NUMA systems. The issue suggests that on a Linux server with NUMA nodes, the default thread count is incorrect. Look at the relevant source file (likely llama bench.cpp) to understand how threads are determined and what logic should be used for NUMA. See comments in the issue for any additional context.