Hacktoberfest 2026：維護者為十月標記出來的 issue，仍然開放、適合新手。 瀏覽 Hacktoberfest issue

ARM I2_S inference produces gibberish/garbage output after commit 112f853 (CPU Optimization update)

未關閉
  • 哪裡不對？
#470 3 則留言 4 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

評估

難度
4/5
預估耗時
3-5 天
新手友好度
48/100
Issue 類型
缺陷
描述清晰度
基本清楚
活躍度
停滯
技術堆疊
cpp
領域
machine-learning, performance

研究方向

使用 run_inference.py 和 i2_s 模型重現 ARM aarch64 推論失敗，然後比較正常運作的提交 404980e 與造成回歸的提交 112f853。先從 ggml-bitnet-mad.cpp 和由 utils/codegen_tl1.py 產生的核心著手，檢查 NEON 和點積路徑；在回報的 ARM 平台上取得一致的輸出，且不造成最佳化回歸，即視為完成。

由索引模型根據 Issue 內容生成。

描述

Description

After the CPU Inference Optimization update (commit 112f853), running inference with i2_s quantization on ARM (aarch64) produces completely incoherent output — random tokens with no relation to the prompt. Rolling back to commit 404980e (the last commit before the optimization merge) restores correct, coherent output.

Environment
  • Hardware: Raspberry Pi 5 (8GB RAM), ARM Cortex-A76 (aarch64)
  • OS: Raspberry Pi OS 64-bit (Debian 12 Bookworm)
  • Compiler: Debian clang version 18.1.8
  • CMake: 3.25.1
  • Python: 3.9 (conda)
  • Model: microsoft/BitNet-b1.58-2B-4T-gguf (ggml-model-i2_s.gguf)
  • Quantization: i2_s
Steps to Reproduce

  1. Clone repo at current HEAD (01eb415):

    git clone --recursive https://github.com/microsoft/BitNet.git
cd BitNet

  2. Generate kernels and build (following Adafruit guide):

    python utils/codegen_tl1.py --model bitnet_b1_58-3B --BM 160,320,320 --BK 64,128,64 --bm 32,64,32
export CC=clang-18 CXX=clang++-18
rm -rf build && mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
cd ..

  3. Download model:

    huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4T

  4. Run inference:

    python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "You are a helpful assistant" -t 4 -cnv
Broken Output (HEAD - 01eb415)
> hi how are you
ri differentorefFly increase Hurtutar run following section underestimateAD Sachs weighedision
cann RICTS Reyn taskfir-ra mark filtr castWATCHB fr ret flatten missionuche purchase parameter
gramhit associatedyuraft runeded take compound sugar contrast unsubedom conveyuffanford...
Working Output (commit 404980e)
> hi
Hello! How can I assist you today?

> what is a raspi 5
The Raspberry Pi 5 is a next-generation model of the Raspberry Pi single-board computer series...

Performance on the working commit: 9.68 tokens/second (4 threads, ARM NEON).

Bisection

The regression was introduced in commit 112f853:

112f853 [feat] I2S kernels for weight & activation parallel on Intel & ARM machine;
        [feat] I2S GEMV & GEMM(llama.cpp);
        [feat] quantize activation & dequantize embedding(llama.cpp);
        [fix] compile bug: cannot define __ARM_FEATURE_DOTPROD(llama.cpp)

The last known working commit is 404980e (one commit before 112f853).

Notes
  • The build completes without errors on both commits — the issue is runtime behavior, not compilation.
  • ggml-bitnet-mad.cpp is compiled and linked in both cases.
  • NEON is detected and enabled (NEON = 1 in system_info output).
  • DOTPROD detection: GGML_COMPILER_SUPPORT_DOTPROD - Failed, but COMPILER_SUPPORTS_ARMV82_DOTPROD - Success.
  • This issue also appears to affect other ARM64 platforms (Ampere/Hetzner CAX servers), not just Raspberry Pi.
  • The Adafruit BitNet on Raspberry Pi guide (published Sept 2025, before the optimization commit) confirms working output on Pi 4 and Pi 5 with the older codebase.

Related to #411 — same root cause. Adding Pi 5 (Cortex-A76 with dotprod) as another confirmed affected platform.

主要語言
C++
星號
40.3k
分支
3.7k
30 天內沒有已合併 PR

環境準備

這個專案沒有提供開發容器、Dockerfile 或貢獻指南，環境需要你自己搭建：先看它的 README，通用步驟見我們的新手貢獻指南。

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

microsoft/BitNet 的其他 Issue

查看 microsoft/BitNet 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。