Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Latent int16 overflow in ggml_vec_dot_i2_i8_s_1x1: 128 maddubs results per lane against a 32767 ceiling

未关闭
  • 哪里不对？
#629 5 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
56/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
c, cpp
领域
backend, performance

调研方向

从 ggml/src/ggml-cpu/quants.c 中的 ggml_vec_dot_i2_i8_s_1x1 开始，跟踪 int16 累加经过 32 次迭代的循环和最终 fold 的过程。使用 BitNet-b1.58-2B-4T 权重和 +127 int8 激活值复现问题，然后将 fallback 的输出与精确的 int64 算术进行比较。当 fallback 不再发生回绕，同时对普通激活值仍能保留正确结果时，即视为完成。

由索引模型根据 Issue 内容生成。

描述

Summary

ggml_vec_dot_i2_i8_s_1x1 accumulates 128 vpmaddubsw results into one
int16 lane before folding. The worst case is 128 * 508 = 65024 against a
32767 ceiling. It is correct in practice only because signed activations
cancel, and it is latent on a default build because the sgemm path takes
the work instead.

I am filing it as hardening, not as a wrong-output bug, and the distinction is
deliberate — see Severity below.

The arithmetic

ggml/src/ggml-cpu/quants.c:

const int group32_num = nb / 32;                     // :1396
...
__m256i accu32 = _mm256_setzero_si256();             // :1413  (int16 lanes)
for (int j = 0; j < 32; j++) {                       // :1414
    ...
    accu32 = _mm256_add_epi16(accu32, _mm256_add_epi16(xq8_0, xq8_1));   // :1439
    accu32 = _mm256_add_epi16(accu32, _mm256_add_epi16(xq8_2, xq8_3));   // :1440
}
accu = _mm256_add_epi32(_mm256_madd_epi16(accu32, one16), accu);         // :1445

32 iterations × 4 planes = 128 vpmaddubsw results per int16 lane. Each is at
most 2 * 2 * 127 = 508, so the lane can reach 65024, roughly twice the
ceiling. The name accu32 suggests 32-bit; the type is __m256i used with
_mm256_add_epi16.

What actually happens

With all activations the same sign the sum does not cancel and the lane wraps.
Measured on the real weights of BitNet-b1.58-2B-4T — 30 tensors at
K = 6912, driven against int8 activations pinned to +127 — 11,998 of
12,000 rows disagree with exact int64 arithmetic, and every error is a
multiple of 65536, the signature of a wrapped int16.

With ordinary int8 activations, 0 of 12,000 rows wrap. That is why it has
never been seen.

Severity: low, and I want to be accurate about why

  • On a build where llamafile_sgemm handles these shapes this function is not
    called at all. The kernel's own counters on my build read
    calls=0 sgemm=52068.
  • It becomes reachable when that path is disabled, which is a supported switch
    rather than an exotic one.
  • Even then it needs same-signed activations across a whole group of 32 blocks.

So: a latent overflow in a fallback, not a correctness bug users are hitting.
I would not have filed it on its own; I am filing it because it is the same
missing bound-derivation as the fold issue I opened alongside this one (#628), in the
same file family, and a fix for one is naturally a fix for both.

Not a duplicate of the ARM reports

#411 and the audit in #525 report an int16 overflow, but it is the NEON
vmlal_s8 accumulation in src/ggml-bitnet-mad.cpp, which does produce visible
garbage. This is a different function, a different instruction set and a
different failure mode — this one is invisible precisely because it cancels.

Suggested fix

Fold every 8 groups rather than every 32, or accumulate in int32 directly.
Upstream ggml's equivalent kernels derive the bound explicitly and stay inside
it — TQ2_0 in arch/x86/quants.c carries // 16-bit sums, because 256*127 still fits — which is the pattern worth copying here.

Reproducible from https://github.com/purpleskulll/bitnet-t5b.

主要语言
C++
星标
40.3k
派生
3.7k
30 天内没有已合并 PR

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

microsoft/BitNet 的其他 Issue

查看 microsoft/BitNet 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。