Apple Silicon Metal + BLAS segfault for I2_S when ubatch >= 32 routes generic MUL_MAT
还没有人认领这个 Issue。
评估
调研方向
从 ggml-blas.cpp 中通用的 BLAS MUL_MAT 支持检查开始，然后使用 i2_s 和 ubatch 值为 31 以及 32 或更高的配置，在 Metal 和 Apple BLAS 上复现该问题。确认 I2_S 保护条件保留了专用路径，并且在启用 BLAS 的情况下，较大的 ubatch 配置不再发生段错误。
由索引模型根据 Issue 内容生成。
描述
Environment
- Platform: Apple Silicon Mac
- Host: Apple M4 Max
- OS: macOS
- Compiler: Homebrew clang 18.1.8
- BitNet / submodule state: BitNet using vendored
3rdparty/llama.cppat Eddie-Wang1120/llama.cpp commit1f86f058de0c3f4098dedae2ae8653c335c868a1 - Model:
microsoft/BitNet-b1.58-2B-4T-gguf/ggml-model-i2_s.gguf - Build flags:
GGML_METAL=ONGGML_ACCELERATE=ONGGML_BLAS=ONGGML_BLAS_VENDOR=AppleBITNET_ARM_TL1=OFF
Problem
On Apple Silicon with Metal enabled, i2_s inference can segfault when BLAS is enabled and the physical micro-batch crosses the BLAS routing threshold.
The crash is tied to physical ubatch, not logical batch:
-b 2048 -ub 31-> stable-b 32 -ub 31-> stable-b 2048 -ub 32-> segfault-b 2048 -ub 512-> segfault
This means the failure starts exactly when the BLAS backend begins claiming the generic MUL_MAT path for larger batches.
Control Experiment
The same Metal runtime is stable when BLAS is disabled:
- BLAS ON +
-b 2048 -ub 512-> segfault - BLAS OFF +
-b 2048 -ub 512-> stable
This strongly suggests the crash is in the BLAS-side handling of GGML_TYPE_I2_S, not in Metal itself and not in the outer chat request schema.
Root Cause
ggml-blas.cpp allows the generic BLAS MUL_MAT path to accept quantized source tensors when ggml_get_type_traits(src0->type)->to_float != NULL.
For GGML_TYPE_I2_S, that is not safe:
I2_Sstores an external scale outside the per-row payload- the generic BLAS dequantize-to-float path assumes self-contained per-row data
- once
ubatch >= 32, BLAS starts claimingMUL_MAT - that eventually crashes in the
i2_sdequant / BLAS matmul path
In crash reports, the top frames consistently land in:
dequantize_row_i2_sggml_backend_blas_mul_mat
Proposed Fix
Reject GGML_TYPE_I2_S in the generic BLAS MUL_MAT support check so that I2_S continues using its specialized non-BLAS path:
return src0->type != GGML_TYPE_I2_S &&
ggml_is_contiguous(src0) &&
ggml_is_contiguous(src1) &&
src1->type == GGML_TYPE_F32 &&
(ne0 >= min_batch && ne1 >= min_batch && ne10 >= min_batch) &&
(src0->type == GGML_TYPE_F32 || ggml_get_type_traits(src0->type)->to_float != NULL);
Result After Patch
After applying the BLAS guard above:
- BLAS ON + Metal +
-b 2048 -ub 512is stable - managed broker end-to-end requests no longer segfault under the same settings
This does not solve all i2_s quality issues, but it does remove the native crash path.
Related Issues
- #468
- #470
- #195
- #411
- 主要语言
- C++
- 星标
- 40.3k
- 派生
- 3.7k
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
microsoft/BitNet 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 86/100
-
i2_s SIGSEGVs at n_ubatch >= 32: BLAS backend dequantises by a row stride 4x the real packed row未关闭
难度 2/5 1-3 小时 新手友好度 76/100
-
难度 2/5 1-3 小时 新手友好度 86/100
-
难度 1/5 1 小时以内 新手友好度 92/100
-
难度 2/5 1-3 小时 新手友好度 76/100
相似的 Issue
-
Feature
难度 1/5 1 小时以内 新手友好度 65/100
Narezzurri/OpenVPN-Config-Manager#95 ·
维护者通常 1 天内回复
-
bot-found bug priority: P1
难度 2/5 1-3 小时 新手友好度 68/100
madenvel/KalinkaPlayer#251 ·
-
难度 2/5 1-3 小时 新手友好度 78/100
sqlitebrowser/sqlitebrowser#4208 ·
-
ROSES ROSES - Student Review
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
-
area/ysql kind/bug priority/medium
难度 2/5 1-3 小时 新手友好度 75/100
yugabyte/yugabyte-db#34552 ·
维护者通常 1 天内回复