[Bug]: I2_S GEMM fast path produces garbage for multi-token prompts on AVX-only CPUs

Commencez dans ggml-cpu.c, au niveau du chemin rapide GEMM I2_S autour de la ligne 1492, et inspectez ggml_gemm_i2_i8_s pour le traitement du stride des colonnes lorsque nr e 1. Compilez avec AVX2 désactivé en utilisant les CMake flags documentés, puis reproduisez le problème avec des prompts courts et longs et comparez le chemin GEMM avec le chemin GEMV fonctionnel autour de la ligne 1217. La tâche est terminée lorsque les prompts I2_S multi-token produisent une sortie cohérente sur des CPU AVX-only sans désactiver le chemin rapide.

Description

On CPUs without AVX2 (e.g., Intel Xeon E5-2690 v2, Ivy Bridge, AVX-only), the I2_S GEMM fast path in ggml-cpu.c:1492 produces corrupt output when processing prompts with more than ~3 tokens. Single-token generation (GEMV path) works correctly.

Symptoms

• 1-3 token prompts → coherent output
• 5+ token prompts → ?????? or garbled output
• The corruption affects the KV cache: even after the prompt is processed, subsequent generation tokens are garbled

Root Cause

The GEMM fast path (ggml_gemm_i2_i8_s) is called for multi-token prompt evaluation (when src1 has multiple columns). The scalar fallback implementation has a bug in how it indexes the I2_S weight matrix and/or activation matrix for column strides > 1.

Workaround

Disabling the GEMM fast path forces I2_S through the dequantize-then-float-matmul path:

// ggml-cpu.c:1492 — change from:
if (src0->type == GGML_TYPE_I2_S && ggml_n_dims(src0) == 2) {
// to:
if (false && src0->type == GGML_TYPE_I2_S && ggml_n_dims(src0) == 2) {

This produces correct results but is slower (~0.6 tok/s prompt eval vs ~26 tok/s for F16 on the same hardware).

Key Distinction from #547

This is distinct from #547/PR #580 which covers the empty-body scalar fallback for ggml_vec_dot_i2_i8_s_* kernels. After applying those fixes (or equivalent scalar implementations), the GEMM path still produces wrong results for multi-token prompts.

The GEMV path (line ~1217) works correctly for single-token generation. The issue is specifically in ggml_gemm_i2_i8_s when called with nr > 1 (multiple activation columns).

Reproduction

# Build with AVX2 disabled (forces scalar fallback)
cmake -B build -DBITNET_ARM_TL1=OFF -DBITNET_X86_TL2=OFF
cmake --build build --target llama-server -j8

# Short prompt works
curl http://localhost:8081/v1/completions \
  -d '{"model":"bitnet","prompt":"Hello","max_tokens":20}'
# → " there! I'm happy to help" ✓

# Longer prompt fails
curl http://localhost:8081/v1/completions \
  -d '{"model":"bitnet","prompt":"The weather today is","max_tokens":20}'
# → "?????" ✗

Also requires fixes #588 (ReLU²) and PR #616 (weight scale direction) for coherent F16 output.

Environment

• CPU: Intel Xeon E5-2690 v2 (Ivy Bridge, AVX only, no AVX2)
• OS: Ubuntu 24.04 LTS
• Compiler: Clang 18.1.3
• CMake flags: -DBITNET_ARM_TL1=OFF -DBITNET_X86_TL2=OFF
• Model: BitNet-b1.58-2B-4T (I2_S format)
• BitNet commit: 390c30775