Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

CPU backend: garbage output (repetition loops) for the qwen35 hybrid family (Qwen3.8-27B / Holo4), while the Vulkan backend serves the same binary and GGUF correctly

Open
  • Is something wrong?
#30,906 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
cpp
Domain
machine-learning

Research direction

Start by comparing the CPU and Vulkan execution paths for the qwen35 Gated DeltaNet layers; the report identifies those layers as the likely source and links related issues #27756 and #29493. Reproduce with the provided llama-server command and Qwen3.8-27B GGUF on CPU, then compare outputs with Vulkan. Done means CPU generation no longer falls into repetition loops and agrees with the working backend on the reported prompts.

Written by the indexing model from the issue text.

Description

Summary

The CPU backend produces garbage for the qwen35 hybrid family (24 Gated DeltaNet + 4 gated full-attention layers), while the Vulkan backend serves the same binary and the same GGUF correctly.

Concretely, on this box:

  • llama.cpp b11756, Qwen3.8-27B-AD-IQ3_XXS.gguf (the base model, a plain IQ3_XXS — not an imatrix/i1 quant), Intel Arc Pro B70:
    • --device none → every prompt degenerates into a repetition loop.
    • --device Vulkan0 → correct answers.
  • Same result with Holo4-27B.i1-IQ4_XS.gguf and with Holo4-27B.i1-IQ3_M.gguf.

This is the mirror image of #21888 (garbage on Vulkan Intel, correct on CPU), so it looks like a separate defect rather than a known one.

What was measured

CPU (--device none):

"2+2"                ->  "+ + + + + + + + + …"
"capital of France"  ->  "capitale capitale capitale capitale …"
"… before writing his report …"  ->  "writingwritingwritingwritingwriting…"
"Paris is the capital of France. ×3 …"  ->  "])])])])])])])])])])])])])])"

Vulkan (--device Vulkan0), same binary, same GGUF:

"2+2"                ->  "4"
"capital of France"  ->  "Paris"
"Paris is the capital of France. ×3 …"  ->  "The first sentence is \"Paris is the capital of France.\""

The CPU output is not random noise: it is a repetition loop whose repeated token is drawn from the prompt ("writing", "capitale", "+"), which is the classic signature of a model that cannot use the context and falls back to the highest-probability continuation.

Repro

# CPU (garbage)
./llama-server -m Qwen3.8-27B-AD-IQ3_XXS.gguf \
  --host 127.0.0.1 --port 18995 -c 32768 --device none -t 8 --jinja \
  --reasoning off --reasoning-format none \
  --chat-template-kwargs '{"enable_thinking":false}'
# then:
curl -s http://127.0.0.1:18995/v1/chat/completions -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"What is 2 + 2? Reply with just the number."}],
       "max_tokens":48,"temperature":0.2,"chat_template_kwargs":{"enable_thinking":false}}'
# -> content: " \"+ \"+ \"+ \"+ \"+ …"

# Vulkan (correct) — same binary, same GGUF
./llama-server -m Qwen3.8-27B-AD-IQ3_XXS.gguf \
  --host 127.0.0.1 --port 18992 -c 32768 -ngl 99 -fa on --device Vulkan0 --jinja \
  --reasoning off --reasoning-format none \
  --chat-template-kwargs '{"enable_thinking":false}'
# -> content: "4"

Environment: Intel Arc Pro B70 (Xe2 / Battlemage), Vulkan driver Mesa 26.2.5 (ANV), Level Zero 1.3+, oneAPI 2026.2, Ubuntu 24.04, tag b11756.

Notes

  • Not the quant: the base IQ3_XXS model fails identically, and so do all five i1 Holo4 GGUFs. Nothing i1-specific is required.
  • Not the chat template: --jinja is on in both runs above, and the rendered prompt is byte-identical between the CPU and Vulkan servers (/apply-template returns the same string).
  • Not the KV cache dtype: the CPU failure is the same with -ctk/-ctv f16 and q4_0.
  • It is specific to this architecture family in a way that looks like the Gated DeltaNet path: the family is 24 GDN + 4 attention layers, and a state-carrying layer whose CPU implementation is wrong would produce exactly this — a model that cannot use the context.
  • Related, possibly the same root cause on the GPU side: #27756 (silent instant-EOS beyond ~130k context for Qwen3.8-27B) and #29493 (test-recurrent-state-rollback failing deterministically for the qwen35 arch). If the CPU recurrent-state path and the GPU one share a code path, this may be the same bug seen from two angles.
Dominant language
C++
Stars
132k
Forks
24.6k
Avg merge
2d 6h
Merged PRs (30d)
500

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ggml-org/llama.cpp

All issues in ggml-org/llama.cpp

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.