Feature Request: gemma4 31B chat template quadratic scan cost
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
Research direction
Start with models/templates/google-gemma-4-31B-it.jinja, especially lines 219-230, and compare it with Google's current Gemma 4 31B implementation in the linked discussion. Decide whether to update the whole template or remove the scan, then verify rendered output remains byte-identical and that render time no longer scales quadratically.
Written by the indexing model from the issue text.
Description
Prerequisites
- I am running the latest code. Mention the version if possible as well.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description
File: models/templates/google-gemma-4-31B-it.jinja, lines 219-230
What it does: For every message, it scans backwards to find the previous non-tool role. After the role is found, the loop does not stop.
What it should be: Google updated the Gemma 4 template. We should either update it to match or remove this scan.
Motivation
Environment: Mac M4 Pro, commit 436f6f8, tested template render times:
The current template scales quadratically with message count. Tested with 64 messages: 11.3 ms -> 3.1 ms and with 184 messages: 72.2 ms -> 10.9 ms (current implementation -> scan-removed version). Rendered output is byte-identical.
The template in the repo has not been updated since #21704 (2026-04-10).
Possible Implementation
Google made the same fix (merged on 2026-07-15): https://huggingface.co/google/gemma-4-31B-it/discussions/118
Possible options are (a) update the whole template to match Google's current implementation or (b) remove this scan. Which option do you prefer?
- Dominant language
- C++
- Stars
- 132k
- Forks
- 24.6k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 501
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ggml-org/llama.cpp
-
bug-unconfirmed
Difficulty 1/5 Under an hour Newbie friendliness 91/100
Maintainers usually reply within 1 day
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
Maintainers usually reply within 1 day
-
Misc. bug: Metal aborts on MUL_MAT with BF16 src1 at K >= 64, batch > 8 (no BF16 mul_mm kernel)Possibly taken @rashid5925 claimed this 1 day ago. Openbug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
ggml-org/llama.cpp#30973 · 4 comments ·
Maintainers usually reply within 1 day
-
Windows bundled llama-server.exe is missing `longPathAware`: GGUF paths > 260 chars fail with "No such file or directory" even when the file exists and LongPathsEnabled=1Possibly taken @origin-labs-ai claimed this 1 day ago. Openbug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
All issues in ggml-org/llama.cpp
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 69/100
notepad-plus-plus/notepad-plus-plus#18553 ·
Maintainers usually reply within 1 day
-
bug docs performance
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 3 days
-
good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
good first issue neural P2
Difficulty 2/5 1-3 hours Newbie friendliness 65/100