Request for Gemma 3 4B & Gemma 4 Integration Support with llama-cpp-python

Open
#2,231 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@acarcay is already working on this.

Since Sep 6, 2026.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
45/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
python

Research direction

Look at the existing model support documentation and integration code in the repository, likely in docs or a models directory. Check the llama.cpp upstream for Gemma support status. Determine the required versions, chat templates, and GGUF formats. The goal is to produce clear documentation or examples for users, not to implement the integration itself.

Written by the indexing model from the issue text.

Description

Hi team,

I would like to request official support and documentation for Gemma 3 4B and Gemma 4 integration within llama-cpp-python.

It would be helpful to have clarification regarding:

  • Current compatibility status
  • Required llama.cpp / llama-cpp-python versions
  • Chat template handling requirements
  • Recommended GGUF formats and quantizations
  • Known limitations or issues
  • Recommended inference parameters for stable usage

Any guidance, implementation notes, or examples would be greatly appreciated (without using ollama)

Kind regards,
Jonathan Park

Dominant language
Python
Stars
10.7k
Forks
1.5k
Avg merge
1h 40m
Merged PRs (30d)
5

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from abetlen/llama-cpp-python

All issues in abetlen/llama-cpp-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.