Add support for Qwen3-vl models

Open
#2,098 6 comments 13 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python

Research direction

The error points to llama_cpp/_internals.py line 78. First, understand the model loading and sampler initialization flow in llama.cpp's C++ core and its Python bindings. Check existing model support definitions and how visual-language models are integrated. 'Done' means Qwen3-vl models load without the AttributeError, likely requiring updates to the model architecture mapping and sampler handling.

Written by the indexing model from the issue text.

Description

Is your feature request related to a problem? Please describe.
When trying to load Qwen3-vl models like Qwen3-VL-235B-A22B-Thinking or Qwen3VL-8B-Instruct, get error messages as following:
site-packages/llama_cpp/_internals.py", line 78, in close
if self.sampler is not None:
AttributeError: 'LlamaModel' object has no attribute 'sampler'

Describe the solution you'd like
Looks that Qwen3-vl models are not supported yet, could you help add the supporting for these Qwen3-vl models?

Dominant language
Python
Stars
10.7k
Forks
1.5k
Avg merge
1h 40m
Merged PRs (30d)
5

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from abetlen/llama-cpp-python

All issues in abetlen/llama-cpp-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.