vllm-project/vllm

[Transformers v5] Base model and LoRA used in test has incorrect `tokenizer_config.json`

Open

#38386 opened on Mar 27, 2026

View on GitHub
 (8 comments) (0 reactions) (1 assignee)Python (80,034 stars) (16,816 forks)batch import
good first issuehelp wanted

Description

This is a sub-issue forming part of the work in https://github.com/vllm-project/vllm/issues/38379, please read the description of this issue before beginning to work on this one.

Which test is failing?

The tokenizer_config.json is incorrect for both the base model and the adapter. If we duplicated these checkpoints and stored them inside https://huggingface.co/vllm-project, then we could own them and update the tokenizer class to be PreTrainedTokenizerFast which will almost always work.

$ pytest tests/lora/test_quant_model.py::test_quant_model_lora[model0]
...
AssertionError: assert ['#f07733: A ...#f08800: A v'] == ['#f07700: A ...#f00000: A v']
[2026-03-27T01:15:06Z]
[2026-03-27T01:15:06Z]   At index 0 diff: '#f07733: A v' != '#f07700: A v'
[2026-03-27T01:15:06Z]
[2026-03-27T01:15:06Z]   Full diff:
[2026-03-27T01:15:06Z]     [
[2026-03-27T01:15:06Z]   -     '#f07700: A v',
[2026-03-27T01:15:06Z]   ?           ^^
[2026-03-27T01:15:06Z]   +     '#f07733: A v',
[2026-03-27T01:15:06Z]   ?           ^^
[2026-03-27T01:15:06Z]   -     '#f00000: A v',
[2026-03-27T01:15:06Z]   ?         ^^
[2026-03-27T01:15:06Z]   +     '#f08800: A v',
[2026-03-27T01:15:06Z]   ?         ^^
[2026-03-27T01:15:06Z]     ]

How to configure my environment?

It's very important that you install both vLLM and Transformers from source so that your test results reflect the current state of both libraries.

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers

Contributor guide

Tech stack
pythonpytorch
Domain
testingmachine learning
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
blocked
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
PythonpytestgitvLLM development environment
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45
Research direction
The failing test is tests/lora/test quant model.py with model0. The expected tokenizer config.json values are shown in the assertion error. The fix requires duplicating the base model and LoRA adapter to the vllm project Hugging Face organization and updating the tokenizer class to PreTrainedTokenizerFast. Check the parent issue #38379 for context and any related changes.