vllm-project/tpu-inference
Vedi su GitHub[Feature] Deprecate pre-quantized model loading via Qwix for the JAX/Flax path
Open
#1680 aperta il 10 feb 2026
good first issue
Metriche repository
- Star
- (348 star)
- Metriche merge PR
- (Metriche PR in attesa)
Descrizione
🚀 The feature, motivation and pitch
Once #1623 is complete, we should fully deprecate using Qwix to load prequantized models (example)
More specifically, we need to do the followng:
- Deprecate the usage of
apply_qwix_on_abstract_model, since we should only be applying Qwix "on-the-fly" (that is, on an unquantized model). We only apply need to Qwix on the abstract model in the case that the model is pre-quantized - Deprecate all of the "default" quantization logic
- Convert Llama4 and GPT-OSS to avoid using Qwix to load the pre-quantized model
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.