Support mixed-type quantization within a single quantization backend

Ouverte
#15,320 0 commentaires 0 réactions 1 personne assignée Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

@sayakpaul y travaille déjà.

Depuis le 17/11/2026.

Évaluation

Cette issue n'a pas encore été évaluée.

Description

Motivation

Other tools like ComfyUI and Unsloth already support mixed-type quants: different layers or modules of a model are quantized with different schemes or precisions. Some layers stay in high precision, and others drop to lower bit-widths. This usually gets a better quality/memory trade-off than quantizing every layer the same way.

Most of the quantization backends Diffusers integrates with already support this natively. Diffusers should let users use that capability.

Proposal

  • Allow mixed quantization types within a single backend (for example, different torchao or bnb configs per module/layer pattern), when the underlying backend framework supports it.
  • Out of scope for now: mixing backends in one checkpoint (for example, some layers bnb and others torchao). Not all backends are feature-complete, so this would be messy. We also haven't seen such checkpoints in practice.

Open questions

  • API design: how users specify per-module/per-layer quant configs (pattern-based mapping, modules_to_not_convert-style extension, etc.).
  • Serialization and loading of mixed-quant checkpoints, plus compatibility with existing pre-quantized checkpoints (for example, from the ComfyUI / Unsloth ecosystems).
  • Which backends support this today, and what's missing in each.

Benchmarks / guidance

As part of this work, produce:

  • Benchmarks (memory, latency) comparing uniform vs. mixed quantization on representative pipelines.
  • Qualitative comparisons of outputs.
  • Better-grounded heuristics for choosing which layers to keep in higher precision. Existing community guides give heuristics, but they're hit and miss.

Requested by Sayak Paul - Slack thread - Agent trace

Langage dominant
Python
Étoiles
34.9k
Forks
7.5k
Merge moyen
3 j 16 h
PR mergées (30 j)
65

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de huggingface/diffusers

Toutes les issues de huggingface/diffusers

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.