Support mixed-type quantization within a single quantization backend

Aperta
#15,320 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@sayakpaul ci sta già lavorando.

Dal 17/11/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

Motivation

Other tools like ComfyUI and Unsloth already support mixed-type quants: different layers or modules of a model are quantized with different schemes or precisions. Some layers stay in high precision, and others drop to lower bit-widths. This usually gets a better quality/memory trade-off than quantizing every layer the same way.

Most of the quantization backends Diffusers integrates with already support this natively. Diffusers should let users use that capability.

Proposal

  • Allow mixed quantization types within a single backend (for example, different torchao or bnb configs per module/layer pattern), when the underlying backend framework supports it.
  • Out of scope for now: mixing backends in one checkpoint (for example, some layers bnb and others torchao). Not all backends are feature-complete, so this would be messy. We also haven't seen such checkpoints in practice.

Open questions

  • API design: how users specify per-module/per-layer quant configs (pattern-based mapping, modules_to_not_convert-style extension, etc.).
  • Serialization and loading of mixed-quant checkpoints, plus compatibility with existing pre-quantized checkpoints (for example, from the ComfyUI / Unsloth ecosystems).
  • Which backends support this today, and what's missing in each.

Benchmarks / guidance

As part of this work, produce:

  • Benchmarks (memory, latency) comparing uniform vs. mixed quantization on representative pipelines.
  • Qualitative comparisons of outputs.
  • Better-grounded heuristics for choosing which layers to keep in higher precision. Existing community guides give heuristics, but they're hit and miss.

Requested by Sayak Paul - Slack thread - Agent trace

Lingua principale
Python
Stelle
34.9k
Fork
7.5k
Merge medio
3g 16h
PR unite (30g)
65

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di huggingface/diffusers

Tutte le issue di huggingface/diffusers

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.