Support mixed-type quantization within a single quantization backend

未关闭
#15,320 0 条评论 0 个 reaction 已指派 1 人 在 GitHub 查看

维护者通常 1 天内回复

@sayakpaul 已经在做这个了。

开始于 2026年11月17日。

评估

这个 Issue 还没有评估数据。

描述

Motivation

Other tools like ComfyUI and Unsloth already support mixed-type quants: different layers or modules of a model are quantized with different schemes or precisions. Some layers stay in high precision, and others drop to lower bit-widths. This usually gets a better quality/memory trade-off than quantizing every layer the same way.

Most of the quantization backends Diffusers integrates with already support this natively. Diffusers should let users use that capability.

Proposal

  • Allow mixed quantization types within a single backend (for example, different torchao or bnb configs per module/layer pattern), when the underlying backend framework supports it.
  • Out of scope for now: mixing backends in one checkpoint (for example, some layers bnb and others torchao). Not all backends are feature-complete, so this would be messy. We also haven't seen such checkpoints in practice.

Open questions

  • API design: how users specify per-module/per-layer quant configs (pattern-based mapping, modules_to_not_convert-style extension, etc.).
  • Serialization and loading of mixed-quant checkpoints, plus compatibility with existing pre-quantized checkpoints (for example, from the ComfyUI / Unsloth ecosystems).
  • Which backends support this today, and what's missing in each.

Benchmarks / guidance

As part of this work, produce:

  • Benchmarks (memory, latency) comparing uniform vs. mixed quantization on representative pipelines.
  • Qualitative comparisons of outputs.
  • Better-grounded heuristics for choosing which layers to keep in higher precision. Existing community guides give heuristics, but they're hit and miss.

Requested by Sayak Paul - Slack thread - Agent trace

主要语言
Python
星标
34.9k
派生
7.5k
平均合并
3 天 16 小时
30 天内合并 PR
65

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

huggingface/diffusers 的其他 Issue

查看 huggingface/diffusers 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。