Support mixed-type quantization within a single quantization backend
维护者通常 1 天内回复
@sayakpaul 已经在做这个了。
开始于 2026年11月17日。
评估
这个 Issue 还没有评估数据。
描述
Motivation
Other tools like ComfyUI and Unsloth already support mixed-type quants: different layers or modules of a model are quantized with different schemes or precisions. Some layers stay in high precision, and others drop to lower bit-widths. This usually gets a better quality/memory trade-off than quantizing every layer the same way.
Most of the quantization backends Diffusers integrates with already support this natively. Diffusers should let users use that capability.
Proposal
- Allow mixed quantization types within a single backend (for example, different torchao or bnb configs per module/layer pattern), when the underlying backend framework supports it.
- Out of scope for now: mixing backends in one checkpoint (for example, some layers bnb and others torchao). Not all backends are feature-complete, so this would be messy. We also haven't seen such checkpoints in practice.
Open questions
- API design: how users specify per-module/per-layer quant configs (pattern-based mapping,
modules_to_not_convert-style extension, etc.). - Serialization and loading of mixed-quant checkpoints, plus compatibility with existing pre-quantized checkpoints (for example, from the ComfyUI / Unsloth ecosystems).
- Which backends support this today, and what's missing in each.
Benchmarks / guidance
As part of this work, produce:
- Benchmarks (memory, latency) comparing uniform vs. mixed quantization on representative pipelines.
- Qualitative comparisons of outputs.
- Better-grounded heuristics for choosing which layers to keep in higher precision. Existing community guides give heuristics, but they're hit and miss.
Requested by Sayak Paul - Slack thread - Agent trace
- 主要语言
- Python
- 星标
- 34.9k
- 派生
- 7.5k
- 平均合并
- 3 天 16 小时
- 30 天内合并 PR
- 65
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
huggingface/diffusers 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 84/100
huggingface/diffusers#15363 ·
维护者通常 1 天内回复
-
`ZImageImg2ImgPipeline` / `ZImageInpaintPipeline`: explicit height/width different from the input image raises RuntimeError可能已有人在做 @chessing234 于 1 天前认领。 未关闭bug
难度 2/5 1-3 小时 新手友好度 70/100
huggingface/diffusers#15337 · 2 条评论 ·
维护者通常 1 天内回复
-
Multi-vector textual inversion uses only the first vector when the token has uppercase letters (transformers v5)可能已有人在做 @sunhouy 于 2 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
huggingface/diffusers#15313 ·
维护者通常 1 天内回复
-
TCDScheduler advances the generator on the final step of a strength-truncated custom schedule可能已有人在做 @SIDDARTHAREDDY8 于 5 天前认领。 未关闭bug
难度 2/5 1-3 小时 新手友好度 74/100
huggingface/diffusers#15288 · 2 条评论 ·
维护者通常 1 天内回复
-
unload_textual_inversion(token) leaves the extra tokens of a multi-vector embedding behind, so it can't be loaded again可能已有人在做 @sohailshk 于 11 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 72/100
huggingface/diffusers#15211 · 3 条评论 ·
维护者通常 1 天内回复
查看 huggingface/diffusers 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 73/100
维护者通常 1 天内回复
-
Unused loot bag group names left in dreamcraft en_US.lang可能已有人在做 关联的 PR 仍在进行中或已合并。 未关闭Status: Triage Suggestion
难度 1/5 1 小时以内 新手友好度 90/100
GTNewHorizons/GT-New-Horizons-Modpack#27789 ·
维护者通常 1 天内回复
-
bug frontend sev-5
难度 2/5 1-3 小时 新手友好度 83/100
-
core good first issue
难度 2/5 1-3 小时 新手友好度 65/100
-
b4b code health next
难度 2/5 1-3 小时 新手友好度 68/100
维护者通常 1 天内回复