Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Support activation-only scale search for PyTorch AWQ calibration

Abierto
  • ¿Algo no está bien?
#2,674 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 2 días

@meenchen ya está trabajando en esto.

Desde el 6/10/2026.

  • #2675 de @thuiaaa — abierto

Evaluación

Este issue todavía no se ha evaluado.

Descripción

Detailed description of the requested feature

Please add an activation-only scale-search option to the PyTorch PTQ AWQ calibration algorithms (at least awq_lite, and therefore awq_full).

The current get_scale implementation searches per-input-channel scales proportional to act_scale ** alpha / weight_scale ** (1 - alpha):
https://github.com/NVIDIA/Model-Optimizer/blob/9f902aefd/modelopt/torch/quantization/model_calib.py#L1481-L1494

Equation (5) of the AWQ paper instead defines the search space as s = s_X ** alpha, where s_X is the mean activation magnitude per input channel:
https://arxiv.org/html/2306.00978v6

The paper authors' current implementation also uses activation-only scaling:
https://github.com/mit-han-lab/llm-awq/blob/main/awq/quantize/auto_scale.py#L119-L125

For historical context, the authors' initial repository commit included a weight term, which was removed in a later commit:
https://github.com/mit-han-lab/llm-awq/commit/02c1dd657d1e

AutoAWQ subsequently added activation-only scaling as an optional mode (duo_scaling=False) in PR #216 while retaining its existing weight-and-activation mode as the default:
https://github.com/casper-hansen/AutoAWQ/pull/216

That PR reported lower WikiText perplexity for Qwen 7B Chat with activation-only scaling (10.224 to 9.949), while the results for Mistral 7B Instruct and Llama 2 7B Chat were approximately unchanged or slightly mixed. This is evidence that the choice can matter by model, not a claim that activation-only scaling is universally better.

Could ModelOpt expose a configuration option for the paper's activation-only search space, alongside the current formula? A backward-compatible default would preserve existing quantization behavior. It would help users reproduce the paper's scale search and compare both methods on their target models.

Timeline

No fixed deadline. This would enable paper-aligned AWQ experiments and model-specific accuracy comparisons.

Describe alternatives you've considered

AutoAWQ exposes duo_scaling=False, but using a separate quantization library does not provide this option within ModelOpt's existing PTQ and export workflows. Replacing ModelOpt's current formula unconditionally could change existing results, so a configurable choice seems preferable.

Target hardware/use case

PyTorch PTQ for LLM weight-only INT4 AWQ (W4A16), including INT4_AWQ_CFG; the same option may also be relevant to other ModelOpt AWQ formats that use the shared scale search.

Lenguaje dominante
Python
Estrellas
5.4k
Forks
767
Merge medio
3 d 11 h
PR fusionados (30 d)
203

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/Model-Optimizer

Todos los issues de NVIDIA/Model-Optimizer

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.