Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

# [ONNX][Autotune] Integrated quantization does not preserve AutoTune Q/DQ placement on ViT

Aperta
  • Qualcosa non va?
#2,123 1 commento 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@ajrasane ci sta già lavorando.

Dal 17/8/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

bug

[ONNX][Autotune] Integrated quantization does not preserve AutoTune Q/DQ placement on ViT

Description

When using ONNX Integrated AutoTune with:

python -m modelopt.onnx.quantization ... --autotune

I found that the Q/DQ placement selected during the AutoTune search is not preserved in the final calibrated INT8 ONNX model.

This also causes a noticeable TensorRT performance regression.

For autotune_schemes_per_region=50:

Model QPS Speedup P50
FP16 baseline 744.41 1.00x 1.34 ms
Standard INT8 PTQ 1019.40 1.37x 0.98 ms
Integrated AutoTune optimized_final.onnx 1045.54 1.41x 0.95 ms
Integrated final calibrated model 915.37 1.23x 1.09 ms

At the consumer-input edge level:

SAME          = 86
AUTOTUNE ONLY = 48
FINAL ONLY    = 0

So all activation boundaries in the final model are already present in the AutoTune model, but 48 AutoTune-selected boundaries disappear in the final calibrated model.

The missing boundaries follow a regular pattern:

12 x Cast -> Sqrt
12 x Div  -> Sqrt_1
12 x Div  -> Sqrt_2
11 x fc1/Add -> Gelu
 1 x Add -> final LayerNormalization

Because each Div Q/DQ is shared by two Sqrt consumers, these 48 missing consumer edges correspond exactly to the 36 missing activation Q/DQ pairs:

122 - 86 = 36

For example:

/blocks/blocks.0/attn/Cast_output_0_cast_to_fp16
    -> /blocks/blocks.0/attn/Sqrt.input[0]

/blocks/blocks.0/attn/Div_output_0

-> /blocks/blocks.0/attn/Sqrt_1.input[0]


/blocks/blocks.0/attn/Div_output_0

-> /blocks/blocks.0/attn/Sqrt_2.input[0]


/blocks/blocks.0/mlp/fc1/Add_output_0

-> /blocks/blocks.0/mlp/act/Gelu.input[0]

Expected behavior

I expected Integrated AutoTune to preserve the Q/DQ placement selected by AutoTune while using calibration data to compute the final quantization scales.

This also appears consistent with the existing integration test:

tests/gpu/onnx/quantization/test_autotune_quantization_integration.py

which compares DQ-fed (node_name, input_index) sets between standalone AutoTune and Integrated AutoTune and asserts that they are equal.

Could you please check whether the final Integrated AutoTune model is expected to preserve the exact AutoTune Q/DQ placement, and if so, fix the topology preservation for this ViT case?

I can provide the ONNX models, calibration data, topology comparison script, and full AutoTune/TensorRT logs if needed.

Lingua principale
Python
Stelle
5.2k
Fork
736
Merge medio
4g 2h
PR unite (30g)
160

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/Model-Optimizer

Tutte le issue di NVIDIA/Model-Optimizer

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.