Myelin "Could not infer output types for operation: dequantize" on SM 8.7 (Orin); identical engine builds on SM 11.0 (Thor) with the same TensorRT 10.16.2.10

未关闭
#4,832 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 6 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
cpp, python

调研方向

首先在 Orin 上使用 trtexec --onnx=quantized_int8.onnx --saveEngine=int8.engine --stronglyTyped 重现该失败,然后在 Thor 上比较相同的 artifact。检查 type.cpp:186 中的类型推断位置,以及 myelin/codeGenerator.cpp:1795 所引用的代码生成路径。完成的标准是 strongly typed INT8 engine 能够在 Orin 上成功构建,同时不削弱部署契约。

由索引模型根据 Issue 内容生成。

描述

Module:Quantization

Summary

Myelin fails to compile a dequantize operation on SM 8.7 (Orin) that compiles successfully on SM 11.0 (Thor), with identical TensorRT 10.16.2.10, identical L4T R39.2, and the byte-identical ONNX file. The two devices differ only in GPU architecture.

Environment

Orin Thor
TensorRT 10.16.2.10-1+cuda13.2 (trtexec v101602) identical
L4T R39 revision 2.0 identical
GPU arch SM 8.7 (Ampere) SM 11.0 (Blackwell)
Result ❌ build fails ✅ builds in 123 s

Producer: NVIDIA ModelOpt 0.44.0 (modelopt.onnx.quantization), INT8 PTQ, entropy calibration, 320 real-image samples. Consumer: trtexec --stronglyTyped.

Reproduction

trtexec --onnx=quantized_int8.onnx --saveEngine=int8.engine --stronglyTyped

Failure

Every tactic is skipped with:

Error Code 9: ... [type.cpp:186: infer_type] Could not infer output types for operation:
291: dequantize: Dequantized input0-(f32[1,16,512,512]) | input0 reshape out_transpose_out
reshape out slice_U28 out-(i8[]), MyelinPW(42188) - Input Dequantization 0 scale
weightsHalf-0.35498H:(f16[]) ... No matching rules found for input operand types
In compileGraph at .../myelin/codeGenerator.cpp:1795

The operand mix is f32 activation + i8 slice + f16 scale. The Ampere codegen path appears to have no rule for that combination; the Blackwell path does.

What this is not

Three different quantization scopes were produced, giving three distinct artifacts. All three fail on Orin with a byte-identical error — same node 291, same MyelinPW(42188), same 0.35498 scale:

Scope Excluded op types ONNX sha256 (16) Orin
plain INT8 none fcf873e63e0a951c ❌
head-aware Sigmoid, ReduceMax, Flatten, TopK, Gather, GatherElements 03d2ab3115dd4de5 ❌
head + stem above + Reshape, Transpose, Slice 540fa615ced07025 ❌

The error not moving while the graph does rules out quantization scope as the cause, and Thor compiling the identical file rules out the model and the calibration. That leaves architecture-specific codegen.

Impact

Strongly-typed INT8 is unavailable on Orin for this detector family. The documented alternative — weakly-typed --int8 --fp16 — permits silent downcast, which our deployment contract forbids.

主要语言
C++
星标
13.5k
派生
2.4k
平均合并
6 天 21 小时
30 天内合并 PR
3

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

NVIDIA/TensorRT 的其他 Issue

查看 NVIDIA/TensorRT 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。