Ultralytics YOLO INT8 build failure of TensorRT 10.16 (MyelinCheckException) on RTX PRO 6000 Blackwell, regression from 10.15

未关闭
#4,906 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 6 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
12/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
python

调研方向

Start from the reproducer in build_int8_implicit.py, which parses yolo26n.onnx and sets INT8 and FP16 flags with static dynamic ranges, and rerun it on tensorrt-cu13 10.15.1.29 and 10.16.1.11 to confirm the split. The failing node is the Quantize op named in the log, and the error comes from the closed-source Myelin code generator, so the fix is not in the open repository. Done means a minimal ONNX reproducer that isolates the failing op, or a confirmed workaround, reported to NVIDIA.

由索引模型根据 Issue 内容生成。

描述

Module:Engine Build

Description

Building an implicit INT8 engine (BuilderFlag.INT8 + BuilderFlag.FP16, with per-tensor dynamic ranges or an IInt8Calibrator) from Ultralytics YOLO YOLO26n and YOLO11n ONNX models fails on TensorRT 10.16.1.11 with an internal Myelin error. The same script succeeds on TensorRT 10.15.1.29, so this looks like a regression in 10.16:

Internal Error: MyelinCheckException: utils.cpp:4054: CHECK(0) failed.
[TRT] [E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [myelin_graph.h:1250: attachExceptionMsgToGraph] MyelinCheckException: utils.cpp:4054: CHECK(0) failed.  In compileGraph at /_src/optimizer/myelin/codeGenerator.cpp:1783
[TRT] [E] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[Quantize 0...output0_castOut]}. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4265)
Model TensorRT 10.15.1.29 TensorRT 10.16.1.11
YOLO26n builds fails
YOLO11n builds fails

FP16 builds of the same models succeed on 10.16.1.11. The Ultralytics export path (yolo export model=yolo26n.pt format=engine quantize=8 data=coco128.yaml) uses an IInt8Calibrator with MINMAX_CALIBRATION on TensorRT 10.x (source). On 10.16.1.11 it fails with the same error for YOLO26n and YOLO26s.

TensorRT 11 removed the implicit INT8 calibrator and DLA, so Ultralytics users who need either still build with TensorRT 10.x. See the Ultralytics TensorRT integration guide.

Environment

TensorRT Version: 10.16.1.11 (fails), 10.15.1.29 (works) (tensorrt-cu13 pip wheels)

NVIDIA GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition (not tested on other GPUs)

NVIDIA Driver Version: 595.99.02

CUDA Version: 13.2

CUDNN Version: N/A

Operating System: Ubuntu 24.04.5 LTS

Python Version (if applicable): 3.12.3

PyTorch Version (if applicable): 2.15.0 (CPU, used only for the ONNX export)

Other: ultralytics 8.4.196, onnx 1.21.0, ONNX opset 18

Baremetal or Container (if so, version): Baremetal

Relevant Files

Model link: yolo26n.pt and yolo11n.pt. The ONNX files are generated by the export command below.

Steps To Reproduce

Commands or scripts:

pip install ultralytics==8.4.196 tensorrt-cu13==10.16.1.11
yolo export model=yolo26n.pt format=onnx imgsz=640  # -> yolo26n.onnx
python build_int8_implicit.py

build_int8_implicit.py:

# Implicit INT8 (calibration-style) build of the FP32 ONNX with INT8 + FP16 builder flags.
# Static dynamic ranges stand in for a calibrator; the IInt8Calibrator path in Ultralytics fails identically.
import tensorrt as trt

logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)
network = builder.create_network(0)
config = builder.create_builder_config()
assert trt.OnnxParser(network, logger).parse_from_file("yolo26n.onnx")
config.set_flag(trt.BuilderFlag.INT8)
config.set_flag(trt.BuilderFlag.FP16)
for i in range(network.num_inputs):
    network.get_input(i).dynamic_range = (0.0, 1.0)
for i in range(network.num_layers):
    layer = network.get_layer(i)
    for j in range(layer.num_outputs):
        layer.get_output(j).dynamic_range = (-8.0, 8.0)
plan = builder.build_serialized_network(network, config)
print(f"TensorRT {trt.__version__}: INT8 build {'succeeded' if plan else 'FAILED'}")

Output:

TensorRT 10.16.1.11: INT8 build FAILED
TensorRT 10.15.1.29: INT8 build succeeded

Have you tried the latest release?: 10.16.1.11 is the latest 10.x release. TensorRT 11.x no longer supports implicit INT8 builds.

Can this model run on other frameworks?: Yes. The ONNX models run in ONNX Runtime, and they build and run on TensorRT 10.15.1.29 (INT8) and 10.16.1.11 (FP16).

主要语言
C++
星标
13.5k
派生
2.4k
平均合并
6 天 21 小时
30 天内合并 PR
3

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

NVIDIA/TensorRT 的其他 Issue

查看 NVIDIA/TensorRT 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。