Ultralytics YOLO INT8 build failure of TensorRT 10.16 (MyelinCheckException) on RTX PRO 6000 Blackwell, regression from 10.15

Aperta
#4,906 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 6 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
12/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python

Direzione di ricerca

Start from the reproducer in build_int8_implicit.py, which parses yolo26n.onnx and sets INT8 and FP16 flags with static dynamic ranges, and rerun it on tensorrt-cu13 10.15.1.29 and 10.16.1.11 to confirm the split. The failing node is the Quantize op named in the log, and the error comes from the closed-source Myelin code generator, so the fix is not in the open repository. Done means a minimal ONNX reproducer that isolates the failing op, or a confirmed workaround, reported to NVIDIA.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Module:Engine Build

Description

Building an implicit INT8 engine (BuilderFlag.INT8 + BuilderFlag.FP16, with per-tensor dynamic ranges or an IInt8Calibrator) from Ultralytics YOLO YOLO26n and YOLO11n ONNX models fails on TensorRT 10.16.1.11 with an internal Myelin error. The same script succeeds on TensorRT 10.15.1.29, so this looks like a regression in 10.16:

Internal Error: MyelinCheckException: utils.cpp:4054: CHECK(0) failed.
[TRT] [E] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [myelin_graph.h:1250: attachExceptionMsgToGraph] MyelinCheckException: utils.cpp:4054: CHECK(0) failed.  In compileGraph at /_src/optimizer/myelin/codeGenerator.cpp:1783
[TRT] [E] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[Quantize 0...output0_castOut]}. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4265)
Model TensorRT 10.15.1.29 TensorRT 10.16.1.11
YOLO26n builds fails
YOLO11n builds fails

FP16 builds of the same models succeed on 10.16.1.11. The Ultralytics export path (yolo export model=yolo26n.pt format=engine quantize=8 data=coco128.yaml) uses an IInt8Calibrator with MINMAX_CALIBRATION on TensorRT 10.x (source). On 10.16.1.11 it fails with the same error for YOLO26n and YOLO26s.

TensorRT 11 removed the implicit INT8 calibrator and DLA, so Ultralytics users who need either still build with TensorRT 10.x. See the Ultralytics TensorRT integration guide.

Environment

TensorRT Version: 10.16.1.11 (fails), 10.15.1.29 (works) (tensorrt-cu13 pip wheels)

NVIDIA GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition (not tested on other GPUs)

NVIDIA Driver Version: 595.99.02

CUDA Version: 13.2

CUDNN Version: N/A

Operating System: Ubuntu 24.04.5 LTS

Python Version (if applicable): 3.12.3

PyTorch Version (if applicable): 2.15.0 (CPU, used only for the ONNX export)

Other: ultralytics 8.4.196, onnx 1.21.0, ONNX opset 18

Baremetal or Container (if so, version): Baremetal

Relevant Files

Model link: yolo26n.pt and yolo11n.pt. The ONNX files are generated by the export command below.

Steps To Reproduce

Commands or scripts:

pip install ultralytics==8.4.196 tensorrt-cu13==10.16.1.11
yolo export model=yolo26n.pt format=onnx imgsz=640  # -> yolo26n.onnx
python build_int8_implicit.py

build_int8_implicit.py:

# Implicit INT8 (calibration-style) build of the FP32 ONNX with INT8 + FP16 builder flags.
# Static dynamic ranges stand in for a calibrator; the IInt8Calibrator path in Ultralytics fails identically.
import tensorrt as trt

logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)
network = builder.create_network(0)
config = builder.create_builder_config()
assert trt.OnnxParser(network, logger).parse_from_file("yolo26n.onnx")
config.set_flag(trt.BuilderFlag.INT8)
config.set_flag(trt.BuilderFlag.FP16)
for i in range(network.num_inputs):
    network.get_input(i).dynamic_range = (0.0, 1.0)
for i in range(network.num_layers):
    layer = network.get_layer(i)
    for j in range(layer.num_outputs):
        layer.get_output(j).dynamic_range = (-8.0, 8.0)
plan = builder.build_serialized_network(network, config)
print(f"TensorRT {trt.__version__}: INT8 build {'succeeded' if plan else 'FAILED'}")

Output:

TensorRT 10.16.1.11: INT8 build FAILED
TensorRT 10.15.1.29: INT8 build succeeded

Have you tried the latest release?: 10.16.1.11 is the latest 10.x release. TensorRT 11.x no longer supports implicit INT8 builds.

Can this model run on other frameworks?: Yes. The ONNX models run in ONNX Runtime, and they build and run on TensorRT 10.15.1.29 (INT8) and 10.16.1.11 (FP16).

Lingua principale
C++
Stelle
13.5k
Fork
2.4k
Merge medio
6g 21h
PR unite (30g)
3

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/TensorRT

Tutte le issue di NVIDIA/TensorRT

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.