[TensorRT 11.x][Blackwell SM120] Internal DDS compiler assertion when IPluginV3 scalar output controls dynamic ONNX Slice

未关闭
#4,878 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 6 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
5/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
cpp, shell

调研方向

The failing assertion is in convertExplicitDDSPluginToImplicit.cpp at line 149, inside the TensorRT myelin optimizer, and the report reproduces it only with a proprietary IPluginV3 plugin (SparseConvolution) feeding two dynamic ONNX Slice ops. Reproducing needs a Blackwell SM120 GPU, TensorRT 11.x, and a substitute plugin that declares a scalar INT32 output without calling IExprBuilder::declareSizeTensor(). Done would mean understanding why the DDS output and size-tensor bookkeeping disagree, which is maintainer-level work on the closed-source optimizer.

由索引模型根据 Issue 内容生成。

描述

Module:Engine Build

Description

We are experiencing what seems an architecture and version-dependent TensorRT engine compilation failure involving a custom IPluginV3 and downstream data-dependent ONNX Slice operations.

The compilation from onnx to engine is done with trtexec.

The behavior differs depending on the GPU architecture and TensorRT version:

  • TensorRT 10.14: Successfully builds the engine on Blackwell SM120.
  • TensorRT 11.1 and 11.4: Fail during engine compilation on Blackwell SM120.
  • TensorRT 11.1 and 11.4: Successfully build the same network on Ampere SM86 and Ada SM89 GPUs (only tested with 11.1).

The original model and custom plugin implementation are proprietary and cannot be shared.

Environment

GPU Compute capability TensorRT 10.14 TensorRT 11.1 TensorRT 11.4
NVIDIA RTX A3000 Laptop SM86 PASS PASS PASS
NVIDIA RTX 4090 SM89 PASS PASS NOT TESTED
NVIDIA RTX PRO 4000 Blackwell SFF SM120 PASS FAIL FAIL

Additional environment information:

  • ONNX opset: 17
  • Custom plugin interface: IPluginV3
  • Engine building: trtexec
  • Plugin loading: --staticPlugins
  • Optimization profile: dynamic input dimensions

Note that in 10.14 we were using implicit trtexec fp32 -> fp16 conversion

Exact Error

The engine build fails with the following internal assertion:

[E] Error[2]:
[convertExplicitDDSPluginToImplicit.cpp::
convertExplicitDDSPluginToImplicit::149]

Error Code 2: Internal Error

Assertion
nodeIdxToDDSOutputIndices.count(i) ==
nodeIdxToSizeTensors.contains(i)
failed.

In convertExplicitDDSPluginToImplicit at
/_src/optimizer/myelin/rewrite/
convertExplicitDDSPluginToImplicit.cpp:149

The error occurs during engine construction, not during inference.

Steps To Reproduce

The relevant graph section consists of a custom IPluginV3 (SparseConvolution) producing fixed-capacity tensors and a scalar indicating how many entries are valid.

The plugin exposes three outputs:

Output Data type Shape Description
0 FP16 [CAPACITY, CHANNELS] Feature data
1 INT32 [CAPACITY, 4] Associated indices
2 INT32 [] Runtime number of valid entries

The scalar is subsequently used to determine the output dimension of two ONNX Slice operations.

Conceptually, the operation is equivalent to:

valid_features = features[:valid_count]
valid_indices = indices[:valid_count]

The count is a runtime value produced by the custom plugin.
The plugin currently declares the count as an ordinary INT32 scalar output and does not explicitly call IExprBuilder::declareSizeTensor().

We would like to confirm whether this pattern is supported by TensorRT 11.x. or a regression on TensorRT 11. on Blackwell path

主要语言
C++
星标
13.5k
派生
2.4k
平均合并
6 天 21 小时
30 天内合并 PR
3

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

NVIDIA/TensorRT 的其他 Issue

查看 NVIDIA/TensorRT 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。