Myelin exec_with_offset failure at run time with builder_optimization_level=4 for part of a dynamic-batch profile (TensorRT 10.16, H100)

オープン
#4,898 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 6 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
30/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python

調査の方向性

Start by reproducing the failure with the supplied ONNX model, TensorRT 10.16, and H100, probing batch sizes 146–256 in profile 1 at builder optimization level 4; compare with level 3. The report does not include a minimal plugin-free reproducer or kernel-level evidence, so first isolate whether the custom plugin is involved and inspect the failing tactic. Done means identifying a reproducible cause or confirming a TensorRT limitation and documenting the result.

索引モデルが issue の本文から書いたものです。

説明

Description

With builder_optimization_level = 4, an FP16 engine with three optimisation profiles builds fine but fails at run time for part of the middle profile. enqueueV3 returns false and logs:

[E] IExecutionContext::enqueueV3: Error Code 1: Myelin ([cask.cpp:1433: exec_with_offset] Platform (Cuda) error In executeMyelinGraph at /_src/runtime/myelin/runner.cpp:878)

The same ONNX built at the default level 3 runs every batch size. At level 4 the engine is 4–10% faster where it does run.

What fails

One dynamic batch dimension, 3 optimisation profiles (min,opt,max): (1,16,32), (33,128,256), (257,512,1024). We ran the engine once at every batch size 1–1024, synchronising after each run, in two ways: through the per-profile execution contexts, and through a fresh fixed-shape context per batch (what CUDA-graph capture uses). Two variants of the network (different input sizes) behave the same:

Profile 0 (1–32) Profile 1 (33–256) Profile 2 (257–1,024)
Variant A 0 / 32 fail 110 / 224 fail, B = 147–256 0 / 768 fail
Variant B 0 / 32 fail 106 / 224 fail, B = 151–256 0 / 768 fail
  • Eager and bound-context runs fail on the same batch sizes, so CUDA-graph capture is not the cause.
  • The error is not sticky: the CUDA context survives each failure and later batch sizes still run.
  • Same ONNX at level 3: 0 failures across 1–1024.

Guess at the cause (not confirmed)

Failures start between B = 146 and B = 147 in a profile with opt shape 128. That is consistent with a launch dimension of about B × 448 crossing CUDA's 65,535 grid limit, but we have not confirmed it at the kernel level. It looks as if the level-4 tactic chosen for profile 1 is not valid for the whole profile range.

Environment

  • TensorRT 10.16 (Python wheel)
  • NVIDIA H100 80 GB (SM90)
  • Driver and CUDA runtime versions were not recorded for these runs.
  • The network also contains a custom plugin (QDP API, ahead-of-time compiled). We have not tried to reproduce the failure without it, so it may be involved. A minimal plugin-free reproducer is not available yet.

Workaround

Serving batches 1–256 from a level-3 engine and 257–1,024 from the level-4 engine avoids the failing range.

Request

Is this a known limitation of level-4 tactic selection for dynamic profiles? I can run a per-batch-size probe on a newer build if that helps.

主要言語
C++
スター
13.5k
フォーク
2.4k
平均マージ
6日 21時間
マージ済み PR(30日)
3

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/TensorRT のほかの issue

NVIDIA/TensorRT の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。