Build failure (Myelin layouts.cpp:2180) or wrong plugin input of TensorRT 10.16 when an IPluginV3 with tactics reads a kHWC8 network output, on Jetson Thor
メンテナーはふだん 6 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 8/100
- issue の種類
- バグ
- 明瞭さ
- 明確に書かれている
- 活発さ
- 活発
- 技術スタック
- cpp
調査の方向性
The standalone repro is repro.cu with CMakeLists.txt. Build it and run ./repro --up2 --tactics --out-hwc8 to confirm the MyelinCheckException at layouts.cpp:2180; that file is inside TensorRT, not this repo. Done means the engine builds and plugin_out matches conv_out, as on TensorRT 10.3 and 10.16.0 x86, and the fix is in TensorRT itself, so a newcomer cannot complete it here.
索引モデルが issue の本文から書いたものです。
説明
Description
On Jetson AGX Thor with TensorRT 10.16.2, a network that contains an IPluginV3 offering custom tactics fails to build when the tensor the plugin reads is also a network output restricted to kHWC8. Either condition alone builds and runs exactly. The same repro builds and is exact with TensorRT 10.3 on Jetson AGX Orin and with TensorRT 10.16.0 on x86.
Internal Error: MyelinCheckException: layouts.cpp:2180: CHECK(always(sym_eql(dims[i + 1], tiles[idx]))) failed.
[trt] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [myelin_graph.h:1250: attachExceptionMsgToGraph] MyelinCheckException: layouts.cpp:2180: CHECK(always(sym_eql(dims[i + 1], tiles[idx]))) failed. In compileGraph at /_src/optimizer/myelin/codeGenerator.cpp:1795
[trt] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[(Unnamed Layer* 0) [Convolution]...(Unnamed Layer* 2) [PluginV3_V3ONE]]}. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4265)
Expected: the engine builds, as with TensorRT 10.3 on Orin and TensorRT 10.16.0 on x86.
Workaround: don't expose custom tactics from such plugins on TensorRT 10.16.
Environment
TensorRT Version: 10.16.2.10 (tensorrt 10.16.2.10-1+cuda13.2, from JetPack)
NVIDIA GPU: Jetson AGX Thor (Thor developer kit)
NVIDIA Driver Version: 595.78
CUDA Version: 13.2 (nvcc 13.2.r13.2)
CUDNN Version: 9.20.0.46 (not used by the repro)
Operating System: Ubuntu 24.04.4 LTS, JetPack 7.2 (L4T R39.2.0)
Python Version (if applicable): N/A (C++ only)
Tensorflow Version (if applicable): N/A
PyTorch Version (if applicable): N/A
Baremetal or Container (if so, version): baremetal
Not affected (same repro, exact output):
- Jetson AGX Orin, JetPack 6 (L4T R36.5), TensorRT 10.3.0.30, CUDA 12.6.
- x86_64, RTX 2080 (Turing), TensorRT 10.16.0.
Relevant Files
Model link: none needed. The network is built with the network-definition API, and the standalone repro is below: repro.cu + CMakeLists.txt, with no dependency besides CUDA and TensorRT.
The network is strongly typed, FP16, with a default builder configuration:
x (2, 64, 128, 224) FP16 -> conv 3x3 + ReLU -> conv_out (network output)
\-> PassHWC8 plugin -> plugin_out (network output, kHWC8)
PassHWC8 is an IPluginV3 (IPluginV3OneCore / IPluginV3OneBuild / IPluginV3OneRuntime) that supports only DataType::kHALF + TensorFormat::kHWC8 on its input and output. It copies its input to its output, so plugin_out is checked exactly against conv_out on the host. With --up2, it instead writes a nearest x2 upsample of the first 16 channels.
CMakeLists.txt
cmake_minimum_required(VERSION 3.18)
if(NOT DEFINED CMAKE_CUDA_COMPILER AND EXISTS /usr/local/cuda/bin/nvcc)
set(CMAKE_CUDA_COMPILER /usr/local/cuda/bin/nvcc)
endif()
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.24)
set(CMAKE_CUDA_ARCHITECTURES native) # the GPU of the machine that builds it
else()
set(CMAKE_CUDA_ARCHITECTURES 75 87) # pass -DCMAKE_CUDA_ARCHITECTURES=<sm> for other GPUs
endif()
endif()
project(trt_plugin_hwc8_repro LANGUAGES CXX CUDA)
set(CMAKE_CXX_STANDARD 17)
find_package(CUDAToolkit REQUIRED)
find_path(TRT_INCLUDE NvInfer.h
HINTS ${TRT_ROOT}/include /usr/include/aarch64-linux-gnu /usr/include/x86_64-linux-gnu ${CUDAToolkit_INCLUDE_DIRS})
find_library(TRT_LIB nvinfer
HINTS ${TRT_ROOT}/lib /usr/lib/aarch64-linux-gnu /usr/lib/x86_64-linux-gnu ${CUDAToolkit_LIBRARY_DIR})
add_executable(repro repro.cu)
target_include_directories(repro PRIVATE ${TRT_INCLUDE})
target_link_libraries(repro PRIVATE ${TRT_LIB} CUDA::cudart)
repro.cu
// Standalone repro: an IPluginV3 (FP16, TensorFormat::kHWC8 in and out) placed after a convolution whose output the
// plugin reads. The plugin is either a pass-through copy, or (--up2) a nearest x2 upsample keeping the first 16
// channels, so its output can be checked exactly against conv_out on the host.
//
// ./repro [--up2] [--tactics] [--out-hwc8] [--consumer] [--resize] [--slice] [--trunk] [--cat] [--cat-producer]
// --up2 : plugin output (N, 16, 2H, 2W) = nearest x2 of the first 16 channels of its input
// --tactics : the plugin offers 8 tactics (getValidTactics); every tactic computes the same thing
// --out-hwc8 : conv_out is also a network output restricted to kHWC8 (the layout the plugin asks for)
// --consumer : conv_out also feeds a second convolution (another output)
// --resize : conv_out also feeds an IResizeLayer (linear x2, another output)
// --slice : conv_out's first batch item also feeds a 1x1 conv (another output, kHWC8)
// --trunk : the plugin output is convolved, concatenated with another conv's output and convolved again
// --cat : the plugin output is concatenated directly with another conv's output, then convolved
// --cat-producer : conv_out's own input comes from a concat -> conv chain
//
// Prints the build status and the number of plugin_out elements that differ from what conv_out implies.
#include <NvInfer.h>
#include <cuda_fp16.h>
#include <cuda_runtime.h>
#include <cmath>
#include <cstdio>
#include <cstring>
#include <memory>
#include <random>
#include <string>
#include <vector>
using namespace nvinfer1;
// out[n][2h + a][2w + b][k] = in[n][h][w][k], k < K (both channel-last, channel strides Ci / K)
static __global__ void up2Kernel(const __half* in, __half* out, int N, int Ci, int H, int W, int K) {
const long long i = blockIdx.x * static_cast<long long>(blockDim.x) + threadIdx.x;
const long long total = static_cast<long long>(N) * 2 * H * 2 * W * K;
if (i >= total) return;
const int k = i % K;
long long t = i / K;
const int x = t % (2 * W);
t /= 2 * W;
const int y = t % (2 * H);
const int n = static_cast<int>(t / (2 * H));
out[i] = in[((static_cast<long long>(n) * H + y / 2) * W + x / 2) * Ci + k];
}
namespace {
class Logger : public ILogger {
void log(Severity s, const char* msg) noexcept override {
if (s <= Severity::kWARNING) std::printf("[trt] %s\n", msg);
}
} gLogger;
#define CK(x) \
do { \
cudaError_t e_ = (x); \
if (e_ != cudaSuccess) { \
std::printf("%s: %s\n", #x, cudaGetErrorString(e_)); \
return 1; \
} \
} while (0)
constexpr char kName[] = "PassHWC8", kVersion[] = "1", kNamespace[] = "repro";
bool gUp2 = false, gTactics = false;
constexpr int kUpK = 16;
// out = in, FP16 kHWC8 (channels padded to 8: N * H * W * roundup(C, 8) elements)
class PassPlugin : public IPluginV3, public IPluginV3OneCore, public IPluginV3OneBuild, public IPluginV3OneRuntime {
public:
IPluginCapability* getCapabilityInterface(PluginCapabilityType t) noexcept override {
if (t == PluginCapabilityType::kCORE) return static_cast<IPluginV3OneCore*>(this);
if (t == PluginCapabilityType::kBUILD) return static_cast<IPluginV3OneBuild*>(this);
return static_cast<IPluginV3OneRuntime*>(this);
}
IPluginV3* clone() noexcept override { return new PassPlugin(*this); }
AsciiChar const* getPluginName() const noexcept override { return kName; }
AsciiChar const* getPluginVersion() const noexcept override { return kVersion; }
AsciiChar const* getPluginNamespace() const noexcept override { return kNamespace; }
int32_t getNbOutputs() const noexcept override { return 1; }
int32_t getOutputDataTypes(DataType* o, int32_t, DataType const* in, int32_t) const noexcept override {
o[0] = in[0];
return 0;
}
int32_t getOutputShapes(DimsExprs const* in, int32_t, DimsExprs const*, int32_t, DimsExprs* out, int32_t,
IExprBuilder& eb) noexcept override {
out[0] = in[0];
if (gUp2) {
out[0].d[1] = eb.constant(kUpK);
out[0].d[2] = eb.operation(DimensionOperation::kPROD, *in[0].d[2], *eb.constant(2));
out[0].d[3] = eb.operation(DimensionOperation::kPROD, *in[0].d[3], *eb.constant(2));
}
return 0;
}
int32_t getNbTactics() noexcept override { return gTactics ? 8 : 0; }
int32_t getValidTactics(int32_t* t, int32_t n) noexcept override {
for (int32_t i = 0; i < n; ++i) t[i] = i + 1;
return 0;
}
int32_t setTactic(int32_t) noexcept override { return 0; }
bool supportsFormatCombination(int32_t pos, DynamicPluginTensorDesc const* io, int32_t, int32_t) noexcept override {
return io[pos].desc.type == DataType::kHALF && io[pos].desc.format == TensorFormat::kHWC8;
}
int32_t configurePlugin(DynamicPluginTensorDesc const*, int32_t, DynamicPluginTensorDesc const*,
int32_t) noexcept override {
return 0;
}
size_t getWorkspaceSize(DynamicPluginTensorDesc const*, int32_t, DynamicPluginTensorDesc const*,
int32_t) const noexcept override {
return 0;
}
int32_t onShapeChange(PluginTensorDesc const*, int32_t, PluginTensorDesc const*, int32_t) noexcept override { return 0; }
int32_t enqueue(PluginTensorDesc const* inDesc, PluginTensorDesc const*, void const* const* inputs, void* const* outputs,
void*, cudaStream_t stream) noexcept override {
const Dims& d = inDesc[0].dims;
const int Ci = static_cast<int>((d.d[1] + 7) / 8 * 8);
if (gUp2) {
const long long total = d.d[0] * 4 * d.d[2] * d.d[3] * kUpK;
up2Kernel<<<static_cast<unsigned>((total + 255) / 256), 256, 0, stream>>>(
static_cast<const __half*>(inputs[0]), static_cast<__half*>(outputs[0]), static_cast<int>(d.d[0]), Ci,
static_cast<int>(d.d[2]), static_cast<int>(d.d[3]), kUpK);
return cudaPeekAtLastError() == cudaSuccess ? 0 : 1;
}
const size_t n = static_cast<size_t>(d.d[0]) * Ci * d.d[2] * d.d[3];
return cudaMemcpyAsync(outputs[0], inputs[0], n * sizeof(__half), cudaMemcpyDeviceToDevice, stream) == cudaSuccess
? 0
: 1;
}
IPluginV3* attachToContext(IPluginResourceContext*) noexcept override { return clone(); }
PluginFieldCollection const* getFieldsToSerialize() noexcept override {
fc_.nbFields = 0;
fc_.fields = nullptr;
return &fc_;
}
private:
PluginFieldCollection fc_{};
};
class PassCreator : public IPluginCreatorV3One {
public:
IPluginV3* createPlugin(AsciiChar const*, PluginFieldCollection const*, TensorRTPhase) noexcept override {
return new PassPlugin();
}
PluginFieldCollection const* getFieldNames() noexcept override { return &fc_; }
AsciiChar const* getPluginName() const noexcept override { return kName; }
AsciiChar const* getPluginVersion() const noexcept override { return kVersion; }
AsciiChar const* getPluginNamespace() const noexcept override { return kNamespace; }
private:
PluginFieldCollection fc_{};
};
// FP16 tensor of format f (kLINEAR NCHW, or kHWC8 = NHWC with C padded to 8) -> NCHW floats
std::vector<float> toNCHW(const std::vector<__half>& v, TensorFormat f, int N, int C, int H, int W) {
std::vector<float> o(static_cast<size_t>(N) * C * H * W);
const int Cp = (C + 7) / 8 * 8;
for (int n = 0; n < N; ++n)
for (int c = 0; c < C; ++c)
for (int h = 0; h < H; ++h)
for (int w = 0; w < W; ++w) {
const size_t src = f == TensorFormat::kHWC8 ? ((static_cast<size_t>(n) * H + h) * W + w) * Cp + c
: ((static_cast<size_t>(n) * C + c) * H + h) * W + w;
o[((static_cast<size_t>(n) * C + c) * H + h) * W + w] = __half2float(v[src]);
}
return o;
}
std::vector<__half> randomHalf(size_t n, float scale, std::mt19937& g) {
std::uniform_real_distribution<float> u(-scale, scale);
std::vector<__half> v(n);
for (auto& x : v) x = __float2half(u(g));
return v;
}
} // namespace
int main(int argc, char** argv) {
bool outHwc8 = false, consumer = false, trunk = false, resize = false, slice = false, cat = false, catProducer = false;
for (int i = 1; i < argc; ++i) {
const std::string a = argv[i];
outHwc8 |= a == "--out-hwc8";
consumer |= a == "--consumer";
trunk |= a == "--trunk";
resize |= a == "--resize";
slice |= a == "--slice";
cat |= a == "--cat";
catProducer |= a == "--cat-producer";
gUp2 |= a == "--up2";
gTactics |= a == "--tactics";
}
static PassCreator creator;
getPluginRegistry()->registerCreator(creator, kNamespace);
std::string var;
for (int i = 1; i < argc; ++i) var += std::string(" ") + argv[i];
std::printf("TensorRT %d.%d.%d, variant:%s\n", getInferLibVersion() / 10000, (getInferLibVersion() / 100) % 100,
getInferLibVersion() % 100, var.c_str());
// shapes of the model where this was found: batch 2 (two camera views), 64 channels, 128 x 224
const int N = 2, C = 64, H = 128, W = 224, K = 16;
std::mt19937 g(1);
std::vector<std::vector<__half>> keep;
auto weights = [&](size_t n, float scale) {
keep.push_back(randomHalf(n, scale, g));
return Weights{DataType::kHALF, keep.back().data(), static_cast<int64_t>(n)};
};
std::unique_ptr<IBuilder> builder(createInferBuilder(gLogger));
std::unique_ptr<INetworkDefinition> net(
builder->createNetworkV2(1u << static_cast<uint32_t>(NetworkDefinitionCreationFlag::kSTRONGLY_TYPED)));
ITensor* x = net->addInput("x", DataType::kHALF, Dims4{N, C, H, W});
ITensor* xin = x;
if (catProducer) { // conv_out = relu(conv(cat(conv(x), x))), as the model's neck output that feeds the plugin
auto* p0 = net->addConvolutionNd(*x, C, Dims2{3, 3}, weights(size_t(C) * C * 9, 1.f / 24.f), weights(C, 0.f));
p0->setPaddingNd(Dims2{1, 1});
ITensor* pc[2] = {net->addActivation(*p0->getOutput(0), ActivationType::kRELU)->getOutput(0), x};
auto* cc = net->addConcatenation(pc, 2);
cc->setAxis(1);
auto* p1 = net->addConvolutionNd(*cc->getOutput(0), C, Dims2{3, 3}, weights(size_t(C) * 2 * C * 9, 1.f / 34.f),
weights(C, 0.f));
p1->setPaddingNd(Dims2{1, 1});
xin = net->addActivation(*p1->getOutput(0), ActivationType::kRELU)->getOutput(0);
}
auto* conv = net->addConvolutionNd(*xin, C, Dims2{3, 3}, weights(size_t(C) * C * 9, 1.f / 24.f), weights(C, 0.1f));
conv->setPaddingNd(Dims2{1, 1});
ITensor* y = net->addActivation(*conv->getOutput(0), ActivationType::kRELU)->getOutput(0);
y->setName("conv_out");
net->markOutput(*y);
if (outHwc8) y->setAllowedFormats(1u << static_cast<uint32_t>(TensorFormat::kHWC8));
PassPlugin plugin;
ITensor* in[1] = {y};
ITensor* z = net->addPluginV3(in, 1, nullptr, 0, plugin)->getOutput(0);
z->setName("plugin_out");
net->markOutput(*z);
z->setAllowedFormats(1u << static_cast<uint32_t>(TensorFormat::kHWC8));
if (consumer) { // y also read by a 1x1 conv
auto* c2 = net->addConvolutionNd(*y, 8, Dims2{1, 1}, weights(size_t(8) * C, 0.1f), weights(8, 0.f));
c2->getOutput(0)->setName("side");
net->markOutput(*c2->getOutput(0));
}
if (slice) { // y's first batch item also read by a 1x1 conv (in the model: guide features of the left view)
auto* sl = net->addSlice(*y, Dims4{0, 0, 0, 0}, Dims4{1, C, H, W}, Dims4{1, 1, 1, 1});
auto* c3 = net->addConvolutionNd(*sl->getOutput(0), 8, Dims2{1, 1}, weights(size_t(8) * C, 0.1f), weights(8, 0.f));
c3->getOutput(0)->setName("left");
net->markOutput(*c3->getOutput(0));
c3->getOutput(0)->setAllowedFormats(1u << static_cast<uint32_t>(TensorFormat::kHWC8));
}
if (resize) { // y also read by a linear x2 resize
auto* r = net->addResize(*y);
const float sc[4] = {1.f, 1.f, 2.f, 2.f};
r->setScales(sc, 4);
r->setResizeMode(InterpolationMode::kLINEAR);
r->setCoordinateTransformation(ResizeCoordinateTransformation::kHALF_PIXEL);
r->getOutput(0)->setName("resized");
net->markOutput(*r->getOutput(0));
}
if (cat) { // cat(plugin_out, conv(x or its x2 resize)) -> conv (kHWC8 output), as in the model's neck
const int Cz = gUp2 ? kUpK : C;
ITensor* xb = x;
if (gUp2) {
auto* r = net->addResize(*x);
const float sc[4] = {1.f, 1.f, 2.f, 2.f};
r->setScales(sc, 4);
xb = r->getOutput(0);
}
auto* b = net->addConvolutionNd(*xb, Cz, Dims2{3, 3}, weights(size_t(Cz) * C * 9, 1.f / 24.f), weights(Cz, 0.f));
b->setPaddingNd(Dims2{1, 1});
ITensor* cin[2] = {z, net->addActivation(*b->getOutput(0), ActivationType::kRELU)->getOutput(0)};
auto* cc = net->addConcatenation(cin, 2);
cc->setAxis(1);
auto* t = net->addConvolutionNd(*cc->getOutput(0), K, Dims2{3, 3}, weights(size_t(K) * 2 * Cz * 9, 1.f / 17.f),
weights(K, 0.f));
t->setPaddingNd(Dims2{1, 1});
t->getOutput(0)->setName("cat_tail");
net->markOutput(*t->getOutput(0));
t->getOutput(0)->setAllowedFormats(1u << static_cast<uint32_t>(TensorFormat::kHWC8));
}
if (trunk) { // cat(conv(plugin_out), conv(x or its x2 resize)) -> conv, as in the model's neck
const int Cz = gUp2 ? kUpK : C;
auto* a = net->addConvolutionNd(*z, K, Dims2{3, 3}, weights(size_t(K) * Cz * 9, 1.f / 24.f), weights(K, 0.f));
a->setPaddingNd(Dims2{1, 1});
ITensor* xb = x;
if (gUp2) {
auto* r = net->addResize(*x);
const float sc[4] = {1.f, 1.f, 2.f, 2.f};
r->setScales(sc, 4);
xb = r->getOutput(0);
}
auto* b = net->addConvolutionNd(*xb, K, Dims2{3, 3}, weights(size_t(K) * C * 9, 1.f / 24.f), weights(K, 0.f));
b->setPaddingNd(Dims2{1, 1});
ITensor* cat[2] = {a->getOutput(0), b->getOutput(0)};
auto* cc = net->addConcatenation(cat, 2);
cc->setAxis(1);
auto* t = net->addConvolutionNd(*cc->getOutput(0), K, Dims2{3, 3}, weights(size_t(K) * 2 * K * 9, 1.f / 17.f),
weights(K, 0.f));
t->setPaddingNd(Dims2{1, 1});
t->getOutput(0)->setName("tail");
net->markOutput(*t->getOutput(0));
}
std::unique_ptr<IBuilderConfig> cfg(builder->createBuilderConfig());
std::unique_ptr<IHostMemory> plan(builder->buildSerializedNetwork(*net, *cfg));
if (!plan) {
std::printf("RESULT: engine build FAILED\n");
return 2;
}
std::unique_ptr<IRuntime> runtime(createInferRuntime(gLogger));
std::unique_ptr<ICudaEngine> engine(runtime->deserializeCudaEngine(plan->data(), plan->size()));
std::unique_ptr<IExecutionContext> ctx(engine->createExecutionContext());
std::vector<void*> bufs;
size_t nXY = 0;
for (int i = 0; i < engine->getNbIOTensors(); ++i) {
const char* name = engine->getIOTensorName(i);
const Dims d = engine->getTensorShape(name);
const int vec = engine->getTensorVectorizedDim(name) >= 0 ? engine->getTensorComponentsPerElement(name) : 1;
size_t n = 1;
for (int k = 0; k < d.nbDims; ++k) n *= k == 1 ? (d.d[k] + vec - 1) / vec * vec : d.d[k];
void* p = nullptr;
CK(cudaMalloc(&p, n * sizeof(__half)));
CK(cudaMemset(p, 0, n * sizeof(__half)));
if (std::strcmp(name, "x") == 0) {
auto hx = randomHalf(n, 1.f, g);
CK(cudaMemcpy(p, hx.data(), n * sizeof(__half), cudaMemcpyHostToDevice));
}
if (std::strcmp(name, "conv_out") == 0) nXY = n;
std::printf("io %-10s format %s\n", name, engine->getTensorFormatDesc(name));
ctx->setTensorAddress(name, p);
bufs.push_back(p);
}
cudaStream_t s;
CK(cudaStreamCreate(&s));
if (!ctx->enqueueV3(s)) {
std::printf("RESULT: enqueue failed\n");
return 3;
}
CK(cudaStreamSynchronize(s));
std::vector<__half> hy(nXY), hz(nXY);
CK(cudaMemcpy(hy.data(), ctx->getTensorAddress("conv_out"), nXY * sizeof(__half), cudaMemcpyDeviceToHost));
CK(cudaMemcpy(hz.data(), ctx->getTensorAddress("plugin_out"), nXY * sizeof(__half), cudaMemcpyDeviceToHost));
const auto fy = toNCHW(hy, engine->getTensorFormat("conv_out"), N, C, H, W);
std::vector<float> expect = fy, fz;
if (gUp2) { // nearest x2 of the first kUpK channels
expect.assign(static_cast<size_t>(N) * kUpK * 4 * H * W, 0.f);
for (int n = 0; n < N; ++n)
for (int k = 0; k < kUpK; ++k)
for (int yy = 0; yy < 2 * H; ++yy)
for (int xx = 0; xx < 2 * W; ++xx)
expect[((static_cast<size_t>(n) * kUpK + k) * 2 * H + yy) * 2 * W + xx] =
fy[((static_cast<size_t>(n) * C + k) * H + yy / 2) * W + xx / 2];
std::vector<__half> hz2(static_cast<size_t>(N) * kUpK * 4 * H * W);
CK(cudaMemcpy(hz2.data(), ctx->getTensorAddress("plugin_out"), hz2.size() * sizeof(__half), cudaMemcpyDeviceToHost));
fz = toNCHW(hz2, engine->getTensorFormat("plugin_out"), N, kUpK, 2 * H, 2 * W);
} else {
fz = toNCHW(hz, engine->getTensorFormat("plugin_out"), N, C, H, W);
}
float maxd = 0.f;
size_t bad = 0;
for (size_t i = 0; i < expect.size(); ++i) {
const float d = std::fabs(expect[i] - fz[i]);
maxd = std::fmax(maxd, d);
bad += d > 0.f;
}
nXY = expect.size();
std::printf("RESULT: max |plugin_out - expected from conv_out| = %g, %zu of %zu elements differ -> %s\n", maxd, bad, nXY,
bad ? "WRONG (the plugin was given a different input than conv_out)" : "OK");
for (void* p : bufs) cudaFree(p);
return bad ? 4 : 0;
}
Steps To Reproduce
Commands or scripts:
cmake -S . -B build && cmake --build build
./build/repro # OK
./build/repro --tactics # OK: the plugin offers 8 tactics
./build/repro --out-hwc8 # OK: conv_out restricted to kHWC8
./build/repro --out-hwc8 --tactics # Thor / TensorRT 10.16.2: engine build FAILED
Output on Thor:
$ ./build/repro
TensorRT 10.16.2, variant:
RESULT: max |plugin_out - expected from conv_out| = 0, 0 of 3670016 elements differ -> OK
$ ./build/repro --tactics
TensorRT 10.16.2, variant: --tactics
RESULT: max |plugin_out - expected from conv_out| = 0, 0 of 3670016 elements differ -> OK
$ ./build/repro --out-hwc8
TensorRT 10.16.2, variant: --out-hwc8
RESULT: max |plugin_out - expected from conv_out| = 0, 0 of 3670016 elements differ -> OK
$ ./build/repro --out-hwc8 --tactics
TensorRT 10.16.2, variant: --out-hwc8 --tactics
Internal Error: MyelinCheckException: layouts.cpp:2180: CHECK(always(sym_eql(dims[i + 1], tiles[idx]))) failed.
[trt] Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [myelin_graph.h:1250: attachExceptionMsgToGraph] MyelinCheckException: layouts.cpp:2180: CHECK(always(sym_eql(dims[i + 1], tiles[idx]))) failed. In compileGraph at /_src/optimizer/myelin/codeGenerator.cpp:1795
[trt] IBuilder::buildSerializedNetwork: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[(Unnamed Layer* 0) [Convolution]...(Unnamed Layer* 2) [PluginV3_V3ONE]]}. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4265)
RESULT: engine build FAILED
The failure is deterministic, and it stays the same in these variations:
- extra consumers of
conv_out(--consumer,--resize,--slice); - the plugin changing the shape (
--up2); - the plugin output feeding a concat (
--cat,--trunk).
What matters is the combination of tactics and a kHWC8-restricted network output on the plugin's input.
Have you tried the latest release?: 10.16.2.10 is the TensorRT shipped with JetPack 7.2, the newest we have on Thor. TensorRT 10.16.0 on x86 is not affected.
Can this model run on other frameworks? N/A: there is no model file, since the network is built with the TensorRT API. The same repro builds and is exact with TensorRT 10.3 (Orin) and 10.16.0 (x86).
- 主要言語
- C++
- スター
- 13.5k
- フォーク
- 2.4k
- 平均マージ
- 6日 21時間
- マージ済み PR(30日)
- 3
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
NVIDIA/TensorRT のほかの issue
-
Module:Polygraphy
難易度 2/5 1〜3時間 初心者へのやさしさ 77/100
メンテナーはふだん 6 日以内に返信
-
Polygraphy NumPy Top-K sorts unsigned and minimum signed integers incorrectly対応中かも @0z5a が 16 日前に担当しました。 オープンModule:Polygraphy
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
メンテナーはふだん 6 日以内に返信
-
Polygraphy INT32 comparison can overflow and falsely pass at zero tolerance対応中かも @0z5a が 16 日前に担当しました。 オープンModule:Polygraphy
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
メンテナーはふだん 6 日以内に返信
-
onnx-graphsurgeon: Graph.toposort() hits RecursionError when a graph has a local function and an If/Loop node対応中かも @gabortar が 39 日前に担当しました。 オープンModule:GraphSurgeon
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
メンテナーはふだん 6 日以内に返信
-
Polygraphy parse_meta accepts a shape with a missing bracket対応中かも @SashaMIT が 51 日前に担当しました。 オープンModule:Polygraphy
難易度 2/5 1〜3時間 初心者へのやさしさ 86/100
メンテナーはふだん 6 日以内に返信
NVIDIA/TensorRT の issue をすべて見る
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 79/100
tenstorrent/tt-umd#4017 ·
メンテナーはふだん 3 日以内に返信
-
difficulty:easy func:gui prio:low type:bug
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
NatronGitHub/Natron#1538 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
hyprwm/hyprsunset#103 · コメント 1 件 ·
-
Feature request
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
qbittorrent/qBittorrent#25338 ·
メンテナーはふだん 2 日以内に返信
-
Needs: Attention Needs: Version Info Platform: Android Platform: tvOS
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
react/react-native#60180 · コメント 3 件 ·
メンテナーはふだん 2 日以内に返信