TensorRT 11.0.0.114 builder finds no valid tactic for BF16 ConvTranspose2d (works fine in FP16)
维护者通常 6 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 38/100
- Issue 类型
- 缺陷
- 描述清晰度
- 基本清楚
- 活跃度
- 冷清
- 技术栈
- python
调研方向
首先运行启用详细日志的最小 TensorRT Python reproducer,并比较 ConvTranspose2d 的 BF16 和 FP16 构建。调查日志所指向的反卷积 tactic 和类型推断路径;当 BF16 网络能够针对报告的 shape 构建出序列化 engine,且不导致 FP16 行为回归时,即视为完成。
由索引模型根据 Issue 内容生成。
描述
TensorRT 11.0.0.114 builder finds no valid tactic for BF16 ConvTranspose2d (works fine in FP16)
Bug description
IBuilder::buildEngineWithConfig fails to build a serialized engine for a network containing
a single IDeconvolutionLayer (ConvTranspose2d) when the layer's I/O tensors are BF16.
The identical layer (same shapes, same strongly-typed network construction) builds
successfully when the dtype is FP16 instead. This reproduces both through the pure
TensorRT Python API (no PyTorch / torch_tensorrt involved) and through torch_tensorrt.dynamo.compile.
Encountered while exporting a real model (a ViT + FPN image encoder) to TensorRT via
torch_tensorrt; a 2x-upsampling nn.ConvTranspose2d(1024, 512, kernel_size=2, stride=2)
layer in the FPN is the only thing blocking the whole graph from compiling in BF16.
Minimal reproducer (pure TensorRT Python API, no torch/torch_tensorrt)
import numpy as np
import tensorrt as trt
TRT_LOGGER = trt.Logger(trt.Logger.VERBOSE)
def build(dtype, label):
builder = trt.Builder(TRT_LOGGER)
flags = 1 << int(trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED)
network = builder.create_network(flags)
config = builder.create_builder_config()
IN_C, OUT_C, K, S = 1024, 512, 2, 2
H = W = 72
inp = network.add_input(name="x", dtype=dtype, shape=(1, IN_C, H, W))
rng = np.random.default_rng(0)
w = rng.standard_normal((IN_C, OUT_C, K, K)).astype(np.float32)
if dtype == trt.DataType.BF16:
import ml_dtypes
w_cast = w.astype(ml_dtypes.bfloat16)
else:
w_cast = w.astype(np.float16)
weights = trt.Weights(dtype, w_cast.ctypes.data, w_cast.size)
deconv = network.add_deconvolution_nd(
input=inp, num_output_maps=OUT_C, kernel_shape=(K, K), kernel=weights,
)
deconv.stride_nd = (S, S)
network.mark_output(deconv.get_output(0))
print(f"\n=== Building {label} engine ===")
serialized = builder.build_serialized_network(network, config)
if serialized is None:
print(f"{label}: BUILD FAILED (builder returned None)")
return False
print(f"{label}: BUILD SUCCEEDED ({serialized.nbytes} bytes)")
return True
print("TensorRT version:", trt.__version__)
build(trt.DataType.BF16, "BF16")
build(trt.DataType.HALF, "FP16")
Output:
TensorRT version: 11.0.0.114
=== Building BF16 engine ===
BF16: BUILD FAILED (builder returned None)
=== Building FP16 engine ===
FP16: BUILD SUCCEEDED (4222860 bytes)
Relevant verbose log lines for the BF16 attempt:
[TRT] [V] =============== Computing costs for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] *************** Autotuning format combination: Half(5308416,5184,72,1) -> Half(10616832,20736,144,1) ***************
[TRT] [V] Skipping CaskDeconvolution: No valid tactics for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] Skipping CaskDeconvolutionV2: No valid tactics for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] (Unnamed Layer* 0) [Deconvolution]: 189 available tactics, 0 unparsable, 94 pruned, 95 remaining after tactic pruning.
(189 tactics are enumerated and timed, but none is accepted — the builder ultimately returns None
with no explicit error printed at default log severity; only visible in VERBOSE.)
Second reproducer: via torch_tensorrt (same root cause, more informative error)
Compiling the same shape through torch_tensorrt.dynamo.compile surfaces an explicit error
instead of a silent None:
import torch
import torch.nn as nn
import torch_tensorrt
class Deconv(nn.Module):
def __init__(self):
super().__init__()
self.dconv = nn.ConvTranspose2d(1024, 512, kernel_size=2, stride=2)
def forward(self, x):
return self.dconv(x)
device = torch.device("cuda")
m = Deconv().eval().to(device).to(torch.bfloat16)
x = torch.randn(1, 1024, 72, 72, device=device, dtype=torch.bfloat16)
with torch.no_grad():
ep = torch.export.export(m, (x,))
trt_ep = torch_tensorrt.dynamo.compile(
ep, inputs=[x],
enabled_precisions={torch.bfloat16},
use_explicit_typing=False,
device=torch_tensorrt.Device(gpu_id=0),
min_block_size=1, # force conversion of this small subgraph
)
Error:
ERROR:torch_tensorrt [TensorRT Conversion Context]:Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [type.cpp:186: infer_type] Could not infer output types for operation: 35: deconv: output0_before_bias- | x-(bf16[1,1024,72,72][]so[3,2,1,0]p[0,0,0,0], mem_prop=0, align=2), [DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution] filterWeightsBFloat16-{-0.00448608, -0.00393677, -0.00421143, 0.00521851, -0.0133057, 0.00176239, 0.0220947, -0.0159912, ...}(bf16[1024,512,2,2][2048,4,2,1]so[3,2,1,0], mem_prop=0, align=2)<entry>, __mye36[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]_alpha-1F:(f32[][]so[], mem_prop=0, align=4)<entry>, __mye37[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]_beta-0F:(f32[][]so[], mem_prop=0, align=4)<entry>, stream = 0 // [DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]
| n_groups: 1 lpad: {0, 0} rpad: {0, 0} pad_mode: 0 strides: {2, 2} dilations: {1, 1} , No matching rules found for input operand types In compileGraph at /_src/optimize
ERROR:torch_tensorrt [TensorRT Conversion Context]:IBuilder::buildEngineWithConfig: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]]}. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4284)
Same layer compiled with enabled_precisions={torch.float16} (fp16 weights/input instead of bf16,
otherwise identical) succeeds with rel_err ≈ 0.0015 vs. the PyTorch eager reference (normal fp16
rounding).
Environment
| tensorrt | 11.0.0.114 |
| torch-tensorrt | 2.13.0 |
| torch | 2.13.0+cu130 |
| CUDA | 13.0 |
| GPU | NVIDIA GeForce RTX 4080 (sm_89) |
| Python | 3.12 |
| OS | Linux |
Additional notes
- Only
BF16is affected;FP16and (implicitly, since it's the historically well-supported
path)FP32work for the identical layer/shape. - Reproduces regardless of
output_padding(default0here) — this is not the same issue as
the older, already-fixed pytorch/TensorRT#3352
(output_padding != 0crash). - Shape used in the repro (
in_channels=1024, out_channels=512, kernel_size=2, stride=2,
72x72 -> 144x144spatial) is the real shape from a production FPN upsampling layer, not
arbitrarily chosen — this isn't a degenerate edge case. - I have not yet tried other kernel/stride/channel combinations to determine how narrow the
affected shape space is; happy to run more combinations if useful for triage.
- 主要语言
- C++
- 星标
- 13.5k
- 派生
- 2.4k
- 平均合并
- 6 天 21 小时
- 30 天内合并 PR
- 3
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
NVIDIA/TensorRT 的其他 Issue
-
Module:Polygraphy
难度 2/5 1-3 小时 新手友好度 77/100
维护者通常 6 天内回复
-
Polygraphy NumPy Top-K sorts unsigned and minimum signed integers incorrectly可能已有人在做 @0z5a 于 16 天前认领。 未关闭Module:Polygraphy
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 6 天内回复
-
Polygraphy INT32 comparison can overflow and falsely pass at zero tolerance可能已有人在做 @0z5a 于 16 天前认领。 未关闭Module:Polygraphy
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 6 天内回复
-
onnx-graphsurgeon: Graph.toposort() hits RecursionError when a graph has a local function and an If/Loop node可能已有人在做 @gabortar 于 39 天前认领。 未关闭Module:GraphSurgeon
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 6 天内回复
-
Polygraphy parse_meta accepts a shape with a missing bracket可能已有人在做 @SashaMIT 于 51 天前认领。 未关闭Module:Polygraphy
难度 2/5 1-3 小时 新手友好度 86/100
维护者通常 6 天内回复
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 91/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 69/100
maplibre/maplibre-native#5072 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 79/100
tenstorrent/tt-umd#4017 ·
维护者通常 3 天内回复
-
difficulty:easy func:gui prio:low type:bug
难度 2/5 1-3 小时 新手友好度 72/100
NatronGitHub/Natron#1538 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
hyprwm/hyprsunset#103 · 1 条评论 ·