TensorRT 11.0.0.114 builder finds no valid tactic for BF16 ConvTranspose2d (works fine in FP16)
Maintainer antworten meist innerhalb von 6 Tagen
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 38/100
- Issue-Typ
- Bug
- Klarheit
- Größtenteils klar
- Aktivitätsstatus
- Ruhig
- Tech-Stack
- python
- Bereich
- machine-learning, performance
Rechercherichtung
Beginne damit, den minimalen TensorRT-Python-Reproducer mit ausführlicher Protokollierung auszuführen und die BF16- und FP16-Builds von ConvTranspose2d zu vergleichen. Untersuche die durch die Logs nahegelegten Deconvolution-Taktik- und Type-Inference-Pfade; abgeschlossen ist die Arbeit, wenn das BF16-Netzwerk für die gemeldete Form eine serialisierte Engine erstellt, ohne das FP16-Verhalten zu verschlechtern.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
TensorRT 11.0.0.114 builder finds no valid tactic for BF16 ConvTranspose2d (works fine in FP16)
Bug description
IBuilder::buildEngineWithConfig fails to build a serialized engine for a network containing
a single IDeconvolutionLayer (ConvTranspose2d) when the layer's I/O tensors are BF16.
The identical layer (same shapes, same strongly-typed network construction) builds
successfully when the dtype is FP16 instead. This reproduces both through the pure
TensorRT Python API (no PyTorch / torch_tensorrt involved) and through torch_tensorrt.dynamo.compile.
Encountered while exporting a real model (a ViT + FPN image encoder) to TensorRT via
torch_tensorrt; a 2x-upsampling nn.ConvTranspose2d(1024, 512, kernel_size=2, stride=2)
layer in the FPN is the only thing blocking the whole graph from compiling in BF16.
Minimal reproducer (pure TensorRT Python API, no torch/torch_tensorrt)
import numpy as np
import tensorrt as trt
TRT_LOGGER = trt.Logger(trt.Logger.VERBOSE)
def build(dtype, label):
builder = trt.Builder(TRT_LOGGER)
flags = 1 << int(trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED)
network = builder.create_network(flags)
config = builder.create_builder_config()
IN_C, OUT_C, K, S = 1024, 512, 2, 2
H = W = 72
inp = network.add_input(name="x", dtype=dtype, shape=(1, IN_C, H, W))
rng = np.random.default_rng(0)
w = rng.standard_normal((IN_C, OUT_C, K, K)).astype(np.float32)
if dtype == trt.DataType.BF16:
import ml_dtypes
w_cast = w.astype(ml_dtypes.bfloat16)
else:
w_cast = w.astype(np.float16)
weights = trt.Weights(dtype, w_cast.ctypes.data, w_cast.size)
deconv = network.add_deconvolution_nd(
input=inp, num_output_maps=OUT_C, kernel_shape=(K, K), kernel=weights,
)
deconv.stride_nd = (S, S)
network.mark_output(deconv.get_output(0))
print(f"\n=== Building {label} engine ===")
serialized = builder.build_serialized_network(network, config)
if serialized is None:
print(f"{label}: BUILD FAILED (builder returned None)")
return False
print(f"{label}: BUILD SUCCEEDED ({serialized.nbytes} bytes)")
return True
print("TensorRT version:", trt.__version__)
build(trt.DataType.BF16, "BF16")
build(trt.DataType.HALF, "FP16")
Output:
TensorRT version: 11.0.0.114
=== Building BF16 engine ===
BF16: BUILD FAILED (builder returned None)
=== Building FP16 engine ===
FP16: BUILD SUCCEEDED (4222860 bytes)
Relevant verbose log lines for the BF16 attempt:
[TRT] [V] =============== Computing costs for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] *************** Autotuning format combination: Half(5308416,5184,72,1) -> Half(10616832,20736,144,1) ***************
[TRT] [V] Skipping CaskDeconvolution: No valid tactics for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] Skipping CaskDeconvolutionV2: No valid tactics for (Unnamed Layer* 0) [Deconvolution]
[TRT] [V] (Unnamed Layer* 0) [Deconvolution]: 189 available tactics, 0 unparsable, 94 pruned, 95 remaining after tactic pruning.
(189 tactics are enumerated and timed, but none is accepted — the builder ultimately returns None
with no explicit error printed at default log severity; only visible in VERBOSE.)
Second reproducer: via torch_tensorrt (same root cause, more informative error)
Compiling the same shape through torch_tensorrt.dynamo.compile surfaces an explicit error
instead of a silent None:
import torch
import torch.nn as nn
import torch_tensorrt
class Deconv(nn.Module):
def __init__(self):
super().__init__()
self.dconv = nn.ConvTranspose2d(1024, 512, kernel_size=2, stride=2)
def forward(self, x):
return self.dconv(x)
device = torch.device("cuda")
m = Deconv().eval().to(device).to(torch.bfloat16)
x = torch.randn(1, 1024, 72, 72, device=device, dtype=torch.bfloat16)
with torch.no_grad():
ep = torch.export.export(m, (x,))
trt_ep = torch_tensorrt.dynamo.compile(
ep, inputs=[x],
enabled_precisions={torch.bfloat16},
use_explicit_typing=False,
device=torch_tensorrt.Device(gpu_id=0),
min_block_size=1, # force conversion of this small subgraph
)
Error:
ERROR:torch_tensorrt [TensorRT Conversion Context]:Error Code: 9: Skipping tactic 0x0000000000000000 due to exception [type.cpp:186: infer_type] Could not infer output types for operation: 35: deconv: output0_before_bias- | x-(bf16[1,1024,72,72][]so[3,2,1,0]p[0,0,0,0], mem_prop=0, align=2), [DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution] filterWeightsBFloat16-{-0.00448608, -0.00393677, -0.00421143, 0.00521851, -0.0133057, 0.00176239, 0.0220947, -0.0159912, ...}(bf16[1024,512,2,2][2048,4,2,1]so[3,2,1,0], mem_prop=0, align=2)<entry>, __mye36[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]_alpha-1F:(f32[][]so[], mem_prop=0, align=4)<entry>, __mye37[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]_beta-0F:(f32[][]so[], mem_prop=0, align=4)<entry>, stream = 0 // [DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]
| n_groups: 1 lpad: {0, 0} rpad: {0, 0} pad_mode: 0 strides: {2, 2} dilations: {1, 1} , No matching rules found for input operand types In compileGraph at /_src/optimize
ERROR:torch_tensorrt [TensorRT Conversion Context]:IBuilder::buildEngineWithConfig: Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[[DECONVOLUTION]-[aten_ops.convolution.default]-[dconv/convolution]]}. In computeCosts at /_src/optimizer/common/tactic/optimizer.cpp:4284)
Same layer compiled with enabled_precisions={torch.float16} (fp16 weights/input instead of bf16,
otherwise identical) succeeds with rel_err ≈ 0.0015 vs. the PyTorch eager reference (normal fp16
rounding).
Environment
| tensorrt | 11.0.0.114 |
| torch-tensorrt | 2.13.0 |
| torch | 2.13.0+cu130 |
| CUDA | 13.0 |
| GPU | NVIDIA GeForce RTX 4080 (sm_89) |
| Python | 3.12 |
| OS | Linux |
Additional notes
- Only
BF16is affected;FP16and (implicitly, since it's the historically well-supported
path)FP32work for the identical layer/shape. - Reproduces regardless of
output_padding(default0here) — this is not the same issue as
the older, already-fixed pytorch/TensorRT#3352
(output_padding != 0crash). - Shape used in the repro (
in_channels=1024, out_channels=512, kernel_size=2, stride=2,
72x72 -> 144x144spatial) is the real shape from a production FPN upsampling layer, not
arbitrarily chosen — this isn't a degenerate edge case. - I have not yet tried other kernel/stride/channel combinations to determine how narrow the
affected shape space is; happy to run more combinations if useful for triage.
- Vorherrschende Sprache
- C++
- Sterne
- 13.5k
- Forks
- 2.4k
- Ø Merge
- 6 T. 21 Std.
- Gemergte PRs (30 T.)
- 3
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Keine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus NVIDIA/TensorRT
-
Module:Polygraphy
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 77/100
Maintainer antworten meist innerhalb von 6 Tagen
-
Polygraphy NumPy Top-K sorts unsigned and minimum signed integers incorrectlyEvtl. vergeben @0z5a hat das vor 16 Tagen übernommen. OffenModule:Polygraphy
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
Maintainer antworten meist innerhalb von 6 Tagen
-
Polygraphy INT32 comparison can overflow and falsely pass at zero toleranceEvtl. vergeben @0z5a hat das vor 16 Tagen übernommen. OffenModule:Polygraphy
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
Maintainer antworten meist innerhalb von 6 Tagen
-
onnx-graphsurgeon: Graph.toposort() hits RecursionError when a graph has a local function and an If/Loop nodeEvtl. vergeben @gabortar hat das vor 39 Tagen übernommen. OffenModule:GraphSurgeon
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 82/100
Maintainer antworten meist innerhalb von 6 Tagen
-
Polygraphy parse_meta accepts a shape with a missing bracketEvtl. vergeben @SashaMIT hat das vor 51 Tagen übernommen. OffenModule:Polygraphy
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 86/100
Maintainer antworten meist innerhalb von 6 Tagen
Alle Issues in NVIDIA/TensorRT
Ähnliche Issues
-
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 91/100
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 69/100
maplibre/maplibre-native#5072 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 79/100
tenstorrent/tt-umd#4017 ·
Maintainer antworten meist innerhalb von 3 Tagen
-
difficulty:easy func:gui prio:low type:bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
NatronGitHub/Natron#1538 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
hyprwm/hyprsunset#103 · 1 Kommentar ·