TensorRT 11.4.0.106 leaves half of FP16 GroupNorm output unwritten with a dynamic height
Los mantenedores suelen responder en 6 días
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 35/100
- Tipo de issue
- Error
- Claridad
- Bien especificado
- Estado de actividad
- Activo
- Stack tecnológico
- numpy, python
- Área
- machine-learning, performance
Línea de trabajo
Start with the self-contained groupnorm_minimal.py reproducer in the issue and run it with TensorRT 11.4.0.106 on a supported NVIDIA GPU. Compare the dynamic and fixed profile results; done means the dynamic profile writes all 32 FP16 values and matches the expected output. The report identifies generated GroupNorm CUDA source as the suspected location, but does not name a repository source file.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Description
A native TensorRT INormalizationLayer configured for GroupNorm returns success but leaves the second channel of each group unchanged in the output buffer. The reproducer builds one GroupNorm operation with identity scale and zero bias and runs it on 32 synthetic FP16 values (64 bytes).
Environment
| Component | Tested configuration |
|---|---|
| TensorRT | tensorrt-cu13==11.4.0.106 |
| OS | Windows 11 Pro x64, build 26300 |
| GPUs | Four GeForce RTX 3060 cards, two GA104 (10de:2487) and two GA106 (10de:2503), all SM 8.6 |
| NVIDIA driver | 616.56, reporting CUDA Driver API 13.4 |
| Python | 3.13.12 with numpy==2.5.4 and cuda-bindings==13.0.3 |
Reproduction
Save the Python block below as groupnorm_minimal.py. Run these commands in a Python environment with the NVIDIA driver available.
python -m pip install numpy==2.5.4 cuda-bindings==13.0.3 tensorrt-cu13==11.4.0.106
python groupnorm_minimal.py
The network input is [1,16,-1,1] in NCHW order, with eight groups and spatial axes mask 0xC. Both engines receive [1,16,2,1] with identical values. The only profile change is the minimum height.
| Engine | MIN | OPT | MAX |
|---|---|---|---|
| Dynamic | [1,16,1,1] |
[1,16,2,1] |
[1,16,2,1] |
| Fixed control | [1,16,2,1] |
[1,16,2,1] |
[1,16,2,1] |
The input is eight repetitions of [-1, -1, +1, +1]. Each group has two channels, mean zero, and variance one. The expected FP16 output is the input after rounding. The program runs each engine twice and pre-fills the output buffer with 0x0000 and 0x3C3C, respectively. A word is counted as untouched when it retains its corresponding prefill in both runs.
Expected and actual results
Both engines should produce all 32 expected FP16 values regardless of the prefill. With the dynamic profile, the 16 values in zero-based channels 1, 3, 5, 7, 9, 11, 13, and 15 retain the prefill. The fixed profile produces all 32 expected values. The reproducer produced this pattern on all four RTX 3060 cards.
TensorRT 11.4.0.106, CUDA driver API 13040, GPU 0
dynamic: 16/32 words retain both prefills; untouched channels=[1, 3, 5, 7, 9, 11, 13, 15]; matches expected=False
fixed: 0/32 words retain both prefills; untouched channels=[]; matches expected=True
REPRODUCED
Root cause
I captured the CUDA source and PTX during engine construction. The generated GroupNorm kernel family __myl_Gnor_0x94d91cb3b6c44782b0bdbbf351529669 contains a missing pair of parentheses in the input and output channel-index expressions, in both the general and specialized kernel entries. With names simplified, the two forms are
// Generated
(index / 1 * height * width) % channelsPerGroup
// Correct grouping of the divisor
(index / (1 * height * width)) % channelsPerGroup
For this input, height = 2, width = 1, and channelsPerGroup = 2. C++ evaluates the generated expression as (index * 2) % 2, which is always zero. Each group therefore addresses its first channel repeatedly and leaves its second channel unchanged. The corrected expression selects both channels.
I changed only the four instances of the divisor grouping in the generated source in a separate diagnostic build. With the same internal compiler and options, the dynamic engine then wrote all 32 elements and matched the expected FP16 output exactly.
As a separate cross-check, the 12×16 vector-kernel reproducer showed the same missing parentheses. Reassembling its original PTX with ptxas 13.4.59 and 13.4.92 at -O0 and -O3 preserved the same 2,304 untouched values out of 3,072. The incorrect indexing is already present in the CUDA source generated by TensorRT.
Self-contained reproducer
"""Reproduce incomplete FP16 GroupNorm output with one TensorRT layer."""
import argparse
from contextlib import ExitStack
from cuda.bindings import driver as cu
import numpy as np
import tensorrt as trt
SHAPE = (1, 16, 2, 1)
def check(result):
status, *values = result
if status != cu.CUresult.CUDA_SUCCESS:
raise RuntimeError(str(status))
return values[0] if values else None
def run(fixed):
logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)
network = builder.create_network(0)
x = network.add_input("x", trt.float16, (1, 16, -1, 1))
affine_shape = (1, 16, 1, 1)
scale = np.ones(affine_shape, dtype=np.float16)
bias = np.zeros_like(scale)
s = network.add_constant(affine_shape, scale).get_output(0)
b = network.add_constant(affine_shape, bias).get_output(0)
norm = network.add_normalization_v2(x, s, b, 0b1100)
norm.num_groups = 8
norm.epsilon = 1e-5
y = norm.get_output(0)
y.name = "y"
network.mark_output(y)
config = builder.create_builder_config()
config.max_aux_streams = 0
profile = builder.create_optimization_profile()
profile.set_shape("x", SHAPE if fixed else (1, 16, 1, 1), SHAPE, SHAPE)
config.add_optimization_profile(profile)
plan = builder.build_serialized_network(network, config)
if plan is None:
raise RuntimeError("TensorRT engine build failed")
runtime = trt.Runtime(logger)
engine = runtime.deserialize_cuda_engine(plan)
context = engine.create_execution_context()
if not context.set_input_shape("x", SHAPE) or tuple(context.get_tensor_shape("y")) != SHAPE:
raise RuntimeError("Unexpected input or output shape")
if any(engine.get_tensor_format(name) != trt.TensorFormat.LINEAR
or engine.get_tensor_dtype(name) != trt.float16 for name in ("x", "y")):
raise RuntimeError("Expected linear FP16 I/O")
# Each group has mean 0 and variance 1. The expected FP16 output is the input.
data = np.tile(np.array([-1, -1, 1, 1], dtype=np.float16), 8)
output = np.empty_like(data)
outputs = []
with ExitStack() as buffers:
x_ptr = check(cu.cuMemAlloc(data.nbytes))
buffers.callback(lambda: check(cu.cuMemFree(x_ptr)))
y_ptr = check(cu.cuMemAlloc(output.nbytes))
buffers.callback(lambda: check(cu.cuMemFree(y_ptr)))
addresses = {"x": x_ptr, "y": y_ptr}
bindings = [int(addresses[engine.get_tensor_name(i)]) for i in range(engine.num_io_tensors)]
check(cu.cuMemcpyHtoD(x_ptr, data.ctypes.data, data.nbytes))
for fill in (0, 0x3C):
check(cu.cuMemsetD8(y_ptr, fill, output.nbytes))
check(cu.cuCtxSynchronize())
if not context.execute_v2(bindings):
raise RuntimeError("TensorRT execution failed")
check(cu.cuMemcpyDtoH(output.ctypes.data, y_ptr, output.nbytes))
outputs.append(output.copy())
untouched = (outputs[0].view(np.uint16) == 0) & (outputs[1].view(np.uint16) == 0x3C3C)
channels = np.flatnonzero(untouched.reshape(16, 2).all(axis=1)).tolist()
matches = all(np.array_equal(result, data) for result in outputs)
print(f'{"fixed" if fixed else "dynamic"}: {untouched.sum()}/32 words retain both prefills; '
f"untouched channels={channels}; matches expected={matches}")
return int(untouched.sum()), matches
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--device", type=int, default=0)
args = parser.parse_args()
check(cu.cuInit(0))
gpu = check(cu.cuDeviceGet(args.device))
primary = check(cu.cuDevicePrimaryCtxRetain(gpu))
try:
check(cu.cuCtxSetCurrent(primary))
print(f"TensorRT {trt.__version__}, CUDA driver API {check(cu.cuDriverGetVersion())}, GPU {args.device}")
untouched, matches = run(fixed=False)
if run(fixed=True) != (0, True):
raise RuntimeError("Fixed-profile control failed")
print("REPRODUCED" if untouched and not matches else "NOT REPRODUCED")
finally:
check(cu.cuDevicePrimaryCtxRelease(gpu))
if __name__ == "__main__":
main()
- Lenguaje dominante
- C++
- Estrellas
- 13.5k
- Forks
- 2.4k
- Merge medio
- 6 d 21 h
- PR fusionados (30 d)
- 3
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de NVIDIA/TensorRT
-
Module:Polygraphy
Dificultad 2/5 1-3 horas Aptitud para principiantes 77/100
Los mantenedores suelen responder en 6 días
-
Polygraphy NumPy Top-K sorts unsigned and minimum signed integers incorrectlyPosiblemente ocupada @0z5a la tomó hace 16 días. AbiertoModule:Polygraphy
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
Los mantenedores suelen responder en 6 días
-
Polygraphy INT32 comparison can overflow and falsely pass at zero tolerancePosiblemente ocupada @0z5a la tomó hace 16 días. AbiertoModule:Polygraphy
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
Los mantenedores suelen responder en 6 días
-
onnx-graphsurgeon: Graph.toposort() hits RecursionError when a graph has a local function and an If/Loop nodePosiblemente ocupada @gabortar la tomó hace 39 días. AbiertoModule:GraphSurgeon
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Los mantenedores suelen responder en 6 días
-
Polygraphy parse_meta accepts a shape with a missing bracketPosiblemente ocupada @SashaMIT la tomó hace 51 días. AbiertoModule:Polygraphy
Dificultad 2/5 1-3 horas Aptitud para principiantes 86/100
Los mantenedores suelen responder en 6 días
Todos los issues de NVIDIA/TensorRT
Issues similares
-
[UBSan] spawn_simulator passes null child_options.args to uv_spawn (execvp nonnull violation)Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 79/100
tenstorrent/tt-umd#4017 ·
Los mantenedores suelen responder en 3 días
-
difficulty:easy func:gui prio:low type:bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
NatronGitHub/Natron#1538 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
hyprwm/hyprsunset#103 · 1 comentario ·
-
Feature request
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
qbittorrent/qBittorrent#25338 ·
Los mantenedores suelen responder en 2 días
-
Needs: Attention Needs: Version Info Platform: Android Platform: tvOS
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
react/react-native#60180 · 3 comentarios ·
Los mantenedores suelen responder en 2 días