Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Bug: TensorRT 11.4 SmolLM3 native-KV prefill plan builds but cannot create an execution context (fused scatter kernel missing)

Abierto
  • ¿Algo no está bien?
#1,922 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
15/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
python
Área
backend

Línea de trabajo

Start with the check_smollm3_contexts.py reproducer in the issue and compare execution-context creation on TensorRT 11.4 with -kgen:fuse_scatter_nd on and off. The issue says the existing workaround from #1902 unblocks SmolLM3, while the remaining failure needs a TensorRT fix; this is done when the affected plan can create a context or the builder fails instead of serializing a plan that cannot.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

AI bug

This was generated by AI during triage.

Affected area

Runtime loading or dispatch

TensorRT-Model-Connect version

39565ca554122730cd57b7adbdbb4614ffa06d5e (main, which already contains the workaround from #1902)

Installation method

Development container

Model and revision

HuggingFaceTB/SmolLM3-3B at a07cc9a04f16550a088caea529712d1d335b0ac1, case smollm3-3b-l0 (BF16, native KV cache, max_sequence_length 65536)

Environment
  • OS: Ubuntu 24.04.5 (development container built from the repository Dockerfile)
  • GPU: NVIDIA GB300
  • Driver: 595.58.03
  • CUDA: 13.4.3
  • TensorRT: 11.4.0.106 fails. 11.1.0.106 works. Both come from the tensorrt-cu13 wheels.
Summary

On TensorRT 11.4, SmolLM3's native-KV prefill plan builds and deserializes, but ICudaEngine::createExecutionContext fails: Myelin cannot find the fused reshape/transpose/scatter kernel it generated for the K/V cache updates (CUDA error 500, function not found in module). Setting the build route -kgen:fuse_scatter_nd=off avoids it, which is the workaround merged in #1902. TensorRT 11.1 creates the context with the fused route enabled.

Minimal reproducer

Run from the repository root in the development container, after downloading the checkpoint. The script builds the smollm3-3b-l0 bundle through the public build API, forces the given build route on every plan, and creates an execution context for each plan right after it is built.

huggingface-cli download HuggingFaceTB/SmolLM3-3B --revision a07cc9a04f16550a088caea529712d1d335b0ac1
MODEL=$(python3 -c "from huggingface_hub import snapshot_download; print(snapshot_download('HuggingFaceTB/SmolLM3-3B', revision='a07cc9a04f16550a088caea529712d1d335b0ac1'))")

PYTHONPATH=core/builder:. python3 check_smollm3_contexts.py "$MODEL" -kgen:fuse_scatter_nd=on   # TensorRT 11.4: prefill context fails
PYTHONPATH=core/builder:. python3 check_smollm3_contexts.py "$MODEL" -kgen:fuse_scatter_nd=off  # TensorRT 11.4: both contexts created

check_smollm3_contexts.py:

"""Build the smollm3-3b-l0 bundle with a forced build route and create an execution context per plan."""

import sys
from pathlib import Path

import tensorrt as trt

from tensorrt_model_connect import BuildRequest, build

model_dir, build_route = Path(sys.argv[1]), sys.argv[2]
original = trt.Builder.build_serialized_network


def build_and_check(self, network, config):
    config.build_route = build_route
    plan = original(self, network, config)
    engine = trt.Runtime(trt.Logger(trt.Logger.ERROR)).deserialize_cuda_engine(plan)
    context = engine.create_execution_context()
    print(f"tensorrt={trt.__version__} layers={network.num_layers} route={build_route} "
          f"context_created={context is not None}", flush=True)
    return plan


trt.Builder.build_serialized_network = build_and_check
build(
    BuildRequest(
        model_dir=model_dir,
        output_path=Path("smollm3-3b-l0.bundle"),
        family="smollm3",
        task="text_generation",
        precision="bf16",
        max_sequence_length=65536,
        tensor_parallel_size=1,
    )
)
Observed behavior
TensorRT -kgen:fuse_scatter_nd Prefill plan context Decode plan context
11.4.0.106 on (11.4 default before #1902) fails created
11.4.0.106 off (#1902 workaround) created created
11.1.0.106 on created created

Each plan has 3,805 layers and contains 72 IKVCacheUpdateLayer updates (36 layers, K and V). The missing function name shows that Myelin fused each layer's K and V update chains (reshape, transpose, reshape, scatter) into one kernel. A standalone network with only the two reshape/transpose chains and IKVCacheUpdateLayers (BF16, cache [1, 4, 65536, 128], dynamic sequence up to 64) builds and creates a context on 11.4, so the failing fusion needs the surrounding decoder layer.

Expected behavior

A plan that buildSerializedNetwork returns should create an execution context, as on TensorRT 11.1. If the fused scatter kernel cannot be emitted, the builder should fall back or fail the build instead of serializing a plan that references a missing function.

Relevant logs
[TRT] [E] ICudaEngine::createExecutionContext: Error Code 1: Myelin ([cudamod.cpp:61: fqn] CUDA error 500 when loading function '__myl_ReshTranReshScatReshTranReshScat_0x1f1068d04883572264fe9bb736e9cdbf_vec'.; [launch.cpp:55: linfo] Function '__myl_ReshTranReshScatReshTranReshScat_0x1f1068d04883572264fe9bb736e9cdbf_vec' not found in module. In MyelinGraphContext at graphContext.cpp:79)
Error: [trtmc] Failed to create TRT execution context
Impact and next steps
  • #1902 already sets -kgen:fuse_scatter_nd=off for SmolLM3's native-KV builds, so main is unblocked. Other families that build native-KV prefill plans with the same update pattern can hit the same failure on TensorRT 11.4.
  • This needs a TensorRT fix; the reproducer above is ready to route to the Myelin team. A checkpoint-free network that triggers the fusion has not been found yet.
Submission checks
  • I searched open and closed issues and found no duplicate.
  • This is not a security vulnerability.
  • I removed secrets, credentials, private URLs, and restricted artifacts.
Lenguaje dominante
Python
Estrellas
283
Forks
75
Merge medio
1 d 23 h
PR fusionados (30 d)
227

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/TensorRT-Model-Connect

Todos los issues de NVIDIA/TensorRT-Model-Connect

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.