Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[Bug] v0.10.0: prebuilt sm_87 (Jetson Orin) CuteDSL artifact missing fmha_v2_d* kernel modules, breaks all LLM engine builds

Open
  • Is something wrong?
#178 3 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
65/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
cmake, cpp
Domain
build-system, embedded-iot

Research direction

Start with cpp/kernels/contextAttentionKernels/cuteDslFMHAV2Runner.h and inspect the aarch64/sm_87 contents under cpp/kernels/cuteDSLArtifact, then reproduce using the documented CMake command on Jetson Orin. Compare the declared fmha_v2 modules with the available artifacts; done means the required modules are present and the edgellmCore build completes for sm_87.

Written by the indexing model from the issue text.

Description

Describe the bug

v0.10.0's CHANGELOG states: "Replaced the legacy embedded-cubin FMHA-v2 backend with CuTe DSL FMHA-v2 and removed the checked-in FMHA-v2 cubin artifacts." The new cpp/kernels/contextAttentionKernels/cuteDslFMHAV2Runner.h unconditionally declares LazyKernelModule<fmha_v2_d64_Kernel_Module_t> (and d128/d256/d512/paged/sw/bidirectional variants), but the prebuilt cpp/kernels/cuteDSLArtifact/aarch64/sm_87/ artifact checked into the repo at this same commit does not contain any fmha_v2_* headers (confirmed via find cpp/kernels/cuteDSLArtifact/aarch64/sm_87 -iname '*fmha*' → no results; only gemm/gdn/moe/int4_fp16_gemm/ffpa families are present). "fmha is always linked" per cpp/CMakeLists.txt comment, so this is not optional — it blocks compilation of edgellmCore entirely on Orin.

Since context-attention FMHA is required for essentially every LLM engine build, this appears to break llm_build for all models on native Jetson Orin (sm_87) at v0.10.0, not just spec-decode paths.

Steps to reproduce
# On Jetson Orin, JetPack 7.2, CUDA 13.2
git clone https://github.com/NVIDIA/TensorRT-Edge-LLM.git
cd TensorRT-Edge-LLM
git checkout v0.10.0
git submodule update --init --recursive
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DTRT_PACKAGE_DIR=/usr \
  -DCMAKE_TOOLCHAIN_FILE=cmake/aarch64_linux_toolchain.cmake \
  -DEMBEDDED_TARGET=jetson-orin -DCUDA_CTK_VERSION=13.2 -DENABLE_CUTE_DSL=ALL
make -j$(nproc)
Actual behavior
cpp/kernels/contextAttentionKernels/cuteDslFMHAV2Runner.h:127:37: error: ‘fmha_v2_d64_Kernel_Module_t’ was not declared in this scope
  127 |     static detail::LazyKernelModule<fmha_v2_d64_Kernel_Module_t> sLLM_d64;
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
make[2]: *** [cpp/CMakeFiles/edgellmCore.dir/build.make:216: cpp/CMakeFiles/edgellmCore.dir/kernels/contextAttentionKernels/cuteDslFMHAV2Runner.cpp.o] Error 1
Expected behavior

cpp/kernels/cuteDSLArtifact/aarch64/sm_87/ should ship the fmha_v2_d* CuteDSL kernel modules matching what the v0.10.0 header requires, same as (presumably) the x86 sm_110/sm_121 artifacts do.

System information (Edge Device)

  • Platform: NVIDIA Jetson Orin NX 16GB (Seeed reComputer J4012)
  • Software release: JetPack 7.2, CUDA 13.2
  • CPU architecture: aarch64
  • GPU compute capability: SM87
  • Build type: Release
  • TensorRT Edge-LLM version: v0.10.0 (71dd1bae032e70771265917ec74d3ff4cad07a10)
  • CMake options: -DEMBEDDED_TARGET=jetson-orin -DCUDA_CTK_VERSION=13.2 -DENABLE_CUTE_DSL=ALL (exact platform-recommended command from the installation docs)
Dominant language
Python
Stars
563
Forks
135
Avg merge
14h 13m
Merged PRs (30d)
1

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/TensorRT-Edge-LLM

All issues in NVIDIA/TensorRT-Edge-LLM

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.