Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

docs: Jetson build prerequisites omit the NVRTC headers (cuda-nvrtc-dev) and the TensorRT plugin headers

Open Beginner friendly
  • Is something wrong?
#244 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 5 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
cmake, cpp, linux
Domain
build-system, documentation

Research direction

The Jetson package list is in docs/source/user_guide/getting_started/installation.md around lines 39-44, under the source workflow for the C++ runtime. The NVRTC requirement is enforced in cmake/FindNVRTC.cmake (lines 112-117), and cpp/plugins/qsaAttentionPlugin/qsaAttentionPlugin.h:20 includes NvInferPlugin.h. Start by adding cuda-nvrtc-dev-13-2 and libnvinfer-headers-plugin-dev to the Jetson install step, with the CUDA suffix matching JetPack. Done means the step lists both packages and a runtime note for cuda-nvrtc-13-2 is included.

Written by the indexing model from the issue text.

Description

Describe the bug

The installation guide's package step for Jetson installs only cmake, build-essential and git (docs/source/user_guide/getting_started/installation.md:39-44 at 0f52705). Two more packages are required to build on JetPack 7.2 and are not mentioned:

  1. cuda-nvrtc-dev-13-2. cmake/FindNVRTC.cmake:112-117 is fatal without nvrtc.h: 
    Plugin build-time NVRTC JIT requires nvrtc.h and libnvrtc.so or libnvrtc_static.a. Set NVRTC_ROOT_DIR or CUDA_DIR/CUDA_TARGET_DIR to a CUDA Toolkit with NVRTC.
    The message points at CMake variables; the fix on a device is one apt package. My Jetson Orin NX on JetPack 7.2 did not have the header until I installed it. I have not checked a freshly flashed image, so I do not know whether a default JetPack install includes it.
  2. libnvinfer-headers-plugin-dev. cpp/plugins/qsaAttentionPlugin/qsaAttentionPlugin.h:20 includes <NvInferPlugin.h>, which lives in that package, not in libnvinfer-dev. A full JetPack image gets it through the tensorrt-dev metapackage; a minimal install (for example a container built from the Jetson apt repository with libnvinfer-dev only) stops at 
    cpp/plugins/qsaAttentionPlugin/qsaAttentionPlugin.h:20: fatal error: NvInferPlugin.h: No such file or directory
    Nothing links libnvinfer_plugin; the header alone is needed.

Impact: nice to have; each costs a failed configure or compile and a search.

Steps/Code to reproduce bug
  1. On a JetPack 7.2 Jetson Orin without cuda-nvrtc-dev-13-2 installed (dpkg -l cuda-nvrtc-dev-13-2), follow installation.md "Source workflow: C++ runtime" steps 1-4 as written.
  2. cmake stops with the NVRTC error above. sudo apt install cuda-nvrtc-dev-13-2 fixes it.
  3. In a minimal environment with libnvinfer-dev but without tensorrt-dev, make stops at the NvInferPlugin.h error. sudo apt install libnvinfer-headers-plugin-dev fixes it.

Build configuration:

cmake .. -DCMAKE_BUILD_TYPE=Release -DTRT_PACKAGE_DIR=/usr -DCMAKE_TOOLCHAIN_FILE=cmake/aarch64_linux_toolchain.cmake -DEMBEDDED_TARGET=jetson-orin -DCUDA_CTK_VERSION=13.2 -DENABLE_CUTE_DSL=ALL

Runtime command used:

# not applicable: configure/compile time
Expected behavior

Step 1 of the Jetson instructions lists the two packages (with the CUDA version suffix matching the JetPack release), and ideally the FindNVRTC.cmake fatal message names cuda-nvrtc-dev-<ver> for Jetson. I can open a small docs PR once this is approved.

System information (Edge Device)

  • Platform (e.g., NVIDIA Jetson Thor): NVIDIA Jetson Orin NX 16 GB (header gap also reproduced in an aarch64 container built from the Jetson r39.2 apt repository)
  • Software release (e.g., JetPack 7.1): JetPack 7.2 (L4T R39.2)
  • CPU architecture: aarch64
  • GPU compute capability (e.g., SM110 for Jetson Thor): SM87
  • Total device memory: 16 GB (unified)
  • Build type (e.g., Release, Debug): Release
  • Library versions:
    • TensorRT Edge-LLM version or commit hash: 0f52705 (docs and cmake read at this commit); the NVRTC gap was hit building 0.10.1 on the device
    • CUDA: 13.2 (cuda-nvrtc-dev-13-2 13.2.86-1)
    • TensorRT: 10.16.2 (libnvinfer-headers-plugin-dev 10.16.2.10-1+cuda13.2)
    • C++ compiler (e.g., GCC 11.4): GCC 13.3.0
  • CMake options used:
    • CMAKE_TOOLCHAIN_FILE: cmake/aarch64_linux_toolchain.cmake
    • EMBEDDED_TARGET: jetson-orin
    • TRT_PACKAGE_DIR: /usr
  • Any other details that may help: the built plugin links libnvrtc.so.13 at run time, so the runtime package cuda-nvrtc-13-2 is needed on deployment devices as well; a sentence on that would help users who copy build/ to another device.
Dominant language
Python
Stars
590
Forks
143
Avg merge
3d 18h
Merged PRs (30d)
2

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/TensorRT-Edge-LLM

All issues in NVIDIA/TensorRT-Edge-LLM

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.