Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Qwen3.8-27B NVFP4 base engine build fails: Myelin removeEmptyProducersFromSubgraph not implemented for PLUGIN_V3 (Jetson Thor, TRT 10.13.3.9 and 10.14.1.48)

Open
  • Is something wrong?
#192 4 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 5 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
python
Domain
ai, embedded-iot

Research direction

Reproduce the failure with tensorrt-edgellm-export and llm_build using the reported Qwen3.8-27B checkpoint and TensorRT versions. Compare the base and MTP draft ONNX graphs, especially the gated_delta_net and causal_conv1d PLUGIN_V3 nodes, then determine whether a supported TensorRT version or builder workaround avoids the Myelin rewrite failure.

Written by the indexing model from the issue text.

Description

Environment

  • Jetson AGX Thor (sm_110), L4T R38.4 (JetPack 7.1-class), CUDA 13.0
  • TensorRT-Edge-LLM 0.10.0 (bb29145)
  • TensorRT tested: 10.13.3.9-1+cuda13.0 (JetPack repo) AND 10.14.1.48-1+cuda13.0 (sbsa CUDA network repo) — same failure on both

Repro

Checkpoint: NVFP4 Qwen3.8-27B (ModelOpt NVFP4, group_size 16, MTP extra tensors), e.g. https://huggingface.co/lyf/Qwen3.8-27B-Heretic-ARA-NVFP4-MTP-VL

  1. tensorrt-edgellm-export CKPT OUT --mtp → succeeds (after locally patching two exporter issues: (a) system scipy 1.8 vs numpy 2.x — from numpy import Inf ImportError when the venv uses --system-site-packages; (b) GDN fusion concatenating single-element weight_scale_2/input_scale buffers into a 4-element vector, which TRT's ONNX parser rejects: importTRT_FP4DynamicQuantize: Scale input must be a scalar — fixed by keeping numel==1 buffers unconcatenated in fuse_gdn_input_projections).
  2. llm_build --onnxDir OUT/llm --engineDir ENG --maxBatchSize 1 --maxInputLen 4096 --maxKVCacheCapacity 32768 --maxVerifyTreeSize 4 --specBase →
[ERROR] IBuilder::buildSerializedNetwork: Error Code 1: Internal Error
(Not implemented for node type PLUGIN_V3. In removeEmptyProducersFromSubgraph
at optimizer/myelin/rewrite/removeEmptyTensors.cpp:257)   // TRT 10.13
... removeEmptyTensors.cpp:306                              // TRT 10.14
  1. The MTP draft engine (--specDraft on OUT/mtp_draft) builds fine on both TRT versions. The draft ONNX contains only trt_edgellm::AttentionPlugin x1; the base ONNX additionally has trt_edgellm::gated_delta_net x48 and trt_edgellm::causal_conv1d x48, so the GDN plugins appear to be the PLUGIN_V3 nodes Myelin cannot handle.
  2. The experimental direct (ONNX-less) builder fails identically at the same Myelin pass, so this is not ONNX-path specific.

Questions

  • Which TRT version was Qwen3.8-27B Day-0 support validated against on Thor? Does it require JetPack 7.2 (CUDA 13.2 / TRT 10.16)?
  • Is there a builder flag/env to route GDN plugin nodes around the failing Myelin rewrite pass?
Dominant language
Python
Stars
590
Forks
143
Avg merge
3d 18h
Merged PRs (30d)
2

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/TensorRT-Edge-LLM

All issues in NVIDIA/TensorRT-Edge-LLM

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.