instanceNormalizationPlugin: use public __half2float instead of private __internal_half2float
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 88/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- cpp
- Domain
- ai-infra-agents, compilers
Research direction
Start with the kHALF branches in plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.cu and plugin/instanceNormalizationPlugin/instanceNormalizationPluginLegacy.cu, specifically their constructor copyWeights lambdas. Confirm both call sites build with nvcc and a non-nvcc CUDA compiler, and verify that half scale and bias conversion retains its existing numeric behavior.
Written by the indexing model from the issue text.
Description
Both InstanceNorm plugin sources convert half scale/bias weights to float with __internal_half2float, which is a private static inline helper inside cuda_fp16.hpp, not part of the public CUDA API. It compiles under nvcc, where that header brings the helper into scope, but fails under other CUDA-compatible compilers (clang-CUDA, and AMD-targeting CUDA toolchains), and it would also stand in the way of a HIP/ROCm port.
Call sites (current main)
plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.cu, in theInstanceNormalizationV3Plugin(float, Weights const&, Weights const&, int32_t, float)constructor'scopyWeightslambda (thekHALFbranch).plugin/instanceNormalizationPlugin/instanceNormalizationPluginLegacy.cu, in theInstanceNormalizationPlugin(float, Weights const&, Weights const&, int32_t, float)constructor'scopyWeightslambda (thekHALFbranch).
Both read:
auto const value = static_cast<unsigned short const*>(input.values);
output.push_back(__internal_half2float(value[c]));
Under a non-nvcc CUDA compiler this errors with:
error: use of undeclared identifier '__internal_half2float'
Proposed fix (public API, one line per site)
Wrap the raw bits in a __half_raw and call the public __half2float:
auto const value = static_cast<unsigned short const*>(input.values);
__half_raw raw;
raw.x = value[c];
output.push_back(__half2float(raw));
Verified
Minimal reproducer compiled with NVIDIA nvcc (CUDA 13.1, sm_75) and with a clang-based CUDA compiler:
| code | nvcc | non-nvcc CUDA |
|---|---|---|
original (__internal_half2float) |
compiles | fails (undeclared identifier) |
fix (__half_raw + __half2float) |
compiles | compiles |
The public __half2float runs the same conversion the private helper does, so the numeric behaviour is unchanged. The change just drops a dependency on an undocumented nvcc internal and makes the plugin portable to other CUDA compilers, at no cost under nvcc.
Filing this first per CONTRIBUTING (issue before PR). I'm happy to send a PR touching both files, DCO signed off, once this is acknowledged.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/TensorRT
-
Module:Demo
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Module:Documentation
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Wrong FP16 results when a fused convolution, activation and residual add reads a strided concat view OpenModule:Accuracy
Difficulty 5/5 Over a week Newbie friendliness 45/100
-
Module:Accuracy
Difficulty 5/5 Over a week Newbie friendliness 28/100
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
infiniflow/infinity#3502 ·