[Issue]: fused_rms_fp8_per_tensor_static_quant truncates the wider first input
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 77/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- machine-learning
Research direction
Read the launcher in aiter/ops/triton/quant/fused_fp8_quant.py around lines 58–66 and compare its block-width logic with the group-quantization sibling. Then inspect the kernel in aiter/ops/triton/_triton_kernels/quant/fused_fp8_quant.py around lines 96–162 and run the reproduction from the issue. Done means both outputs process every column and normalize using each input’s full-row mean square.
Written by the indexing model from the issue text.
Description
Describe the bug
fused_rms_fp8_per_tensor_static_quant returns wrong normalized values when inp2 is narrower than inp1. With first width 1536 and second width 512, the kernel processes only the first 512 columns of the first input. For inputs filled with one, the first normalized value is about 1.73 instead of 1.0. Columns from 512 onward are not written.
The launcher initially computes the block width from inp1, then replaces that width with the rounded width of inp2. The kernel uses that single block for both inputs. It divides the partial sum by the full first width. The script requests the unquantized first output and enables deterministic allocation fill to expose missing stores.
The linked lines are unchanged on main as of 2026-11-17.
Reproduce
import torch
from aiter.ops.triton.quant.fused_fp8_quant import fused_rms_fp8_per_tensor_static_quant
from aiter.ops.triton.utils.types import get_fp8_e4m3_dtype
dev = "cuda" if torch.cuda.is_available() else "cpu"
torch.set_default_device(dev)
torch.manual_seed(0)
torch.use_deterministic_algorithms(True)
x1 = torch.ones(2, 1536).bfloat16()
x2 = torch.ones(2, 512).bfloat16()
q, out1, out2, _ = fused_rms_fp8_per_tensor_static_quant(
x1, torch.ones(1536), 1e-6, torch.tensor([0.02]),
inp2=x2, inp2_weight=torch.ones(512), inp2_epsilon=1e-6,
dtype_quant=get_fp8_e4m3_dtype(), output_unquantized_inp1=True)
ref = x1.float() * torch.rsqrt(x1.float().square().mean(-1, keepdim=True) + 1e-6)
print('First normalized value:', out1[0, 0].item(), '; expected:', ref[0, 0].item())
print('Normalized value at column 600:', out1[0, 600].item(), '; expected:', ref[0, 600].item())
assert torch.allclose(out1.float(), ref, atol=0.01, rtol=0.01), 'The narrower second input truncates the first input.'
Output:
First normalized value: 1.734375 ; expected: 0.9999995231628418
Normalized value at column 600: nan ; expected: 0.9999995231628418
AssertionError: The narrower second input truncates the first input.
Expected behavior
Normalize every column of each input with its own full-row mean square. Constant-one inputs produce normalized values near 1.0 across both outputs.
Suggested fix
Set the block width to the maximum of the two rounded input widths, as the group-quantization sibling already does.
Versions / environment
AITER ecd79bcb5d8a (2026-11-17).
torch 2.14.1, triton 3.8.0.
Target configuration: gfx942.
Reproduced with Triton's interpreter mode (TRITON_INTERPRET=1).
The root cause is host-side launch-config logic. The same truncated column range is used on gfx942/gfx950.
- Dominant language
- Python
- Stars
- 616
- Forks
- 709
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 321
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ROCm/aiter
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
Maintainers usually reply within 1 day
-
[Issue]: test_mla_decode_pagesize64 fails instead of skipping on MI450 A0 (B0-only ASM)Possibly taken @feifei14119 claimed this 5 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ROCm/aiter#7779 · 1 assignee ·
Maintainers usually reply within 1 day
Similar issues
-
integration: media_player
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
home-assistant/core#193344 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
daniel-ospina/tortoise#13607 ·
Maintainers usually reply within 1 day
-
documentation priority: medium
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day