[Issue]: fused_rms_fp8_per_tensor_static_quant truncates the wider first input

Open Beginner friendly
#8,143 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
77/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, pytorch

Research direction

Read the launcher in aiter/ops/triton/quant/fused_fp8_quant.py around lines 58–66 and compare its block-width logic with the group-quantization sibling. Then inspect the kernel in aiter/ops/triton/_triton_kernels/quant/fused_fp8_quant.py around lines 96–162 and run the reproduction from the issue. Done means both outputs process every column and normalize using each input’s full-row mean square.

Written by the indexing model from the issue text.

Description

Describe the bug

fused_rms_fp8_per_tensor_static_quant returns wrong normalized values when inp2 is narrower than inp1. With first width 1536 and second width 512, the kernel processes only the first 512 columns of the first input. For inputs filled with one, the first normalized value is about 1.73 instead of 1.0. Columns from 512 onward are not written.

The launcher initially computes the block width from inp1, then replaces that width with the rounded width of inp2. The kernel uses that single block for both inputs. It divides the partial sum by the full first width. The script requests the unquantized first output and enables deterministic allocation fill to expose missing stores.

The linked lines are unchanged on main as of 2026-11-17.

Reproduce
import torch
from aiter.ops.triton.quant.fused_fp8_quant import fused_rms_fp8_per_tensor_static_quant
from aiter.ops.triton.utils.types import get_fp8_e4m3_dtype

dev = "cuda" if torch.cuda.is_available() else "cpu"
torch.set_default_device(dev)
torch.manual_seed(0)
torch.use_deterministic_algorithms(True)
x1 = torch.ones(2, 1536).bfloat16()
x2 = torch.ones(2, 512).bfloat16()
q, out1, out2, _ = fused_rms_fp8_per_tensor_static_quant(
    x1, torch.ones(1536), 1e-6, torch.tensor([0.02]),
    inp2=x2, inp2_weight=torch.ones(512), inp2_epsilon=1e-6,
    dtype_quant=get_fp8_e4m3_dtype(), output_unquantized_inp1=True)
ref = x1.float() * torch.rsqrt(x1.float().square().mean(-1, keepdim=True) + 1e-6)
print('First normalized value:', out1[0, 0].item(), '; expected:', ref[0, 0].item())
print('Normalized value at column 600:', out1[0, 600].item(), '; expected:', ref[0, 600].item())
assert torch.allclose(out1.float(), ref, atol=0.01, rtol=0.01), 'The narrower second input truncates the first input.'

Output:

First normalized value: 1.734375 ; expected: 0.9999995231628418
Normalized value at column 600: nan ; expected: 0.9999995231628418
AssertionError: The narrower second input truncates the first input.
Expected behavior

Normalize every column of each input with its own full-row mean square. Constant-one inputs produce normalized values near 1.0 across both outputs.

Suggested fix

Set the block width to the maximum of the two rounded input widths, as the group-quantization sibling already does.

Versions / environment

AITER ecd79bcb5d8a (2026-11-17).
torch 2.14.1, triton 3.8.0.
Target configuration: gfx942.
Reproduced with Triton's interpreter mode (TRITON_INTERPRET=1).
The root cause is host-side launch-config logic. The same truncated column range is used on gfx942/gfx950.

Dominant language
Python
Stars
616
Forks
709
Avg merge
3d 10h
Merged PRs (30d)
321

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ROCm/aiter

All issues in ROCm/aiter

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.