[Issue]: fused_flatten_mxfp4_quant uses padded segment offsets for non-power-of-two widths

Open Beginner friendly
#8,144 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, pytorch

Research direction

Read the wrapper in aiter/ops/triton/quant/fused_mxfp4_quant.py around lines 340–352 and the stores in aiter/ops/triton/_triton_kernels/quant/fused_mxfp4_quant.py around lines 465–480. Reproduce with the provided (1, 3, 96) input and check packed and scale segment offsets against logical-width strides. Done when the outputs are 144 bytes of 0x77 and nine scale bytes of 127.

Written by the indexing model from the issue text.

Description

Describe the bug

fused_flatten_mxfp4_quant leaves gaps and writes beyond its outputs when the last input dimension is not a power of two. The script uses shape (1, 3, 96). All values are 6, so each packed output byte must be 119 and each scale byte must be 127. The output instead contains unwritten gaps at segment boundaries.

The wrapper rounds 96 to a 128-element processing block but allocates output for 288 logical elements. The stores use that padded block width to locate each segment. Packed segments start 64 bytes apart instead of 48. Scale segments start four bytes apart instead of three. The last segment consequently extends beyond both output tensors.

The linked lines are unchanged on main as of 2026-11-17.

Reproduce
import torch
from aiter.ops.triton.quant.fused_mxfp4_quant import fused_flatten_mxfp4_quant

dev = "cuda" if torch.cuda.is_available() else "cpu"
torch.set_default_device(dev)
torch.manual_seed(0)
torch.use_deterministic_algorithms(True)
x = torch.full((1, 3, 96), 6.0, dtype=torch.bfloat16)
q, scales = fused_flatten_mxfp4_quant(x)
# Each block has scale 1 (E8M0 byte 127). FP4 value 6 has code 7.
expected_q = torch.full_like(q, 0x77)
expected_scales = torch.full_like(scales, 127)
print('Packed FP4 bytes at segment 1:', q[0, 48:56].tolist())
print('Expected:', expected_q[0, 48:56].tolist())
print('Scale bytes:', scales[0].tolist())
print('Expected:', expected_scales[0].tolist())
assert torch.equal(q, expected_q) and torch.equal(scales, expected_scales), 'Output segments use the padded tile stride.'

Output:

Packed FP4 bytes at segment 1: [255, 255, 255, 255, 255, 255, 255, 255]
Expected: [119, 119, 119, 119, 119, 119, 119, 119]
Scale bytes: [127, 127, 127, 255, 127, 127, 127, 255, 127]
Expected: [127, 127, 127, 127, 127, 127, 127, 127, 127]
AssertionError: Output segments use the padded tile stride.
Expected behavior

Flattening 288 values must produce 144 packed bytes equal to 0x77 and nine E8M0 scale bytes equal to 127. E8M0 byte 127 represents scale 1.

Suggested fix

Use N2 // 2 as the packed segment stride and ceil(N2 / 32) as the scale segment stride. Keep masks based on the logical width.

Versions / environment

AITER ecd79bcb5d8a (2026-11-17).
torch 2.14.1, triton 3.8.0.
Target configuration: gfx942.
Reproduced with Triton's interpreter mode (TRITON_INTERPRET=1).
The root cause is output-index/config logic. The same padded offsets are used on gfx942/gfx950.
The interpreter verification reserves extra backing storage after small outputs. Deterministic allocation fill exposes the unwritten gaps; kernel arithmetic and returned shapes stay unchanged.

Dominant language
Python
Stars
616
Forks
709
Avg merge
3d 10h
Merged PRs (30d)
321

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ROCm/aiter

All issues in ROCm/aiter

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.