[Issue]: fused_flatten_mxfp4_quant uses padded segment offsets for non-power-of-two widths
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- machine-learning
Research direction
Read the wrapper in aiter/ops/triton/quant/fused_mxfp4_quant.py around lines 340–352 and the stores in aiter/ops/triton/_triton_kernels/quant/fused_mxfp4_quant.py around lines 465–480. Reproduce with the provided (1, 3, 96) input and check packed and scale segment offsets against logical-width strides. Done when the outputs are 144 bytes of 0x77 and nine scale bytes of 127.
Written by the indexing model from the issue text.
Description
Describe the bug
fused_flatten_mxfp4_quant leaves gaps and writes beyond its outputs when the last input dimension is not a power of two. The script uses shape (1, 3, 96). All values are 6, so each packed output byte must be 119 and each scale byte must be 127. The output instead contains unwritten gaps at segment boundaries.
The wrapper rounds 96 to a 128-element processing block but allocates output for 288 logical elements. The stores use that padded block width to locate each segment. Packed segments start 64 bytes apart instead of 48. Scale segments start four bytes apart instead of three. The last segment consequently extends beyond both output tensors.
The linked lines are unchanged on main as of 2026-11-17.
Reproduce
import torch
from aiter.ops.triton.quant.fused_mxfp4_quant import fused_flatten_mxfp4_quant
dev = "cuda" if torch.cuda.is_available() else "cpu"
torch.set_default_device(dev)
torch.manual_seed(0)
torch.use_deterministic_algorithms(True)
x = torch.full((1, 3, 96), 6.0, dtype=torch.bfloat16)
q, scales = fused_flatten_mxfp4_quant(x)
# Each block has scale 1 (E8M0 byte 127). FP4 value 6 has code 7.
expected_q = torch.full_like(q, 0x77)
expected_scales = torch.full_like(scales, 127)
print('Packed FP4 bytes at segment 1:', q[0, 48:56].tolist())
print('Expected:', expected_q[0, 48:56].tolist())
print('Scale bytes:', scales[0].tolist())
print('Expected:', expected_scales[0].tolist())
assert torch.equal(q, expected_q) and torch.equal(scales, expected_scales), 'Output segments use the padded tile stride.'
Output:
Packed FP4 bytes at segment 1: [255, 255, 255, 255, 255, 255, 255, 255]
Expected: [119, 119, 119, 119, 119, 119, 119, 119]
Scale bytes: [127, 127, 127, 255, 127, 127, 127, 255, 127]
Expected: [127, 127, 127, 127, 127, 127, 127, 127, 127]
AssertionError: Output segments use the padded tile stride.
Expected behavior
Flattening 288 values must produce 144 packed bytes equal to 0x77 and nine E8M0 scale bytes equal to 127. E8M0 byte 127 represents scale 1.
Suggested fix
Use N2 // 2 as the packed segment stride and ceil(N2 / 32) as the scale segment stride. Keep masks based on the logical width.
Versions / environment
AITER ecd79bcb5d8a (2026-11-17).
torch 2.14.1, triton 3.8.0.
Target configuration: gfx942.
Reproduced with Triton's interpreter mode (TRITON_INTERPRET=1).
The root cause is output-index/config logic. The same padded offsets are used on gfx942/gfx950.
The interpreter verification reserves extra backing storage after small outputs. Deterministic allocation fill exposes the unwritten gaps; kernel arithmetic and returned shapes stay unchanged.
- Dominant language
- Python
- Stars
- 616
- Forks
- 709
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 321
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ROCm/aiter
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
Maintainers usually reply within 1 day
-
[Issue]: test_mla_decode_pagesize64 fails instead of skipping on MI450 A0 (B0-only ASM)Possibly taken @feifei14119 claimed this 5 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ROCm/aiter#7779 · 1 assignee ·
Maintainers usually reply within 1 day
Similar issues
-
integration: media_player
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
home-assistant/core#193344 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
daniel-ospina/tortoise#13607 ·
Maintainers usually reply within 1 day
-
documentation priority: medium
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day