# [Bug] `_l2_swizzle` in mlp.py counts tiles in int32 (wrong tile mapping when `S` or `hidden_dim` is within a block of 2^31)
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 77/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning, performance
Research direction
Read src/liger_kernel/ops/mlp.py, especially _l2_swizzle at line 33 and the four SwiGLU kernel call sites listed in the issue. Start by checking the tile count arithmetic against the reproduction script’s Python integer reference for dimensions near 2^31. Done means the swizzle maps every tested program id to the reference tile, including the four boundary shapes described.
Written by the indexing model from the issue text.
Description
🐛 Describe the bug
_l2_swizzle in mlp.py:33 maps a program id to its output tile (pid_m, pid_n). It starts from
num_pid_m = tl.cdiv(M, BLOCK_M)
num_pid_n = tl.cdiv(N, BLOCK_N)
where M and N are the kernel's int32 shape arguments. tl.cdiv(x, d) is (x + d - 1) // d, so once M > 2**31 - BLOCK_M (or N > 2**31 - BLOCK_N) the sum wraps to a negative number. The group arithmetic below it then hands out the wrong tiles: some tiles are computed twice, others never, and some programs get a negative pid_m, so their descriptor loads and stores fall outside the tensor.
The SwiGLU MLP kernels all go through it:
_swiglu_kernel_forwardL104 and_swiglu_kernel_forward_inferenceL196:_l2_swizzle(pid, S, hidden_dim, ...)_swiglu_kernel_backward_dGUL263:_l2_swizzle(pid, S, hidden_dim, ...)_swiglu_kernel_backward_dIL362:_l2_swizzle(pid, S, dim, ...)
The launch grid is computed on the host with Python ints (triton.cdiv(S, BLOCK_M) * triton.cdiv(hidden_dim, BLOCK_N)), so the launch itself goes through and nothing reports an error; the output just has missing and duplicated tiles.
To be upfront: this only bites when one of the dimensions is within BLOCK_M/BLOCK_N of 2^31 (in the runs below the other one is 1024), and the outputs [B, S, hidden_dim] are then terabytes, so the full kernels cannot be run at these shapes on current GPUs. It is the same 32-bit index arithmetic as #804, #1482, #1497 and #1570, where offsets were widened to int64, so it seemed worth widening here too.
Reproduce
The kernels themselves can't be launched at these sizes, so this runs _l2_swizzle (copied unchanged) for the first and last 16 program ids of the grid the wrapper would launch, and compares with the same arithmetic on Python ints:
import torch
import triton
import triton.language as tl
@triton.jit
def _l2_swizzle(pid, M, N, BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, GROUP_SIZE_M: tl.constexpr):
# copied from src/liger_kernel/ops/mlp.py
num_pid_m = tl.cdiv(M, BLOCK_M)
num_pid_n = tl.cdiv(N, BLOCK_N)
num_pid_in_group = GROUP_SIZE_M * num_pid_n
group_id = pid // num_pid_in_group
base_pid_m = group_id * GROUP_SIZE_M
effective_group_size_m = tl.minimum(num_pid_m - base_pid_m, GROUP_SIZE_M)
pid_in_group = pid % num_pid_in_group
pid_m = pid_in_group % effective_group_size_m + base_pid_m
pid_n = pid_in_group // effective_group_size_m
return pid_m, pid_n
@triton.jit
def swizzle_kernel(pids, out, S, hidden_dim, BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, GROUP_SIZE_M: tl.constexpr):
i = tl.program_id(0)
pid_m, pid_n = _l2_swizzle(tl.load(pids + i), S, hidden_dim, BLOCK_M, BLOCK_N, GROUP_SIZE_M)
tl.store(out + 2 * i, pid_m)
tl.store(out + 2 * i + 1, pid_n)
def reference(pid, M, N, BLOCK_M, BLOCK_N, GROUP_SIZE_M): # the same, on Python ints
num_pid_m, num_pid_n = -(-M // BLOCK_M), -(-N // BLOCK_N)
num_pid_in_group = GROUP_SIZE_M * num_pid_n
base_pid_m = pid // num_pid_in_group * GROUP_SIZE_M
effective_group_size_m = min(num_pid_m - base_pid_m, GROUP_SIZE_M)
pid_in_group = pid % num_pid_in_group
return pid_in_group % effective_group_size_m + base_pid_m, pid_in_group // effective_group_size_m
BLOCK_M, BLOCK_N, GROUP_SIZE_M = 64, 128, 8
for S, hidden_dim in [(2**31 - 1, 1024), (1024, 2**31 - 1), (2**31 - BLOCK_M, 1024), (1024, 2**31 - BLOCK_N)]:
grid = triton.cdiv(S, BLOCK_M) * triton.cdiv(hidden_dim, BLOCK_N) # as the wrapper launches it
pids = list(range(16)) + list(range(grid - 16, grid))
out = torch.empty(2 * len(pids), device="cuda", dtype=torch.int32)
swizzle_kernel[(len(pids),)](torch.tensor(pids, device="cuda", dtype=torch.int32), out, S, hidden_dim,
BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, GROUP_SIZE_M=GROUP_SIZE_M)
got = [tuple(t) for t in out.view(-1, 2).tolist()]
wrong = [(p, g, reference(p, S, hidden_dim, BLOCK_M, BLOCK_N, GROUP_SIZE_M))
for p, g in zip(pids, got) if g != reference(p, S, hidden_dim, BLOCK_M, BLOCK_N, GROUP_SIZE_M)]
print(f"S={S}, hidden_dim={hidden_dim}: {len(wrong)}/{len(pids)} tiles wrong", wrong[:2])
S=2147483647, hidden_dim=1024: 24/32 tiles wrong [(8, (8, 0), (0, 1)), (9, (9, 0), (1, 1))]
S=1024, hidden_dim=2147483647: 16/32 tiles wrong [(268435440, (-16, 0), (8, 16777214)), (268435441, (-15, 0), (9, 16777214))]
S=2147483584, hidden_dim=1024: 0/32 tiles wrong []
S=1024, hidden_dim=2147483520: 0/32 tiles wrong []
Program 8 should compute tile (0, 1) but computes (8, 0), which is program 64's tile; near the end of the second grid pid_m comes out negative. With S = 2**31 - BLOCK_M and hidden_dim = 2**31 - BLOCK_N (the largest values whose cdiv still fits in int32) every tile is right.
The fix is to count the tiles in int64 and narrow the result again (pid_m < cdiv(S, BLOCK_M), so pid_m * BLOCK_M and the descriptor offsets stay int32):
@triton.jit
def _l2_swizzle(pid, M, N, BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, GROUP_SIZE_M: tl.constexpr):
- num_pid_m = tl.cdiv(M, BLOCK_M)
- num_pid_n = tl.cdiv(N, BLOCK_N)
+ # int64: M + BLOCK_M - 1 overflows int32 once M > 2**31 - BLOCK_M
+ num_pid_m = tl.cdiv(M.to(tl.int64), BLOCK_M)
+ num_pid_n = tl.cdiv(N.to(tl.int64), BLOCK_N)
num_pid_in_group = GROUP_SIZE_M * num_pid_n
group_id = pid // num_pid_in_group
base_pid_m = group_id * GROUP_SIZE_M
@@
pid_in_group = pid % num_pid_in_group
pid_m = pid_in_group % effective_group_size_m + base_pid_m
pid_n = pid_in_group // effective_group_size_m
- return pid_m, pid_n
+ return pid_m.to(tl.int32), pid_n.to(tl.int32)
With this change the script above prints 0/32 tiles wrong for all four shapes. The patch is attached.
Versions
torch 2.9.1+cu128, triton 3.5.1, CUDA 12.8, NVIDIA GeForce RTX 4060 Ti (8 GB), driver 595.91.07. Liger-Kernel main @ 781a8ae (_l2_swizzle is the same at 488acbd).
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 656
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 27
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from linkedin/Liger-Kernel
-
LigerPolyNorm defaults to in_place=True (docstring says False) and overwrites shared gradientsPossibly taken @nazek52 claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
linkedin/Liger-Kernel#1647 ·
Maintainers usually reply within 3 days
-
[Bug] Experimental embedding kernels overflow int32 row offsets (illegal memory access past 2^31 output elements)Possibly taken @0xDexe claimed this 8 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
linkedin/Liger-Kernel#1570 · 3 comments ·
Maintainers usually reply within 3 days
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
linkedin/Liger-Kernel#1661 ·
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
linkedin/Liger-Kernel#1660 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
linkedin/Liger-Kernel#1659 ·
Maintainers usually reply within 3 days
All issues in linkedin/Liger-Kernel
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
GoogleCloudPlatform/professional-services-data-validator#1899 ·
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
mohanraj00/entelos#200 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
sweep
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
bketelsen/truenas-mcp#77 ·
Maintainers usually reply within 1 day
-
Console writes URL credentials to the local log, though AGENTS.md says secrets never reach diskOpen
Difficulty 1/5 Under an hour Newbie friendliness 86/100
sandbox-quantum/switch#2163 ·
Maintainers usually reply within 1 day