# [Bug] `_l2_swizzle` in mlp.py counts tiles in int32 (wrong tile mapping when `S` or `hidden_dim` is within a block of 2^31)

Open Beginner friendly
#1,658 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
77/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python

Research direction

Read src/liger_kernel/ops/mlp.py, especially _l2_swizzle at line 33 and the four SwiGLU kernel call sites listed in the issue. Start by checking the tile count arithmetic against the reproduction script’s Python integer reference for dimensions near 2^31. Done means the swizzle maps every tested program id to the reference tile, including the four boundary shapes described.

Written by the indexing model from the issue text.

Description

🐛 Describe the bug

_l2_swizzle in mlp.py:33 maps a program id to its output tile (pid_m, pid_n). It starts from

num_pid_m = tl.cdiv(M, BLOCK_M)
num_pid_n = tl.cdiv(N, BLOCK_N)

where M and N are the kernel's int32 shape arguments. tl.cdiv(x, d) is (x + d - 1) // d, so once M > 2**31 - BLOCK_M (or N > 2**31 - BLOCK_N) the sum wraps to a negative number. The group arithmetic below it then hands out the wrong tiles: some tiles are computed twice, others never, and some programs get a negative pid_m, so their descriptor loads and stores fall outside the tensor.

The SwiGLU MLP kernels all go through it:

The launch grid is computed on the host with Python ints (triton.cdiv(S, BLOCK_M) * triton.cdiv(hidden_dim, BLOCK_N)), so the launch itself goes through and nothing reports an error; the output just has missing and duplicated tiles.

To be upfront: this only bites when one of the dimensions is within BLOCK_M/BLOCK_N of 2^31 (in the runs below the other one is 1024), and the outputs [B, S, hidden_dim] are then terabytes, so the full kernels cannot be run at these shapes on current GPUs. It is the same 32-bit index arithmetic as #804, #1482, #1497 and #1570, where offsets were widened to int64, so it seemed worth widening here too.

Reproduce

The kernels themselves can't be launched at these sizes, so this runs _l2_swizzle (copied unchanged) for the first and last 16 program ids of the grid the wrapper would launch, and compares with the same arithmetic on Python ints:

import torch
import triton
import triton.language as tl


@triton.jit
def _l2_swizzle(pid, M, N, BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, GROUP_SIZE_M: tl.constexpr):
    # copied from src/liger_kernel/ops/mlp.py
    num_pid_m = tl.cdiv(M, BLOCK_M)
    num_pid_n = tl.cdiv(N, BLOCK_N)
    num_pid_in_group = GROUP_SIZE_M * num_pid_n
    group_id = pid // num_pid_in_group
    base_pid_m = group_id * GROUP_SIZE_M
    effective_group_size_m = tl.minimum(num_pid_m - base_pid_m, GROUP_SIZE_M)
    pid_in_group = pid % num_pid_in_group
    pid_m = pid_in_group % effective_group_size_m + base_pid_m
    pid_n = pid_in_group // effective_group_size_m
    return pid_m, pid_n


@triton.jit
def swizzle_kernel(pids, out, S, hidden_dim, BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, GROUP_SIZE_M: tl.constexpr):
    i = tl.program_id(0)
    pid_m, pid_n = _l2_swizzle(tl.load(pids + i), S, hidden_dim, BLOCK_M, BLOCK_N, GROUP_SIZE_M)
    tl.store(out + 2 * i, pid_m)
    tl.store(out + 2 * i + 1, pid_n)


def reference(pid, M, N, BLOCK_M, BLOCK_N, GROUP_SIZE_M):  # the same, on Python ints
    num_pid_m, num_pid_n = -(-M // BLOCK_M), -(-N // BLOCK_N)
    num_pid_in_group = GROUP_SIZE_M * num_pid_n
    base_pid_m = pid // num_pid_in_group * GROUP_SIZE_M
    effective_group_size_m = min(num_pid_m - base_pid_m, GROUP_SIZE_M)
    pid_in_group = pid % num_pid_in_group
    return pid_in_group % effective_group_size_m + base_pid_m, pid_in_group // effective_group_size_m


BLOCK_M, BLOCK_N, GROUP_SIZE_M = 64, 128, 8
for S, hidden_dim in [(2**31 - 1, 1024), (1024, 2**31 - 1), (2**31 - BLOCK_M, 1024), (1024, 2**31 - BLOCK_N)]:
    grid = triton.cdiv(S, BLOCK_M) * triton.cdiv(hidden_dim, BLOCK_N)  # as the wrapper launches it
    pids = list(range(16)) + list(range(grid - 16, grid))
    out = torch.empty(2 * len(pids), device="cuda", dtype=torch.int32)
    swizzle_kernel[(len(pids),)](torch.tensor(pids, device="cuda", dtype=torch.int32), out, S, hidden_dim,
                                 BLOCK_M=BLOCK_M, BLOCK_N=BLOCK_N, GROUP_SIZE_M=GROUP_SIZE_M)
    got = [tuple(t) for t in out.view(-1, 2).tolist()]
    wrong = [(p, g, reference(p, S, hidden_dim, BLOCK_M, BLOCK_N, GROUP_SIZE_M))
             for p, g in zip(pids, got) if g != reference(p, S, hidden_dim, BLOCK_M, BLOCK_N, GROUP_SIZE_M)]
    print(f"S={S}, hidden_dim={hidden_dim}: {len(wrong)}/{len(pids)} tiles wrong", wrong[:2])
S=2147483647, hidden_dim=1024: 24/32 tiles wrong [(8, (8, 0), (0, 1)), (9, (9, 0), (1, 1))]
S=1024, hidden_dim=2147483647: 16/32 tiles wrong [(268435440, (-16, 0), (8, 16777214)), (268435441, (-15, 0), (9, 16777214))]
S=2147483584, hidden_dim=1024: 0/32 tiles wrong []
S=1024, hidden_dim=2147483520: 0/32 tiles wrong []

Program 8 should compute tile (0, 1) but computes (8, 0), which is program 64's tile; near the end of the second grid pid_m comes out negative. With S = 2**31 - BLOCK_M and hidden_dim = 2**31 - BLOCK_N (the largest values whose cdiv still fits in int32) every tile is right.

The fix is to count the tiles in int64 and narrow the result again (pid_m < cdiv(S, BLOCK_M), so pid_m * BLOCK_M and the descriptor offsets stay int32):

 @triton.jit
 def _l2_swizzle(pid, M, N, BLOCK_M: tl.constexpr, BLOCK_N: tl.constexpr, GROUP_SIZE_M: tl.constexpr):
-    num_pid_m = tl.cdiv(M, BLOCK_M)
-    num_pid_n = tl.cdiv(N, BLOCK_N)
+    # int64: M + BLOCK_M - 1 overflows int32 once M > 2**31 - BLOCK_M
+    num_pid_m = tl.cdiv(M.to(tl.int64), BLOCK_M)
+    num_pid_n = tl.cdiv(N.to(tl.int64), BLOCK_N)
     num_pid_in_group = GROUP_SIZE_M * num_pid_n
     group_id = pid // num_pid_in_group
     base_pid_m = group_id * GROUP_SIZE_M
@@
     pid_in_group = pid % num_pid_in_group
     pid_m = pid_in_group % effective_group_size_m + base_pid_m
     pid_n = pid_in_group // effective_group_size_m
-    return pid_m, pid_n
+    return pid_m.to(tl.int32), pid_n.to(tl.int32)

With this change the script above prints 0/32 tiles wrong for all four shapes. The patch is attached.

Versions

torch 2.9.1+cu128, triton 3.5.1, CUDA 12.8, NVIDIA GeForce RTX 4060 Ti (8 GB), driver 595.91.07. Liger-Kernel main @ 781a8ae (_l2_swizzle is the same at 488acbd).

Dominant language
Python
Stars
6.7k
Forks
656
Avg merge
4d 12h
Merged PRs (30d)
27

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from linkedin/Liger-Kernel

All issues in linkedin/Liger-Kernel

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.