# [Bug] Row loops `for i in range(0, n_cols, BLOCK_SIZE)` wrap their int32 counter when `n_cols` is just below 2^31 (illegal memory access)

Open
#1,573 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

@Cross2pro is already working on this.

Since Oct 31, 2026.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, pytorch

Research direction

Start by reading the listed kernel loops in src/liger_kernel/ops/cross_entropy.py, fused_ce_tvd.py, fused_linear_kl_div.py, group_norm.py, jsd.py, kl_div.py, tvd.py, utils.py, and vocab_parallel_cross_entropy.py. Check how each loop counter and any loop-carried offsets are typed, then reproduce with the supplied _kldiv_kernel_forward or element_mul_kernel example. Done means the affected loops count safely for dimensions just below 2^31 and the reproduction completes without illegal memory access.

Written by the indexing model from the issue text.

Description

🐛 Describe the bug

Many kernels walk a row with for i in range(0, n_cols, BLOCK_SIZE). Triton passes an int argument as i32 when it fits and as i64 otherwise (specialize_impl), so for n_cols < 2^31 the loop counter is int32. After the last block the counter is BLOCK_SIZE * ceil(n_cols / BLOCK_SIZE), which does not fit in an int32 exactly when

2^31 - BLOCK_SIZE < n_cols < 2^31

It wraps to a negative number and the loop goes on with negative offsets: X_offsets < n_cols still holds, so it loads and stores up to 4 GiB before the row. For n_cols >= 2^31, n_cols itself is passed as an i64, the counter is i64 and the loop is fine. So the failing shapes are the last BLOCK_SIZE - 1 lengths below 2^31; BLOCK_SIZE is 16384 to 65536 in these kernels.

This is unlikely to happen in practice. In all of the kernels below except group_norm, n_cols is the vocabulary size (the local shard for vocab-parallel CE), and today's largest vocabularies are around 2^18 (Gemma 3 has 262,144). Hitting it takes one of these losses on a single row of about 2^31 elements (4 GiB in bf16), with a length that falls in that window. In group_norm, n_cols is (num_channels / num_groups) * H * W for one sample, so it takes num_groups=1 and a sample of about 4 GiB. Still, none of the wrappers rejects such a shape, and when it does happen the result is an out-of-bounds access rather than an error.

Affected loops on current main (781a8ae):

Reproduce

_kldiv_kernel_forward and element_mul_kernel launched the way their wrappers launch them, on one row of n_cols elements (one 4 GiB bf16 buffer; for KL-div it serves as both prediction and target):

import torch
from liger_kernel.ops.kl_div import _kldiv_kernel_forward
from liger_kernel.ops.utils import element_mul_kernel

n_cols = 2**31 - 1
y = torch.full((1, n_cols), -1.0, device="cuda", dtype=torch.bfloat16)
loss = torch.zeros(1, device="cuda", dtype=torch.float32)
_kldiv_kernel_forward[(1,)](y, y.stride(0), y, y.stride(0), loss, loss.stride(0), n_cols, 1e-10,
                            BLOCK_SIZE=16384, log_target=True, reduction=1, num_warps=16)
torch.cuda.synchronize()
# torch.AcceleratorError: CUDA error: an illegal memory access was encountered
x = torch.ones((1, n_cols), device="cuda", dtype=torch.bfloat16)
g = torch.full((1,), 2.0, device="cuda", dtype=torch.bfloat16)
element_mul_kernel[(1,)](x, x.stride(0), g, n_cols, BLOCK_SIZE=32768, num_warps=32)
torch.cuda.synchronize()
# torch.AcceleratorError: CUDA error: an illegal memory access was encountered

With n_cols = 2**31 - BLOCK_SIZE (the last i + BLOCK_SIZE is 2**31 - BLOCK_SIZE, still an int32) both are fine. So are n_cols = 2**31 and 2**31 + 1: there Triton passes n_cols as an i64, and the compiled loop is an scf.for ... : i64 in the TTIR. Each case ran in its own process:

kernel n_cols = 2**31 - BLOCK_SIZE n_cols = 2**31 - 1 n_cols = 2**31, 2**31 + 1 2**31 - 1, int64 counter
_kldiv_kernel_forward (BLOCK_SIZE=16384) ✅ loss 0.0 ❌ illegal memory access ✅ loss 0.0 ✅ loss 0.0
element_mul_kernel (BLOCK_SIZE=32768) ✅ every element doubled ❌ illegal memory access ✅ every element doubled ✅ every element doubled

The fix is to count in int64:

-    for i in range(0, n_cols, BLOCK_SIZE):
+    for i in range(0, n_cols.to(tl.int64), BLOCK_SIZE):

and likewise for the other loops above (tl.range too). In _kldiv_kernel_backward the loop also reassigns offsets, so its value before the loop needs the same type (offsets = tl.arange(0, BLOCK_SIZE).to(tl.int64)), or Triton rejects the loop-carried type change. The full change for all the kernels listed above is attached as a patch.

Versions

torch 2.9.1+cu128, triton 3.5.1, CUDA 12.8, NVIDIA GeForce RTX 4060 Ti (8 GB), driver 595.91.07.

Dominant language
Python
Stars
6.7k
Forks
656
Avg merge
4d 12h
Merged PRs (30d)
27

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from linkedin/Liger-Kernel

All issues in linkedin/Liger-Kernel

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.