# [Bug] Row loops `for i in range(0, n_cols, BLOCK_SIZE)` wrap their int32 counter when `n_cols` is just below 2^31 (illegal memory access)
Maintainers usually reply within 3 days
@Cross2pro is already working on this.
Since Oct 31, 2026.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 55/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- machine-learning
Research direction
Start by reading the listed kernel loops in src/liger_kernel/ops/cross_entropy.py, fused_ce_tvd.py, fused_linear_kl_div.py, group_norm.py, jsd.py, kl_div.py, tvd.py, utils.py, and vocab_parallel_cross_entropy.py. Check how each loop counter and any loop-carried offsets are typed, then reproduce with the supplied _kldiv_kernel_forward or element_mul_kernel example. Done means the affected loops count safely for dimensions just below 2^31 and the reproduction completes without illegal memory access.
Written by the indexing model from the issue text.
Description
🐛 Describe the bug
Many kernels walk a row with for i in range(0, n_cols, BLOCK_SIZE). Triton passes an int argument as i32 when it fits and as i64 otherwise (specialize_impl), so for n_cols < 2^31 the loop counter is int32. After the last block the counter is BLOCK_SIZE * ceil(n_cols / BLOCK_SIZE), which does not fit in an int32 exactly when
2^31 - BLOCK_SIZE < n_cols < 2^31
It wraps to a negative number and the loop goes on with negative offsets: X_offsets < n_cols still holds, so it loads and stores up to 4 GiB before the row. For n_cols >= 2^31, n_cols itself is passed as an i64, the counter is i64 and the loop is fine. So the failing shapes are the last BLOCK_SIZE - 1 lengths below 2^31; BLOCK_SIZE is 16384 to 65536 in these kernels.
This is unlikely to happen in practice. In all of the kernels below except group_norm, n_cols is the vocabulary size (the local shard for vocab-parallel CE), and today's largest vocabularies are around 2^18 (Gemma 3 has 262,144). Hitting it takes one of these losses on a single row of about 2^31 elements (4 GiB in bf16), with a length that falls in that window. In group_norm, n_cols is (num_channels / num_groups) * H * W for one sample, so it takes num_groups=1 and a sample of about 4 GiB. Still, none of the wrappers rejects such a shape, and when it does happen the result is an out-of-bounds access rather than an error.
Affected loops on current main (781a8ae):
cross_entropy.py:liger_cross_entropy_kernelL106, L151, L207fused_ce_tvd.py: L88, L115 (forward), L181, L192 (backward)fused_linear_kl_div.py:_kl_div_kernelL68, L78, L92, L108group_norm.py:_group_norm_forward_kernelL64, L85 (overhidden_size)jsd.py:_jsd_kernelL48, L53kl_div.py:_kldiv_kernel_forwardL76,_kldiv_kernel_backwardL116tvd.py:_tv_distance_kernelL77, L86utils.py:element_mul_kernelL140vocab_parallel_cross_entropy.py: L87, L108 (forward), L149, L159 (backward)
Reproduce
_kldiv_kernel_forward and element_mul_kernel launched the way their wrappers launch them, on one row of n_cols elements (one 4 GiB bf16 buffer; for KL-div it serves as both prediction and target):
import torch
from liger_kernel.ops.kl_div import _kldiv_kernel_forward
from liger_kernel.ops.utils import element_mul_kernel
n_cols = 2**31 - 1
y = torch.full((1, n_cols), -1.0, device="cuda", dtype=torch.bfloat16)
loss = torch.zeros(1, device="cuda", dtype=torch.float32)
_kldiv_kernel_forward[(1,)](y, y.stride(0), y, y.stride(0), loss, loss.stride(0), n_cols, 1e-10,
BLOCK_SIZE=16384, log_target=True, reduction=1, num_warps=16)
torch.cuda.synchronize()
# torch.AcceleratorError: CUDA error: an illegal memory access was encountered
x = torch.ones((1, n_cols), device="cuda", dtype=torch.bfloat16)
g = torch.full((1,), 2.0, device="cuda", dtype=torch.bfloat16)
element_mul_kernel[(1,)](x, x.stride(0), g, n_cols, BLOCK_SIZE=32768, num_warps=32)
torch.cuda.synchronize()
# torch.AcceleratorError: CUDA error: an illegal memory access was encountered
With n_cols = 2**31 - BLOCK_SIZE (the last i + BLOCK_SIZE is 2**31 - BLOCK_SIZE, still an int32) both are fine. So are n_cols = 2**31 and 2**31 + 1: there Triton passes n_cols as an i64, and the compiled loop is an scf.for ... : i64 in the TTIR. Each case ran in its own process:
| kernel | n_cols = 2**31 - BLOCK_SIZE |
n_cols = 2**31 - 1 |
n_cols = 2**31, 2**31 + 1 |
2**31 - 1, int64 counter |
|---|---|---|---|---|
_kldiv_kernel_forward (BLOCK_SIZE=16384) |
✅ loss 0.0 | ❌ illegal memory access | ✅ loss 0.0 | ✅ loss 0.0 |
element_mul_kernel (BLOCK_SIZE=32768) |
✅ every element doubled | ❌ illegal memory access | ✅ every element doubled | ✅ every element doubled |
The fix is to count in int64:
- for i in range(0, n_cols, BLOCK_SIZE):
+ for i in range(0, n_cols.to(tl.int64), BLOCK_SIZE):
and likewise for the other loops above (tl.range too). In _kldiv_kernel_backward the loop also reassigns offsets, so its value before the loop needs the same type (offsets = tl.arange(0, BLOCK_SIZE).to(tl.int64)), or Triton rejects the loop-carried type change. The full change for all the kernels listed above is attached as a patch.
Versions
torch 2.9.1+cu128, triton 3.5.1, CUDA 12.8, NVIDIA GeForce RTX 4060 Ti (8 GB), driver 595.91.07.
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 656
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 27
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from linkedin/Liger-Kernel
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
linkedin/Liger-Kernel#1658 ·
Maintainers usually reply within 3 days
-
LigerPolyNorm defaults to in_place=True (docstring says False) and overwrites shared gradientsPossibly taken @nazek52 claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
linkedin/Liger-Kernel#1647 ·
Maintainers usually reply within 3 days
-
[Bug] Experimental embedding kernels overflow int32 row offsets (illegal memory access past 2^31 output elements)Possibly taken @0xDexe claimed this 8 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
linkedin/Liger-Kernel#1570 · 3 comments ·
Maintainers usually reply within 3 days
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
linkedin/Liger-Kernel#1661 ·
Maintainers usually reply within 3 days
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
linkedin/Liger-Kernel#1660 ·
Maintainers usually reply within 3 days
All issues in linkedin/Liger-Kernel
Similar issues
-
author-page correction move-papers
Difficulty 1/5 Under an hour Newbie friendliness 78/100
acl-org/acl-anthology#10978 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
GoogleCloudPlatform/professional-services-data-validator#1899 ·
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
HKUDS/CLI-Anything#652 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Project-N-E-K-O/N.E.K.O#3813 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Dao-AILab/flash-attention#3191 ·
Maintainers usually reply within 4 days