[Bug] Experimental embedding kernels overflow int32 row offsets (illegal memory access past 2^31 output elements)

Open Beginner friendly
#1,570 3 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

@0xDexe is already working on this.

Since Nov 10, 2026.

  • #1615 by @0xDexe — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python

Research direction

Read src/liger_kernel/ops/experimental/embedding.py, focusing on the row offset calculations in embedding_forward_kernel and embedding_backward_kernel. Cast offsets_m to int64 before multiplying by embedding_dim at both sites, then run or adapt the reproducer in the issue to verify outputs match the reference beyond the 2^31 element boundary.

Written by the indexing model from the issue text.

Description

🐛 Describe the bug

embedding_forward_kernel and embedding_backward_kernel in src/liger_kernel/ops/experimental/embedding.py compute the row offset as offsets_m[:, None] * embedding_dim in int32. offsets_m is tl.program_id(0) * BLOCK_SIZE_M + tl.arange(0, BLOCK_SIZE_M) and embedding_dim is a tl.constexpr, so the product stays int32 and wraps once n_indices * embedding_dim reaches 2^31.

  • forward, embedding.py:36: output_offsets = offsets_m[:, None] * embedding_dim + offsets_n[None, :]. The store goes before output.
  • backward, embedding.py:62: grad_output_ptr + offsets_m[:, None] * embedding_dim + offsets_n[None, :]. The load reads before grad_output.

The gather from the table (indices[:, None] * embedding_dim) is fine, since indices is int64.

For embedding_dim=512, row 2^22 starts at 2^22 * 512 = 2^31, which wraps to -2^31, so its stores land 4 GiB before output. With embedding_dim=4096 the limit is already reached at 524,288 tokens (e.g. batch 64 × seq 8192), a 4 GiB bf16 output. Nothing in LigerEmbeddingFunction checks for it. If the memory below output happens to be mapped, the forward kernel can overwrite it silently instead of crashing.

Reproduce

The kernels launched the way LigerEmbeddingFunction launches them (blocks of 128 × 128 for embedding_dim=512):

import torch, triton
from liger_kernel.ops.experimental.embedding import embedding_forward_kernel, embedding_backward_kernel

n, D, V = 2**22 + 16, 512, 16
grid = (triton.cdiv(n, 128), triton.cdiv(D, 128))
indices = torch.arange(n, device="cuda") % V

# forward
embeddings = torch.randn(V, D, device="cuda", dtype=torch.bfloat16)
output = torch.zeros(n, D, device="cuda", dtype=torch.bfloat16)
embedding_forward_kernel[grid](embeddings, indices, output, n, embedding_dim=D, BLOCK_SIZE_M=128, BLOCK_SIZE_N=128)
torch.cuda.synchronize()
# torch.AcceleratorError: CUDA error: an illegal memory access was encountered

# backward (separate process)
grad_output = torch.ones(n, D, device="cuda", dtype=torch.bfloat16)
grad_weight = torch.zeros(V, D, device="cuda", dtype=torch.float32)
embedding_backward_kernel[grid](grad_output, grad_weight, indices, n, embedding_dim=D, BLOCK_SIZE_M=128, BLOCK_SIZE_N=128)
torch.cuda.synchronize()
# torch.AcceleratorError: CUDA error: an illegal memory access was encountered

With n = 2**22 (largest offset 2^31 - 1) both are fine. Results, comparing the forward against embeddings[indices] and the backward against the index counts (fp32 grad_weight, so the sums are exact):

kernel n = 2**22, main n = 2**22 + 16, main n = 2**22 + 16, int64 cast
forward ✅ 0 rows differ ❌ illegal memory access ✅ 0 rows differ
backward ✅ 0 entries differ ❌ illegal memory access ✅ 0 entries differ

The fix is casting offsets_m to int64 before the multiply, at both sites:

-    output_offsets = offsets_m[:, None] * embedding_dim + offsets_n[None, :]
+    output_offsets = offsets_m[:, None].to(tl.int64) * embedding_dim + offsets_n[None, :]
-        grad_output_ptr + offsets_m[:, None] * embedding_dim + offsets_n[None, :],
+        grad_output_ptr + offsets_m[:, None].to(tl.int64) * embedding_dim + offsets_n[None, :],
Versions

torch 2.9.1+cu128, triton 3.5.1, CUDA 12.8, NVIDIA GeForce RTX 4060 Ti (8 GB), driver 595.91.07. Liger-Kernel main @ 781a8ae.

Dominant language
Python
Stars
6.7k
Forks
656
Avg merge
4d 12h
Merged PRs (30d)
27

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from linkedin/Liger-Kernel

All issues in linkedin/Liger-Kernel

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.