#[Bug] Cross entropy, fused CE+TVD and GRPO's log-softmax return NaN when a row's first block of logits is all -inf
Maintainers usually reply within 3 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 55/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- machine-learning
Research direction
Read the online softmax update in src/liger_kernel/ops/cross_entropy.py, src/liger_kernel/ops/fused_ce_tvd.py, and src/liger_kernel/ops/grpo_loss.py, along with the fix pattern in issue #1525. Reproduce the leading all--inf block case for cross entropy, fused CE+TVD, and GRPO, including the sequence-level GRPO path mentioned in the issue. Done means these paths return finite results matching PyTorch, with finite gradients where applicable.
Written by the indexing model from the issue text.
Description
🐛 Describe the bug
liger_cross_entropy_kernel, _fused_ce_tvd_forward_kernel, _selective_log_softmax_kernel and _grpo_loss_fwd_kernel compute each row's log-sum-exp with an online softmax over blocks of logits, starting from max = -inf, sumexp = 0. If every logit of the row's first block is -inf, the running max is still -inf after it:
m_new = tl.maximum(m, block_max) # -inf
d = d * tl.exp2((m - m_new) * LOG2_E) + tl.sum(tl.exp2((X_block - m_new) * LOG2_E))
m = m_new
Both m - m_new and X_block - m_new are -inf - (-inf) = NaN, so the running sum becomes NaN and stays NaN through the remaining blocks. The row's log-sum-exp is NaN, and so is everything computed from it:
liger_cross_entropy_kernel: the loss and every entry of the gradient._fused_ce_tvd_forward_kernel: the CE, the TVD and the saved log-sum-exp. This happens for the student logits and, by the same code, for the teacher's (then the TVD is NaN)._selective_log_softmax_kernel(fused_selective_log_softmax) and_grpo_loss_fwd_kernel: the row's log-probability, and from it the GRPO loss.
The rows are valid: they have finite logits after the first block, and the label points at one of them. torch.nn.functional.cross_entropy and torch.softmax give finite results for them.
This needs a vocabulary larger than one block (32768 logits for cross entropy, 16384 for fused CE+TVD, 2048 for the GRPO kernels) whose first block is all masked, e.g. by logits.masked_fill(~allowed, float("-inf")) when only tokens past the first block's ids are allowed. LigerCrossEntropyLoss and LigerFusedLinearCrossEntropyLoss both launch the cross-entropy kernel.
It is the same pattern #1525 fixes in _softmax_multi_block_forward_kernel. On current main (781a8ae):
liger_cross_entropy_kernelL139, L183–L185_fused_ce_tvd_forward_kernelL96–L107_selective_log_softmax_kernelL58–L60_grpo_loss_fwd_kernelL158–L160
Reproduce
Cross entropy:
import torch
from liger_kernel.transformers import LigerCrossEntropyLoss
torch.manual_seed(0)
V = 50257
x = torch.randn(4, V, device="cuda")
x[:, :32768] = float("-inf") # the first block masked
y = torch.randint(32768, V, (4,), device="cuda") # labels among the unmasked logits
print(torch.nn.functional.cross_entropy(x, y, reduction="none"))
xg = x.clone().requires_grad_(True)
loss = LigerCrossEntropyLoss(reduction="none")(xg, y)
loss.sum().backward()
print(loss, xg.grad.isnan().sum().item())
| loss | NaN gradient entries | |
|---|---|---|
torch |
[10.2685, 11.4343, 10.1693, 9.7875] |
0 |
Liger main |
[nan, nan, nan, nan] |
201028 (all) |
| with the fix below | [10.2685, 11.4343, 10.1693, 9.7875] |
0 |
With the fix, the gradient matches torch's to 5e-10.
Fused CE+TVD:
import torch
from liger_kernel.ops.fused_ce_tvd import LigerFusedCETVDFunction
torch.manual_seed(0)
V = 50257
s = torch.randn(4, V, device="cuda")
t = torch.randn(4, V, device="cuda")
s[:, :16384] = float("-inf") # the student's first block masked
y = torch.randint(16384, V, (4,), device="cuda")
ce, tvd = LigerFusedCETVDFunction.apply(s, t, y, -100)
print(ce, tvd)
print(torch.nn.functional.cross_entropy(s, y, reduction="none"),
0.5 * (torch.softmax(s, -1) - torch.softmax(t, -1)).abs().sum(-1))
| CE | TVD | |
|---|---|---|
torch |
[11.0964, 9.6125, 12.2725, 12.3862] |
[0.6105, 0.6117, 0.6126, 0.6141] |
Liger main |
[nan, nan, nan, nan] |
[nan, nan, nan, nan] |
| with the fix below | [11.0964, 9.6125, 12.2725, 12.3862] |
[0.6105, 0.6117, 0.6126, 0.6141] |
With the teacher's first block masked instead, main gives a NaN TVD, and the fix matches torch to 1.2e-7. With no masked logits, both kernels agree with torch before and after the fix.
GRPO's selective log-softmax:
import torch
from liger_kernel.ops.grpo_loss import fused_selective_log_softmax
torch.manual_seed(0)
B, L, N = 2, 3, 4096
logits = torch.randn(B, L + 1, N, device="cuda")
logits[..., :2048] = float("-inf") # the first block (BLOCK_N = 2048) masked
input_ids = torch.randint(2048, N, (B, L), device="cuda")
print(fused_selective_log_softmax(logits, input_ids, temperature=0.9))
print(torch.log_softmax(logits[:, :-1] / 0.9, -1).gather(-1, input_ids[..., None])[..., 0])
| log-probabilities | |
|---|---|
torch |
[[-7.1026, -8.7077, -8.2264], [-9.7784, -6.3727, -11.8797]] |
Liger main |
[[nan, nan, nan], [nan, nan, nan]] |
| with the fix below | [[-7.1026, -8.7077, -8.2264], [-9.7784, -6.3727, -11.8797]] |
Suggested fix
Rescale against 0 while the running max is still -inf, as #1525 does for softmax. At that point the running sum is still 0, so the result is unchanged.
m_new = tl.maximum(m, block_max)
- d = d * tl.exp2((m - m_new) * LOG2_E) + tl.sum(tl.exp2((X_block - m_new) * LOG2_E))
+ # Avoid -inf - (-inf) = NaN while every logit so far is -inf.
+ ref_m = tl.where(m_new == float("-inf"), 0.0, m_new)
+ d = d * tl.exp2((m - ref_m) * LOG2_E) + tl.sum(tl.exp2((X_block - ref_m) * LOG2_E))
m = m_new
_fused_ce_tvd_forward_kernel takes the same change for student_sumexp and teacher_sumexp, and the two GRPO kernels for alpha and l_i. The full patch for the three files, against 781a8ae, is in Liger_online_softmax_masked_leading_block.patch.
Repository follow-up (2026-11-02): the tracked _grpo_loss_fwd_kernel_seq also uses this update.
The local fix.patch now covers that sequence-level path. Its original capture is
reproduced before generating the corrected PTX with the original binding; the verification records
are in the GRPO follow-up.
This follow-up adds no GPU measurements to the reproduction results above.
Reproduce
No response
Versions
RTX 4060 Ti (sm_89), torch 2.9.1+cu128, Triton 3.5.1, Liger main (781a8ae).
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 656
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 27
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from linkedin/Liger-Kernel
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
linkedin/Liger-Kernel#1658 ·
Maintainers usually reply within 3 days
-
LigerPolyNorm defaults to in_place=True (docstring says False) and overwrites shared gradientsPossibly taken @nazek52 claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
linkedin/Liger-Kernel#1647 ·
Maintainers usually reply within 3 days
-
[Bug] Experimental embedding kernels overflow int32 row offsets (illegal memory access past 2^31 output elements)Possibly taken @0xDexe claimed this 8 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
linkedin/Liger-Kernel#1570 · 3 comments ·
Maintainers usually reply within 3 days
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
linkedin/Liger-Kernel#1661 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
linkedin/Liger-Kernel#1659 ·
Maintainers usually reply within 3 days
All issues in linkedin/Liger-Kernel
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
GoogleCloudPlatform/professional-services-data-validator#1899 ·
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
mohanraj00/entelos#200 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
sweep
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
bketelsen/truenas-mcp#77 ·
Maintainers usually reply within 1 day
-
Console writes URL credentials to the local log, though AGENTS.md says secrets never reach diskOpen
Difficulty 1/5 Under an hour Newbie friendliness 86/100
sandbox-quantum/switch#2163 ·
Maintainers usually reply within 1 day