bench_mla_decode crashes with --model deepseek: randint from=1 >= to=1
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 65/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- python, pytorch
- Domain
- performance
Research direction
Run op_tests/op_benchmarks/triton/bench_mla_decode.py --model deepseek -o to reproduce the crash. Then inspect test_mla_decode_rope.py, especially input_helper and the S=0 cases generated by the benchmark configuration. Done means the deepseek benchmark completes without the invalid torch.randint range.
Written by the indexing model from the issue text.
Description
Description
bench_mla_decode.py crashes when run with --model deepseek due to test_mla_decode_rope.input_helper calling torch.randint(1, S+1, ...) where S=0.
Reproduction
python3 op_tests/op_benchmarks/triton/bench_mla_decode.py --model deepseek -o
Error
File ".../test_mla_decode_rope.py", line 38, in input_helper
seqlens = torch.randint(1, S + 1, (B,), dtype=torch.int32, device=device)
RuntimeError: random_ expects 'from' to be less than 'to', but got from=1 >= to=1
Root Cause
The model benchmark config generates S=0 for some batch size / sequence length combinations, and input_helper doesn't guard against S <= 0.
Environment
- GPU: AMD Instinct MI355X (gfx950)
- PyTorch: 2.8.0+rocm7.2
- AITER: commit a89006c
🤖 Generated with Claude Code
- Dominant language
- Python
- Stars
- 565
- Forks
- 585
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 371
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ROCm/aiter
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
[MLA][gfx942] Native qh64 fp8 persistent decode kernel GPU-faults at page_size=1 (block_size=1) Open
Difficulty 2/5 Half a day Newbie friendliness 72/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100