nvidia: SROA before the optimization pipeline lowers registers in many kernels, but adds spills to one
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- compilers, rust
- Domain
- compilers, performance
Research direction
Start by reading the linked issue #3588 and tracing the linked-IR callback in deviceImageOf, where the early SROA pass was applied before the -O3 pipeline. Compare pass ordering and inlining for the listed kernels, especially mla_heads_at; the ptxas sweep table is at /mnt/scratch/glm-vx-claude-d134c2f5/for-vx94/3588/ptxas-6a1d-vs-3588.md on the dev box. Done means identifying what early SROA changes and validating a narrower pass with GPU timings.
Written by the indexing model from the issue text.
Description
An early version of #3588 ran SROA on each NVIDIA kernel before the -O3 pipeline (in the linked-IR callback of deviceImageOf). #3588 no longer does that, because it raised the spills of one kernel. But a ptxas sweep over glm-vx's 96 kernels (ptxas 13.3, sm_90) showed that it also lowered the register count of many others. Unrolling does not explain these changes: they went away when the early SROA went away.
Fewer registers:
| Kernel | Registers |
|---|---|
| fp8_moe_cvt_down | 123 -> 107 |
| fp8_matvec4 | 48 -> 40 |
| bf16_matvec4 | 40 -> 32 |
| fp8_matvec_fma | 63 -> 56 |
| fp8_matvec_cvt_tiles3_ref | 101 -> 96 |
| topk_sort, topk_sort_at | 26 -> 22 |
| fp8_matvec_cvt_tiles3 | 92 -> 88 |
| about 8 more | 1 or 2 fewer |
Worse:
| Kernel | Change |
|---|---|
| mla_heads_at | spills 16/16 B -> 52/72 B, stack 16 -> 48 B, 32 registers |
| fp8_moe_cvt_down_ref | 113 -> 123 registers |
| softmax_rows, fp8_moe_fma_gate_up | 2 more registers |
| about 4 more | 1 more register |
The question: what does the early SROA change that -O3 does not get back? A likely place is the order of SROA and inlining of the device helper functions. If the cause is clear, a narrower step may keep the gains without the mla_heads_at spills. Fewer registers is not the same as faster, so a change here needs GPU timings.
glm-vx's full table: /mnt/scratch/glm-vx-claude-d134c2f5/for-vx94/3588/ptxas-6a1d-vs-3588.md on the dev box.
- Dominant language
- Rust
- Stars
- 35
- Forks
- 5
- Avg merge
- 12h 26m
- Merged PRs (30d)
- 799
Getting set up
We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from vx-lang/Vx
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
bug testing
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
`pub struct` gives "Expected 'fn' after 'pub'": the error should say that a struct needs no `pub`Open
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
bug devex
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
longbridge/gpui-kit#4031 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
neuralmagic/crucible#465 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
compiler P3
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
HerringtonDarkholme/hd-lang#2162 ·
Maintainers usually reply within 1 day
-
embedded SQL: pin correlated-guard behaviour when a closed variable is reassigned inside a branchOpenpriority: low
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
jonathanong/no-mistakes#3627 ·
Maintainers usually reply within 1 day