nvidia: SROA before the optimization pipeline lowers registers in many kernels, but adds spills to one

Open
#3,589 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
compilers, rust

Research direction

Start by reading the linked issue #3588 and tracing the linked-IR callback in deviceImageOf, where the early SROA pass was applied before the -O3 pipeline. Compare pass ordering and inlining for the listed kernels, especially mla_heads_at; the ptxas sweep table is at /mnt/scratch/glm-vx-claude-d134c2f5/for-vx94/3588/ptxas-6a1d-vs-3588.md on the dev box. Done means identifying what early SROA changes and validating a narrower pass with GPU timings.

Written by the indexing model from the issue text.

Description

An early version of #3588 ran SROA on each NVIDIA kernel before the -O3 pipeline (in the linked-IR callback of deviceImageOf). #3588 no longer does that, because it raised the spills of one kernel. But a ptxas sweep over glm-vx's 96 kernels (ptxas 13.3, sm_90) showed that it also lowered the register count of many others. Unrolling does not explain these changes: they went away when the early SROA went away.

Fewer registers:

Kernel Registers
fp8_moe_cvt_down 123 -> 107
fp8_matvec4 48 -> 40
bf16_matvec4 40 -> 32
fp8_matvec_fma 63 -> 56
fp8_matvec_cvt_tiles3_ref 101 -> 96
topk_sort, topk_sort_at 26 -> 22
fp8_matvec_cvt_tiles3 92 -> 88
about 8 more 1 or 2 fewer

Worse:

Kernel Change
mla_heads_at spills 16/16 B -> 52/72 B, stack 16 -> 48 B, 32 registers
fp8_moe_cvt_down_ref 113 -> 123 registers
softmax_rows, fp8_moe_fma_gate_up 2 more registers
about 4 more 1 more register

The question: what does the early SROA change that -O3 does not get back? A likely place is the order of SROA and inlining of the device helper functions. If the cause is clear, a narrower step may keep the gains without the mla_heads_at spills. Fewer registers is not the same as faster, so a change here needs GPU timings.

glm-vx's full table: /mnt/scratch/glm-vx-claude-d134c2f5/for-vx94/3588/ptxas-6a1d-vs-3588.md on the dev box.

Dominant language
Rust
Stars
35
Forks
5
Avg merge
12h 26m
Merged PRs (30d)
799

Getting set up

We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from vx-lang/Vx

All issues in vx-lang/Vx

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.