gpu: keep the accumulator in registers when two matmuls in a loop write the same tile

Open
#3,399 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
22/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
rust
Domain
compilers

Research direction

Start from the pass added in #3381 that promotes a tensor-core accumulator to registers across a loop, and find where it decides a tile is single-matmul only. The reproducer is the loop in the issue body: two matmuls writing the same tile offset, with a second sa/sb buffer feeding the second one. Done means that shape also keeps acc[w] in registers, verified against the existing pass tests and this two-buffer loop.

Written by the indexing model from the issue text.

Description

The pass that keeps a tensor-core accumulator in registers through a loop (#3381) takes a tile that one matmul reads and writes. A loop with two matmuls into the same tile keeps the accumulator in shared memory:

for kp in 0..k / 64 {
  // copies of two K tiles into sa[0], sb[0] and sa[1], sb[1]
  for w in 0..4 across subgroups {
    acc[w] = acc[w] + (sa[0][w] as f32) @ (sb[0] as f32);
  }
  for w in 0..4 across subgroups {
    acc[w] = acc[w] + (sa[1][w] as f32) @ (sb[1] as f32);
  }
}

This is the shape of a kernel with two tile buffers, where the copy of the second tile runs while the subgroups use the first. All the matmuls into the tile at the same offset should share one set of registers.

Dominant language
Rust
Stars
35
Forks
5
Avg merge
12h 26m
Merged PRs (30d)
799

Getting set up

We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from vx-lang/Vx

All issues in vx-lang/Vx

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.