Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Gluon all-gather reloads peer heap bases inside the tile loop

Open
  • Is something wrong?
#591 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

@vishalshetagar is already working on this.

Since Oct 18, 2026.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Active
Tech stack
hpc, python
Domain
distributed-systems, performance

Research direction

Start in iris/ccl/gluon/all_gather.py around lines 113-158, where the local heap base is already hoisted but peer bases are loaded per destination inside the tile loop. Hoist each destination's ptr_delta and the loop-invariant output_ptrs out of the loops, then check iris/ccl/gluon/all_to_all.py near line 130 for the same pattern. Done means correctness matches torch.distributed on 2, 4 and 8 ranks, and benchmark/ccl/bench_all_gather.py shows before and after numbers.

Written by the indexing model from the issue text.

Description

gluon good first issue help wanted iris python

Gluon all-gather reloads peer heap bases inside the tile loop

Follow-up to #585, where this was raised and deferred.

1. Hoist the peer heap bases

iris/ccl/gluon/all_gather.py:

113:  local_base = gl.load(ctx.heap_bases + iris_rank)      # hoisted, once
115:  for tile_id in range(pid, total_tiles, COMM_SMS):
147:      for rank_idx in range(world_size):
150:          output_ptrs = output_ptr + output_offsets      # loop-invariant
158:              target_base = gl.load(ctx.heap_bases + target_iris_rank)   # per dest, per tile

The comment at line 111 says the hoist "eliminates redundant gl.load(heap_bases)
calls in the inner store loop", but only the local base was hoisted. At 8 ranks
the loop still does 7 peer loads per tile. heap_bases, rank_start, rank_stride
and world_size are all tile-invariant, so each destination's ptr_delta can be
computed once before the tile loop — the same fix #585 applied to the TDM path in
d626261b. output_ptrs at line 150 is invariant across rank_idx too.

2. INLINE grouping, deferred from #585

From that review thread:

The grouping idea seems most useful for the register store path (gl.store),
where it bounds live values; I'd leave that to a later PR.

That is this file. These stores hold data live in registers across the whole
destination loop, and the docstring already warns the path is register-sensitive:
"Larger tiles cause register spilling and performance collapse."

Work

  • Hoist the per-destination ptr_delta out of the tile loop
  • Hoist output_ptrs out of the rank_idx loop
  • Confirm in the ISA whether range(world_size) at line 147 unrolls — it is
    range, not tl.static_range, and the grouping argument rests on it
  • Try INLINE grouping at 4 and 8; measure VGPR count and spill, not just time
  • Check iris/ccl/gluon/all_to_all.py:130 for the same pattern
  • Correctness vs torch.distributed on 2/4/8 ranks before timing
  • Before/after through benchmark/ccl/bench_all_gather.py

We expect a performance improvement here; it needs measuring.

Dominant language
Python
Stars
203
Forks
54
Avg merge
3d 23h
Merged PRs (30d)
23

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ROCm/iris

All issues in ROCm/iris

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.