Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[Feature]: Add rooted collectives broadcast, scatter, gather, reduce or pull all-to-all

Open
  • Is something wrong?
#561 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
backend, data-engineering, distributed-systems

Research direction

Review the unmerged branches listed, especially muhaawad/ccl for the most recent broadcast and reduce implementations. Examine the host API in ctx.ccl and device tile API in iris/mem/triton/context.py. Start by understanding the existing all_gather and all_reduce implementations to see the pattern. The work involves adding new collectives like broadcast, scatter, gather, reduce, and pull all-to-all, requiring integration with both Triton and Gluon backends.

Written by the indexing model from the issue text.

Description

benchmarks gluon iris
Suggestion Description

There has been past work on enabling these rooted collectives. There are two main motivations:

  1. We need them to establish an MI450 baseline across various communication kernel implementations, tune and optimize them.
  2. They can be adapted per use case, moving away from the RCCL dependency for comm/compute fusion in TokenSpeed.

Current state on main

Collective Host API (ctx.ccl.*) Device tile API (iris/mem/triton/context.py)
all_gather ✅ ✅
all_reduce ✅ ✅ atomic / spinlock / one-shot / ring / two-shot
reduce_scatter ✅ ✅
all_to_all ✅ push only ✅ push only
broadcast ❌ ❌
scatter ❌ ❌
gather ❌ ⚠️ single-rank tile pull only (gather(tile, src_view, source_rank))
reduce ❌ ❌
all_to_all (pull) ❌ ❌

Existing work (unmerged branches, no open PR)

Branch Last commit What it adds
muhaawad/broadcast-reduce Mar 28, 2026 broadcast, reduce, with tests and benchmarks
impl/collectives-unified May 2, 2026 broadcast, scatter, gather, reduce, send/recv, barrier, in both Triton and Gluon
muhaawad/new-collectives May 2, 2026 same set as above, Triton only
muhaawad/new-collectives-v2 May 6, 2026 same set, plus ring and two-phase broadcast, and reworked all-gather / all-reduce / all-to-all
muhaawad/ccl Jul 10, 2026 broadcast, reduce (large Triton reduce), barrier; the most recent of these

Related all-to-all work (all push-based)

  • #417: port all-to-all to Gluon, with a benchmark sweep
  • #492: MoEDispatcher for expert-parallel token dispatch/combine (variable all-to-all)
  • muhaawad/alltoall-pairwise-rebased: pairwise-exchange all-to-all and AllToAllv
  • muhaawad/a2a-tdm-example: standalone all-to-all example using the Gluon Tensor Data Mover
Operating System

No response

GPU

No response

ROCm Component

No response

Dominant language
Python
Stars
202
Forks
52
Avg merge
5d 2h
Merged PRs (30d)
12

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ROCm/iris

All issues in ROCm/iris

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.