SDMA is Triton-only, and no collective uses it

Abierto
  • ¿Algo no está bien?
#593 2 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@ariscript ya está trabajando en esto.

Desde el 16/10/2026.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
18/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
distributed-systems, hpc

Línea de trabajo

Start by comparing how copy_engine_ctx and use_copy_engine are threaded through put in iris/mem/triton/ops.py (around line 280) with the Gluon put in iris/mem/gluon/context.py, then read the all-gather kernels in iris/ccl/triton/ and iris/ccl/gluon/. The first step is the Gluon signature mirror. Done means the Gluon call accepts the same arguments and the SDMA all-gather matches torch.distributed on 2, 4 and 8 ranks. The design choices (a Config engine axis versus a variant name) need maintainer input before coding.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement gluon help wanted iris

SDMA is Triton-only, and no collective uses it

Checked on main @ 6462f028.

Where SDMA exists today

iris/mem/triton/ops.py exposes copy_engine_ctx / use_copy_engine on four
entry points:

op def at
put 280
atomic_add 511
atomic_cas 638
quiet 964

Where it does not

Gluon has no SDMA surface. iris/mem/gluon/context.py mirrors the Triton RMA
set — load, store, get, put, copy, and nine atomics — but none of them
take a copy-engine argument, and sdma appears nowhere under iris/mem/gluon/.

No collective uses SDMA on either backend. grep -r "sdma\|copy_engine" iris/ccl/ returns nothing. Every kernel under iris/ccl/triton/ and
iris/ccl/gluon/ moves bytes with CU loads and stores.

So SDMA is available to someone hand-writing a Triton kernel against the RMA
primitives, and to nobody else. ctx.ccl.* cannot reach it.

Note the shape this leaves us in: the Gluon backend does have TDM
(iris/ccl/gluon/all_gather_tdm.py, #585, gfx1250), so Gluon got the newer
data-movement engine but not MI300-series copy engines, and the collectives got
neither.

Scope

Two pieces, in order:

  1. Gluon copy-engine surface — add copy_engine_ctx / use_copy_engine to
    IrisDeviceCtx.put (and the atomics that have it on the Triton side), matching
    the Triton signatures so kernels can move between backends.
  2. An SDMA collective — one collective, one backend, as a proof point. All-gather
    is the natural first: the transfer is a contiguous per-destination copy, which
    is exactly the contiguous_copy=True path put already supports.

Work

  • Mirror copy_engine_ctx / use_copy_engine in iris/mem/gluon/context.py
  • Add a copy-engine all-gather variant behind all_gather_variant="sdma"
  • Decide whether Config grows an engine axis or SDMA stays a variant name
  • Correctness vs torch.distributed on 2/4/8 ranks before timing
  • Benchmark vs the CU-driven variants through benchmark/ccl/bench_all_gather.py,
    which already has a comm_sms axis — the interesting number is CU count at
    equal bandwidth, not bandwidth at equal CU count

Related

#568 (engines= / copy-engine API), #573 (triggered SDMA), #585 (TDM all-gather).

Lenguaje dominante
Python
Estrellas
203
Forks
56
Merge medio
3 d 7 h
PR fusionados (30 d)
31

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de ROCm/iris

Todos los issues de ROCm/iris

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.