Device-aware Auto() tuning: reduce default items_per_thread leaves ~1.8x on AMD

Abierto
  • ¿Algo no está bien?
#150 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
55/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
julia
Área
backend, performance

Línea de trabajo

Start by reading the existing reduce_tuning(::Backend, ::Type) hooks and the oneAPI extension's predicate_tuning pattern; check the current reduce defaults and tests or benchmarks that exercise Auto() reductions. Add ROCBackend-specific defaults near items_per_thread = 16 and block_size = 512, keeping CUDA and iGPU defaults unchanged. Verify with the existing reduction tests and compare AMD bandwidth against the reported baseline.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

The Auto() reduce path currently uses a fixed default of items_per_thread = 2 (with block_size = 256). That value is fine on CUDA, where reduce already runs at the memory-bandwidth wall, but it leaves a large amount of performance on the table on AMD, where the same kernel runs at roughly half the card's bandwidth.

Sweeping items_per_thread shows the knee is vendor-dependent: CUDA is already saturated at ipt = 2, while AMD does not reach its bandwidth wall until ipt is around 8 to 16. A single hardcoded global default cannot be right for both.

The fix is to make the Auto() defaults device-informed rather than global, through the existing per-backend tuning hooks (reduce_tuning(::Backend, ::Type) / sort_tuning(::Backend, ::Type), same pattern as the oneAPI extension's predicate_tuning). No new mechanism is needed, only per-backend values.

Measurements

items_per_thread sweep on a 64M-element reduce, block_size = 256, effective bandwidth in GB/s (higher is better). The card's achievable bandwidth on the AMD part is about 315 GB/s.

items_per_thread AMD RDNA4 (gfx1200) NVIDIA RTX 5080
1 120 356
2 (current default) 195 564
4 261 566
8 263 566
16 290 568

On CUDA the default is already at the knee (a bump buys about 1%). On AMD the default sits well below the knee, and raising ipt is worth roughly 1.5x.

Apples-to-apples against AK.reduce's own default Auto() config (same algorithm and finalize path, only the config changed), AMD RDNA4:

config 64M 16M
default (ipt=2, bs=256) 159 GB/s 148 GB/s
ipt=4, bs=512 264 GB/s -
ipt=8, bs=512 264 GB/s -
ipt=16, bs=512 292 GB/s 251 GB/s

So just raising the AMD reduce config to around ipt=16, bs=512 is about 1.8x for free, within the existing Auto() structure. The last ~8% to the 315 GB/s wall is multi-pass overhead, which a true single-pass reduction would close, but that is a separate change.

Proposal

  • Keep the CUDA defaults as they are (already at the bandwidth wall).
  • Raise the AMD reduce defaults toward ipt ~= 16, block_size = 512 via reduce_tuning(::ROCBackend, ::Type).
  • Leave iGPU backends (Metal, Intel) at low ipt; register pressure there likely wants the opposite direction, so they should be measured before changing, not bumped by analogy.

Environment

  • AMD: RDNA4 gfx1200, AMDGPU 2.8.0, KernelAbstractions 0.9.42, KernelInterface 0.3.0
  • NVIDIA: RTX 5080, CUDA.jl, Julia 1.12
  • Measured against AcceleratedKernels 0.5 (main). Small CUDA sizes (1M to 16M) are L2-cache-resident on the 5080 and were excluded; only 64M+ is DRAM-meaningful.
Lenguaje dominante
Julia
Estrellas
204
Forks
20
Merge medio
2 d 19 h
PR fusionados (30 d)
41

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de JuliaGPU/AcceleratedKernels.jl

Todos los issues de JuliaGPU/AcceleratedKernels.jl

Issues similares

Más issues de Julia

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.