Device-aware Auto() tuning: reduce default items_per_thread leaves ~1.8x on AMD

オープン
  • 何か問題がありますか？
#150 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
55/100
issue の種類
機能追加
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
julia
領域
backend, performance

調査の方向性

Start by reading the existing reduce_tuning(::Backend, ::Type) hooks and the oneAPI extension's predicate_tuning pattern; check the current reduce defaults and tests or benchmarks that exercise Auto() reductions. Add ROCBackend-specific defaults near items_per_thread = 16 and block_size = 512, keeping CUDA and iGPU defaults unchanged. Verify with the existing reduction tests and compare AMD bandwidth against the reported baseline.

索引モデルが issue の本文から書いたものです。

説明

Summary

The Auto() reduce path currently uses a fixed default of items_per_thread = 2 (with block_size = 256). That value is fine on CUDA, where reduce already runs at the memory-bandwidth wall, but it leaves a large amount of performance on the table on AMD, where the same kernel runs at roughly half the card's bandwidth.

Sweeping items_per_thread shows the knee is vendor-dependent: CUDA is already saturated at ipt = 2, while AMD does not reach its bandwidth wall until ipt is around 8 to 16. A single hardcoded global default cannot be right for both.

The fix is to make the Auto() defaults device-informed rather than global, through the existing per-backend tuning hooks (reduce_tuning(::Backend, ::Type) / sort_tuning(::Backend, ::Type), same pattern as the oneAPI extension's predicate_tuning). No new mechanism is needed, only per-backend values.

Measurements

items_per_thread sweep on a 64M-element reduce, block_size = 256, effective bandwidth in GB/s (higher is better). The card's achievable bandwidth on the AMD part is about 315 GB/s.

items_per_thread AMD RDNA4 (gfx1200) NVIDIA RTX 5080
1 120 356
2 (current default) 195 564
4 261 566
8 263 566
16 290 568

On CUDA the default is already at the knee (a bump buys about 1%). On AMD the default sits well below the knee, and raising ipt is worth roughly 1.5x.

Apples-to-apples against AK.reduce's own default Auto() config (same algorithm and finalize path, only the config changed), AMD RDNA4:

config 64M 16M
default (ipt=2, bs=256) 159 GB/s 148 GB/s
ipt=4, bs=512 264 GB/s -
ipt=8, bs=512 264 GB/s -
ipt=16, bs=512 292 GB/s 251 GB/s

So just raising the AMD reduce config to around ipt=16, bs=512 is about 1.8x for free, within the existing Auto() structure. The last ~8% to the 315 GB/s wall is multi-pass overhead, which a true single-pass reduction would close, but that is a separate change.

Proposal

  • Keep the CUDA defaults as they are (already at the bandwidth wall).
  • Raise the AMD reduce defaults toward ipt ~= 16, block_size = 512 via reduce_tuning(::ROCBackend, ::Type).
  • Leave iGPU backends (Metal, Intel) at low ipt; register pressure there likely wants the opposite direction, so they should be measured before changing, not bumped by analogy.

Environment

  • AMD: RDNA4 gfx1200, AMDGPU 2.8.0, KernelAbstractions 0.9.42, KernelInterface 0.3.0
  • NVIDIA: RTX 5080, CUDA.jl, Julia 1.12
  • Measured against AcceleratedKernels 0.5 (main). Small CUDA sizes (1M to 16M) are L2-cache-resident on the 5080 and were excluded; only 64M+ is DRAM-meaningful.
主要言語
Julia
スター
204
フォーク
20
平均マージ
2日 19時間
マージ済み PR（30日）
41

環境構築

このプロジェクトには開発コンテナ、Dockerfile、コントリビューションガイドがありません。まず README を読み、一般的な手順ははじめてのコントリビューションガイドを参照してください。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

JuliaGPU/AcceleratedKernels.jl のほかの issue

JuliaGPU/AcceleratedKernels.jl の issue をすべて見る

似ている issue

Julia の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。