performance regression on Ring SP

Aperta
#266 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
55/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
python, pytorch
Ambito
distributed-systems, machine-learning, performance

Direzione di ricerca

Start with diffsynth_engine/layers/attention/backends/sdpa.py around lines 99–113 and trace how Ring SP selects the SDPA backend. Reproduce the comparison with the provided Qwen Image 2512, GPU, and kernel benchmarks, then verify that the affected Ring SP/CP paths no longer incur the slower kernel without changing other backends.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Description

In v1 branch, deploying DiffSynth engine with 2 GPUs without NVLink and inferencing model Qwen Image 2512, ring SP2 is much slower than Ulysses SP2.

However, the total communication volume is identical for both strategies when using 2 GPUs while ring CP can overlap same comunication time spend with computation. Which means CP would be faster theoretically.

Reason Explanations

In v1 branch, when use ring sp (or cp) with sdpa, the attention backend would be chose as _scaled_dot_product_efficient_attention and this kernel is far slower than _scaled_dot_product_flash_attention. Other backends would not be influenced.

https://github.com/modelscope/DiffSynth-Engine/blob/c23ece5030101974d1691538e4515d6a2f1e013a/diffsynth_engine/layers/attention/backends/sdpa.py#L99-L113

Detailed comparison

On 4 RTX Pro 5000 Blackwell, for one 1024x1024 picture with 5 steps, the benchmarks table can be concluded as below.

kernels Efficient/kernel Torch Flash/kernel FA4/kernel FA4 vs Flash Torch Flash/step FA4/step
cp2cfg 733.068 us 287.672 us 279.057 us -2.99% 276.576 ms 275.910 ms
cp4 197.949 us 78.009 us 75.263 us -3.52% 442.760 ms 425.265 ms

As we can see, the Torch Flash kernel is much faster than Efficient kernel.

Lingua principale
Python
Stelle
432
Fork
51
Merge medio
3g 5h
PR unite (30g)
1

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di modelscope/DiffSynth-Engine

Tutte le issue di modelscope/DiffSynth-Engine

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.