GPU load balancing: improve accuracy of device loads for a mix of kernels; idle-time-based incremental balancing

Aperta
  • Qualcosa non va?
#3,976 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
22/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
distributed-systems, hpc, performance

Direzione di ricerca

This is a design follow-up to #3968 (GreedyRefineCentralGPULB and CUPTI). Start from hapiNormalizeCuptiLoads and how interval device time is split by sms_used. The payload asks for two per-object numbers (SM-seconds and serial kernel elapsed time), a group-time prediction, and idle-time-based incremental neighborhood moves. Done means those metrics and strategy exist and comments no longer call the load placement-invariant. No files or tests are named beyond that function.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Follow-up to #3968 (GreedyRefineCentralGPULB, CUPTI measurement). Design assumption: one device per process; device sharing across processes is left to chare migration.

1. The measured load does not travel with the object

hapiNormalizeCuptiLoads splits each interval of device time among the kernels running in it, weighted by sms_used (occupancy-model estimate from launch geometry). Summed over a device this equals its busy time, and the comment calls it "to first order invariant to placement". That holds only when the device is saturated. When it is not, the number is elapsed time under the current concurrency:

  • ten chares each running a 1-SM kernel concurrently for 10 ms are charged 1 ms each; moved alone to an idle device, one of them costs 10 ms;
  • an object measured alone on an idle device carries its full elapsed time to a saturated device, where it would overlap and cost a fraction.

So after migration an object meets a different kernel mix and its cost changes in a way the metric does not model. Undersaturated devices are latency-bound (elapsed time is the cost); saturated ones are throughput-bound (SM-seconds is). The balancer decides what to add to a device, which is the throughput question.

Suggestion: record two numbers per object, Σ dt·sms_used/total_sms (SM-seconds, intrinsic) and Σ dt over the object's own kernels (its serial chain, a latency floor), and predict a group's time as max(total SM-seconds of the group, longest single chain). Two doubles per object; the strategy's group load becomes a prediction rather than a sum of shares. Until then, the manual and the code comment should not call the metric placement-invariant.

2. Device idle time as spare capacity; incremental, neighbor-only steps

Kale: measure or estimate idle time on each device (no kernel resident) as the indicator of its capacity to absorb new work, and use it in more frequent, incremental balancing steps confined to a neighborhood (some definition of neighbors, e.g. a subset of the processes a process communicates with) rather than only at centralized steps. CUPTI already gives the per-device timeline, so idle fraction per device is available from the same sweep; a distributed or neighborhood strategy would move objects from saturated devices toward devices with idle time, with the SM-seconds figure above saying how much of that idle time an object would consume.

Lingua principale
C++
Stelle
241
Fork
62
Merge medio
1g 1h
PR unite (30g)
121

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di charmplusplus/charm

Tutte le issue di charmplusplus/charm

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.