GPU load balancing: improve accuracy of device loads for a mix of kernels; idle-time-based incremental balancing

Abierto
  • ¿Algo no está bien?
#3,976 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
22/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
cpp
Área
distributed-systems, hpc, performance

Línea de trabajo

This is a design follow-up to #3968 (GreedyRefineCentralGPULB and CUPTI). Start from hapiNormalizeCuptiLoads and how interval device time is split by sms_used. The payload asks for two per-object numbers (SM-seconds and serial kernel elapsed time), a group-time prediction, and idle-time-based incremental neighborhood moves. Done means those metrics and strategy exist and comments no longer call the load placement-invariant. No files or tests are named beyond that function.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Follow-up to #3968 (GreedyRefineCentralGPULB, CUPTI measurement). Design assumption: one device per process; device sharing across processes is left to chare migration.

1. The measured load does not travel with the object

hapiNormalizeCuptiLoads splits each interval of device time among the kernels running in it, weighted by sms_used (occupancy-model estimate from launch geometry). Summed over a device this equals its busy time, and the comment calls it "to first order invariant to placement". That holds only when the device is saturated. When it is not, the number is elapsed time under the current concurrency:

  • ten chares each running a 1-SM kernel concurrently for 10 ms are charged 1 ms each; moved alone to an idle device, one of them costs 10 ms;
  • an object measured alone on an idle device carries its full elapsed time to a saturated device, where it would overlap and cost a fraction.

So after migration an object meets a different kernel mix and its cost changes in a way the metric does not model. Undersaturated devices are latency-bound (elapsed time is the cost); saturated ones are throughput-bound (SM-seconds is). The balancer decides what to add to a device, which is the throughput question.

Suggestion: record two numbers per object, Σ dt·sms_used/total_sms (SM-seconds, intrinsic) and Σ dt over the object's own kernels (its serial chain, a latency floor), and predict a group's time as max(total SM-seconds of the group, longest single chain). Two doubles per object; the strategy's group load becomes a prediction rather than a sum of shares. Until then, the manual and the code comment should not call the metric placement-invariant.

2. Device idle time as spare capacity; incremental, neighbor-only steps

Kale: measure or estimate idle time on each device (no kernel resident) as the indicator of its capacity to absorb new work, and use it in more frequent, incremental balancing steps confined to a neighborhood (some definition of neighbors, e.g. a subset of the processes a process communicates with) rather than only at centralized steps. CUPTI already gives the per-device timeline, so idle fraction per device is available from the same sweep; a distributed or neighborhood strategy would move objects from saturated devices toward devices with idle time, with the SM-seconds figure above saying how much of that idle time an object would consume.

Lenguaje dominante
C++
Estrellas
241
Forks
62
Merge medio
1 d 1 h
PR fusionados (30 d)
121

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de charmplusplus/charm

Todos los issues de charmplusplus/charm

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.