Feature: run the minimax_music3 DiT and depth decoder in bf16 under precision=bf16 (DiT 16x to 19x faster per call)

Abierto
  • ¿Algo no está bien?
#2,053 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
42/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python, pytorch
Área
machine-learning

Línea de trabajo

Start with the minimax_music3 family implementation and its precision handling, then trace how the DiT, depth decoder, and language-model KV cache are built and used. Reproduce the reported measurements on the specified model and GPUs, and check existing end-to-end quality thresholds before changing precision or attention lengths. Done means bf16 precision speeds up the proposed components while meeting measured quality thresholds, and decode attention processes only filled cache rows.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Request type: Improvement to existing behavior

Problem and use case

families/minimax_music3 requires precision=bf16, but only the language model is built in bf16. The DiT, the RVQ depth decoder, the condition encoder and the vocoder are built in float32 with TF32 off, so their GEMMs run on CUDA cores. For a 20 s song (the minimax-music3-l0 prompt, native runtime, 500 frames) the engine time is dominated by these float32 parts:

GPU Engine time per song Language model Depth decoder DiT Vocoder
B200 49.4 s 13.44 s 12.19 s 23.57 s 0.23 s
B300 43.3 s 12.34 s 12.22 s 18.56 s 0.21 s

The same graphs built with bf16 blocks (a test-harness build, not offered by the family) are much faster per call:

Engine GPU Default, float32 (ms) Same graph, bf16 (ms) Speedup
DiT call B200 / B300 98.36 / 77.25 5.28 / 4.67 18.6x / 16.5x
Depth decoder call B200 / B300 1.628 / 1.641 0.405 / 0.410 4.0x / 4.0x

A second, separate cost: every language-model decode step attends over the whole static 9216-row KV cache. In a B200 kernel profile of the bf16 step (11.0 ms), the decode attention kernel takes 5.4 ms (36 calls over 9216 rows), the cache concatenation 0.46 ms and the GEMMs about 4.3 ms, so the step is bound by attention over unused cache rows, not by weight reads.

Proposed outcome

  1. Under precision=bf16, build the DiT blocks and the depth decoder in bf16 (keeping float32 only where measurement shows it is needed, for example the timestep embedding, stem convolutions or norms), with E2E thresholds set from measured bf16-vs-reference numbers. Optionally the same for the condition encoder and vocoder if quality allows.
  2. Limit the language-model decode attention to the filled cache length instead of all 9216 rows.

Model and target details

MiniMaxAI/MiniMax-Music3 @ fbdf52fbaaca799592917417eb05f1899f1255ec (the minimax-music3-l0 manifest); TensorRT 11.4; B200 / B300. The DiT has 36 blocks (hidden 2048, 60 calls per 8 s window); the depth decoder has 4 layers (hidden 4096, 15 calls per frame, 7515 calls per 20 s song).

Alternatives considered

FP8 for these parts was measured (not merged): it gets most of its gain over the default from running the DiT blocks in bf16 (FP8 DiT call 4.25 ms against 5.28 ms for the bf16 graph, 1.24x), so a plain bf16 build captures most of the benefit with no quantization.

Additional context

Found during an FP8 study of this family (not merged).

Lenguaje dominante
Python
Estrellas
288
Forks
73
Merge medio
1 d 3 h
PR fusionados (30 d)
273

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de NVIDIA/TensorRT-Model-Connect

Todos los issues de NVIDIA/TensorRT-Model-Connect

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.