Feature: run the minimax_music3 DiT and depth decoder in bf16 under precision=bf16 (DiT 16x to 19x faster per call)

オープン
  • 何か問題がありますか？
#2,053 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
42/100
issue の種類
機能追加
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python, pytorch
領域
machine-learning

調査の方向性

Start with the minimax_music3 family implementation and its precision handling, then trace how the DiT, depth decoder, and language-model KV cache are built and used. Reproduce the reported measurements on the specified model and GPUs, and check existing end-to-end quality thresholds before changing precision or attention lengths. Done means bf16 precision speeds up the proposed components while meeting measured quality thresholds, and decode attention processes only filled cache rows.

索引モデルが issue の本文から書いたものです。

説明

enhancement

Request type: Improvement to existing behavior

Problem and use case

families/minimax_music3 requires precision=bf16, but only the language model is built in bf16. The DiT, the RVQ depth decoder, the condition encoder and the vocoder are built in float32 with TF32 off, so their GEMMs run on CUDA cores. For a 20 s song (the minimax-music3-l0 prompt, native runtime, 500 frames) the engine time is dominated by these float32 parts:

GPU Engine time per song Language model Depth decoder DiT Vocoder
B200 49.4 s 13.44 s 12.19 s 23.57 s 0.23 s
B300 43.3 s 12.34 s 12.22 s 18.56 s 0.21 s

The same graphs built with bf16 blocks (a test-harness build, not offered by the family) are much faster per call:

Engine GPU Default, float32 (ms) Same graph, bf16 (ms) Speedup
DiT call B200 / B300 98.36 / 77.25 5.28 / 4.67 18.6x / 16.5x
Depth decoder call B200 / B300 1.628 / 1.641 0.405 / 0.410 4.0x / 4.0x

A second, separate cost: every language-model decode step attends over the whole static 9216-row KV cache. In a B200 kernel profile of the bf16 step (11.0 ms), the decode attention kernel takes 5.4 ms (36 calls over 9216 rows), the cache concatenation 0.46 ms and the GEMMs about 4.3 ms, so the step is bound by attention over unused cache rows, not by weight reads.

Proposed outcome

  1. Under precision=bf16, build the DiT blocks and the depth decoder in bf16 (keeping float32 only where measurement shows it is needed, for example the timestep embedding, stem convolutions or norms), with E2E thresholds set from measured bf16-vs-reference numbers. Optionally the same for the condition encoder and vocoder if quality allows.
  2. Limit the language-model decode attention to the filled cache length instead of all 9216 rows.

Model and target details

MiniMaxAI/MiniMax-Music3 @ fbdf52fbaaca799592917417eb05f1899f1255ec (the minimax-music3-l0 manifest); TensorRT 11.4; B200 / B300. The DiT has 36 blocks (hidden 2048, 60 calls per 8 s window); the depth decoder has 4 layers (hidden 4096, 15 calls per frame, 7515 calls per 20 s song).

Alternatives considered

FP8 for these parts was measured (not merged): it gets most of its gain over the default from running the DiT blocks in bf16 (FP8 DiT call 4.25 ms against 5.28 ms for the bf16 graph, 1.24x), so a plain bf16 build captures most of the benefit with no quantization.

Additional context

Found during an FP8 study of this family (not merged).

主要言語
Python
スター
288
フォーク
73
平均マージ
1日 3時間
マージ済み PR（30日）
273

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/TensorRT-Model-Connect のほかの issue

NVIDIA/TensorRT-Model-Connect の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。