[MPSNDArrayMatrixMultiplication encodeToCommandBuffer:] destination ... is too large for kernel on M3 Max 64 GB

Aperta
#67 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
42/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
c, macos

Direzione di ricerca

Start by reproducing the failure with ./h3 using the two commands in the issue on an M3 Max with 64 GB, then trace the MPSNDArrayMatrixMultiplication attention path that materializes the score matrix. Investigate a chunked or tiled SDPA path, or an MPSGraph scaled-dot-product fallback for large sequences. Done means both reported 8-second cases avoid the kernel abort while the small README validation continues to work.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Reproduction
Any generation where the combined video+audio token sequence grows past ~26k. Two confirmed failures, both at denoise 0/20:
A. 1024×576, 8 s (192 frames):
./h3 -d ./MiniMax-H3 -p "..." --width 1024 --height 576 --seconds 8
--steps 20 --layers 45 --reuse 2 -o outputs/out.mp4

B. 896×512, 8 s (192 frames): same command with --width 896 --height 512.
Both abort with:
MPSNDArrayMultiaryKernel.mm:2644: failed assertion `[MPSNDArrayMatrixMultiplication encodeToCommandBuffer:] destination: [
<MPSTemporaryNDArray ...> Rank: 4, MPSDataTypeBFloat16, Device: Apple M3 Max

  • dimensionLengths = [ 33654, 33654, 56, 1, ... ] (case A)
  • dimensionLengths = [ 26358, 26358, 56, 1, ... ] (case B)
  • strideElements = [ 1, 33656, ... ] / [ 1, 26360, ... ]
    ] is too large for kernel. Encode failed.'
    zsh: abort

Analysis
The failing op is an explicitly materialized attention-score matrix: 56 heads × S×S BF16, i.e. 56 × S² × 2 bytes:

Canvas / duration | Sequence S | Score-matrix footprint
1024×576 × 8 s | 33,654 | ≈ 127 GB
896×512 × 8 s | 26,358 | ≈ 78 GB

where S ≈ (W/32)×(H/32)×⌈frames/4⌉ + audio tokens. Both footprints far exceed a 64 GB unified-memory machine. Development appears to happen on a 128 GB M5 Max, where the 78 GB case fits — which is presumably why this hasn't been caught. M3 Max also has no Metal 4/TensorOps attention path, so it falls into this MPS path even at shapes that work on M5.
Meanwhile small sequences work fine on the same machine: the README's 512×512 / 22-frame fox validation runs correctly.

Expected behavior
M3-class / ≤ 64 GB machines need a fallback that doesn't materialize the full S×S score matrix — e.g. a chunked/tiled SDPA (flash-attention style), or dispatching to an MPSGraph scaled-dot-product path when S exceeds a memory-derived threshold.

Suggested immediate mitigation for users
Keep 56 × S² × 2 ≤ ~22 GB of temp headroom (DiT at --layers 45 already resident at ~33 GB). On a 64 GB machine that means S ≲ 14,000, e.g. 864×480 × 5 s (S = 14,339) is the largest validated-geometry shape expected to work; longer videos must be produced as chained shorter segments via --ref-video.
Happy to run any diagnostics / patched builds on this machine.

Lingua principale
C
Stelle
3.1k
Fork
268
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di antirez/h3.c

Tutte le issue di antirez/h3.c

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.