FastMetal-14B-QAD hangs before first denoise step on M4 Max (1.3B/5B fine; GPU command buffer never completes)

オープン
  • 何か問題がありますか？
#2,016 コメント 2 件 リアクション 0 件 担当者 1 名 GitHub で見る

メンテナーはふだん 2 日以内に返信

@Ishxn20 がすでに取り組んでいます。

2026年11月6日 から。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
48/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python
領域
machine-learning, performance

調査の方向性

Start with examples/inference/basic/mlx_wan_prompt_to_video.py and reproduce using the documented command with the 14B checkpoint; compare against the working 1.3B and 5B runs. Read the MLX issue #3797 mentioned in the report and investigate where the 14B run stalls after weights load. Done means identifying the cause or a reliable diagnostic and confirming the 14B run reaches its first denoise step.

索引モデルが issue の本文から書いたものです。

説明

installation performance platform scope: attention scope: docs scope: inference scope: kernel scope: model

Describe the bug

FastMetal-14B-QAD hangs indefinitely on Apple M4 Max (128 GB, macOS 27.0.1) when running the documented MLX text-to-video example. The process never reaches the first denoise step. Reproduced twice, including with OMP_NUM_THREADS=1 MKL_NUM_THREADS=1.

FastMetal-1.3B-QAD and FastMetal-5B-QAD work perfectly on the same machine, same script, same environment (1.3B: 3.87 GiB peak, matching the published baseline exactly; 5B 720p: 9.337 GiB peak / 122.1 s denoise, both matching the published numbers).

Command

.venv/bin/python examples/inference/basic/mlx_wan_prompt_to_video.py \
  --model-root /Users/yanyu/models/FastMetal-14B-QAD \
  --mlx-checkpoint /Users/yanyu/models/FastMetal-14B-QAD \
  --height 480 --width 832 --num-frames 81 \
  --output-path video14b.mp4 \
  --prompt "A fox runs through a misty pine forest, leaves kicking up behind it."

Last log line: Loading weights: 100%|██| 242/242 — then silence (no denoise step 1/3 ever).

Environment

  • MacBook Pro, Apple M4 Max (40-core GPU), 128 GB unified memory
  • macOS 27.0.1 (build 26A434)
  • Python 3.13, FastVideo installed via pip install -e '.[mlx]'
  • mlx 0.32.4 / mlx-lm 0.32.2 / mlx-metal 0.32.4
  • Model downloaded via hf download FastVideo/FastMetal-14B-QAD (39 GB on disk, all 8 safetensors open cleanly)

Evidence that the hang is in the 14B DiT load / first step (not text encoding)

  1. Text encoding completed on the first attempt: FastVideo wrote the prompt-embeds cache at ~/.cache/fastvideo/prompt_embeds/04cc776e*.npy (its .npy.json fingerprint points at the 14B text_encoder root) before the hang. Subsequent runs hit this cache instantly and still hang — so the hang is strictly after prompt encoding.
  2. sample <pid> stack (macOS): main thread parked in _PySemaphore_Wait; all libmlx ThreadPool workers idle in condition_variable::wait; one Metal CommandQueueDispatch thread is live inside IOGPUCommandQueueSubmitCommandBuffers — i.e. a GPU command buffer was submitted and never completes (kernel-level hang/livelock signature), while the CPU side is doing nothing.
  3. CPU time frozen: ps cpu_time stops advancing (~15 s total after 10+ min wall); process sits at 0.0–0.3% CPU.
  4. Partial residency: at hang, system-wide wired memory is only ~12.7 GiB (baseline ~7 GiB) while process RSS is ~14.8 GiB — the 14.4 GB DiT checkpoint is not fully resident on the GPU when it stalls.
  5. Not shader compilation: MTLCompilerService idle, Metal shader cache byte-identical over 5 minutes of waiting.

What we ruled out

  • Missing --mlx-checkpoint (the classic silent-online-fetch hang): both args passed on every attempt.
  • libomp/OpenMP deadlock: OMP_NUM_THREADS=1 changes nothing; 1.3B/5B run fine in the same env.
  • Network wait: zero TCP connections on the hung process.
  • Memory pressure: Metal budget 107.5 GiB; the published 14B peak is 21.68 GiB. System had >60 GiB free at hang.

Suspicion

Given mlx issue #3797 (NAX split-K GEMM NaN/livelock family on Apple GPUs for specific feed-forward shapes) already affected Qwen-Image-Flash at 4096-token sequences, the 14B Wan DiT may hit a related kernel path on M4 Max at this size. The 1.3B/5B models don't reach that regime.

Happy to run any diagnostics or test a fixed checkpoint/runtime build on this machine.

Full local dossier

We keep a Chinese dossier with raw sample output and a reproducible timeline at: yyc3-22:~/models/apple-silicon-stack/results/fastmetal-14b-hang-dossier.md.

主要言語
Python
スター
4.6k
フォーク
486
平均マージ
3日 15時間
マージ済み PR（30日）
74

環境構築

  • Dockerfile・Docker Compose ファイルなし
  • プルリクエストのテンプレートあり
  • コントリビューションガイドなし

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

hao-ai-lab/FastVideo のほかの issue

hao-ai-lab/FastVideo の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。