WanToDance: hardcoded `device='cuda'` in music encoder construction crashes model loading on Ascend NPU
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 75/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python, pytorch
- Domain
- ai-infra-agents
Research direction
The issue is in diffsynth/models/wan_video_dit.py line 462 and diffsynth/models/wantodance.py line 156. Remove the hardcoded device='cuda' argument from the WanToDanceMusicEncoderLayer constructor. Verify the fix by running a test that loads the model on a non-CUDA device, such as CPU or NPU, to ensure no assertion error occurs.
Written by the indexing model from the issue text.
Description
Summary
WanModel.__init__ builds the WanToDance music encoder with a hardcoded CUDA device (diffsynth/models/wan_video_dit.py:462):
self.music_encoder.append(
WanToDanceMusicEncoderLayer(
d_model=latent_dim,
nhead=nhead,
dim_feedforward=ff_size,
dropout=dropout,
activation=activation,
batch_first=True,
rotary=rotary,
device='cuda',
)
)
WanToDanceMusicEncoderLayer forwards device to nn.MultiheadAttention(..., device=device) (diffsynth/models/wantodance.py:156), so the attention parameters are allocated on CUDA at construction time.
Impact
On any build without CUDA (Ascend NPU, CPU-only, ...) the model cannot be constructed at all, before the surrounding pipeline ever gets a chance to move it to the target device:
AssertionError: Torch not compiled with CUDA enabled
This block is gated by wantodance_enable_global / wantodance_enable_dynamicfps / wantodance_enable_unimodel, i.e. loading a WanToDance model on Ascend fails outright. Every sibling layer in the same block (nn.Linear, nn.Sequential) leaves the device to the caller's .to(device), so the hardcoded argument is also inconsistent with the rest of the file — and with diffsynth.core.device.npu_compatible_device, which the project already ships for NPU support.
Reproduction
# Ascend 910B4, torch 2.15.0.dev + torch_npu 2.15.0.dev
from wantodance import WanToDanceMusicEncoderLayer
WanToDanceMusicEncoderLayer(d_model=256, nhead=4, dim_feedforward=1024,
dropout=0.1, batch_first=True, rotary=None,
device='cuda')
# AssertionError: Torch not compiled with CUDA enabled
Proposed fix
Drop the hardcoded device='cuda'; the module is moved to the real device by the pipeline afterwards, exactly like its sibling layers. Verified on real Ascend 910B4 hardware (construction + forward pass on NPU); I will open a PR referencing this issue.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 12h 53m
- Merged PRs (30d)
- 42
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from modelscope/DiffSynth-Studio
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
modelscope/DiffSynth-Studio#1702 ·
-
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
modelscope/DiffSynth-Studio#1668 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
modelscope/DiffSynth-Studio#1499 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
modelscope/DiffSynth-Studio#1373 · 5 comments · 1 reaction ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
modelscope/DiffSynth-Studio#1709 · 1 comment ·
All issues in modelscope/DiffSynth-Studio
Similar issues
-
area: harness bug status: needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Human-Agent-Society/reef#625 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 1/5 Under an hour Newbie friendliness 80/100
learningequality/kolibri#15351 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Name consistency Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
eellak/triplestore#65 · 1 comment ·