Qwen-Image-2.1 2K输出,多参考图 <=5, 推理时OOM
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 48/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
Research direction
Start with pipelines/qwen_image_21.py at lines 106 and 336, then inspect models/qwen_image_21_dit.py at lines 189 and 534 where the attention mask is built. Compare the positive and negative CFG paths with the reported PyTorch flex_attention stack and CUDA allocations. Done means identifying why mask creation exhausts memory for 2K output with up to five reference images and documenting or validating a reproducible resolution.
Written by the indexing model from the issue text.
Description
(1)7B的Qwen-Image-2.1推理 180G显存的B200还显存不够,错误堆栈如下:
(2)9B的FLux2-klein-base-9B 推理相同的数据,显存都够用,没有仔细统计,但是一般不超过80G. 为什么Qwen-Image-2.1 OOM?
433 [rank0]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
434 [rank0]: return func(*args, **kwargs)
435 [rank0]: File "/xxx/diffsynth/pipelines/qwen_image_21.py", line 106, in __call__
436 [rank0]: noise_pred = self.cfg_guided_model_fn(
437 [rank0]: File "/xxx/diffsynth/diffusion/base_pipeline.py", line 343, in cfg_guided_model_fn
438 [rank0]: noise_pred_nega = model_fn(**inputs_nega, **inputs_shared, **inputs_others)
439 [rank0]: File "/xxx/diffsynth/pipelines/qwen_image_21.py", line 336, in model_fn_qwen_image_21
440 [rank0]: model_output = dit(
441 [rank0]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
442 [rank0]: return self._call_impl(*args, **kwargs)
443 [rank0]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
444 [rank0]: return forward_call(*args, **kwargs)
445 [rank0]: File "/xxx/diffsynth/models/qwen_image_21_dit.py", line 534, in forward
446 [rank0]: attention_mask = build_qwenimage21_block_causal_mask(
447 [rank0]: File "/xxx/diffsynth/models/qwen_image_21_dit.py", line 189, in build_qwenimage21_block_causal_mask
448 [rank0]: return create_block_mask(
449 [rank0]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/attention/flex_attention.py", line 897, in create_block_mask
450 [rank0]: partial_block_mask, full_block_mask = _convert_mask_to_block_mask(
451 [rank0]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/attention/flex_attention.py", line 697, in _convert_mask_to_block_mask
452 [rank0]: mask_block_sum = mask.sum(
453 [rank0]: torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.16 GiB. GPU 0 has a total capacity of 178.35 GiB of which 32.23 GiB is free. Including non-PyTorch memory, this
process has 146.10 GiB memory in use. Of the allocated memory 76.70 GiB is allocated by PyTorch, and 68.62 GiB is reserved by PyTorch but unallocated.
983 [rank2]: image = pipe(
984 [rank2]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
985 [rank2]: return func(*args, **kwargs)
986 [rank2]: File "/xxx/diffsynth/pipelines/qwen_image_21.py", line 106, in __call__
987 [rank2]: noise_pred = self.cfg_guided_model_fn(
988 [rank2]: File "/xxx/diffsynth/diffusion/base_pipeline.py", line 335, in cfg_guided_model_fn
989 [rank2]: noise_pred_posi = model_fn(**inputs_posi, **inputs_shared, **inputs_others)
990 [rank2]: File "/xxx/diffsynth/pipelines/qwen_image_21.py", line 336, in model_fn_qwen_image_21
991 [rank2]: model_output = dit(
992 [rank2]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
993 [rank2]: return self._call_impl(*args, **kwargs)
994 [rank2]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
995 [rank2]: return forward_call(*args, **kwargs)
996 [rank2]: File "/xxx/diffsynth/models/qwen_image_21_dit.py", line 534, in forward
997 [rank2]: attention_mask = build_qwenimage21_block_causal_mask(
998 [rank2]: File "/xxx/diffsynth/models/qwen_image_21_dit.py", line 189, in build_qwenimage21_block_causal_mask
999 [rank2]: return create_block_mask(
1000 [rank2]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/attention/flex_attention.py", line 897, in create_block_mask
1001 [rank2]: partial_block_mask, full_block_mask = _convert_mask_to_block_mask(
1002 [rank2]: File "/usr/local/miniconda3/envs/qwen21/lib/python3.10/site-packages/torch/nn/attention/flex_attention.py", line 697, in _convert_mask_to_block_mask
1003 [rank2]: mask_block_sum = mask.sum(
1004 [rank2]: torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 72.38 GiB. GPU 2 has a total capacity of 178.35 GiB of which 21.79 GiB is free. Including non-PyTorch memory,
this process has 156.54 GiB memory in use. Of the allocated memory 50.40 GiB is allocated by PyTorch, and 105.36 GiB is reserved by PyTorch but unallocated. If reserved but unallocated
memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/
cuda.html#environment-variables)
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 15m
- Merged PRs (30d)
- 46
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from modelscope/DiffSynth-Studio
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
modelscope/DiffSynth-Studio#1702 ·
-
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
modelscope/DiffSynth-Studio#1668 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
modelscope/DiffSynth-Studio#1499 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
modelscope/DiffSynth-Studio#1373 · 5 comments · 1 reaction ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
modelscope/DiffSynth-Studio#1704 ·
All issues in modelscope/DiffSynth-Studio
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100