[Qwen-Image 2.1 / Z-Image] Complex-valued RoPE hard-fails on backends without complex64 kernels (Ascend NPU: aclnnIndex EZ1001)

未关闭
#15,338 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
52/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python, pytorch

调研方向

Read transformer_qwenimage21.py, transformer_z_image.py, and the existing ROPE_PER_DEVICE and real-valued rotary path in transformer_qwenimage.py. Check the behavior described for Ascend NPU, CoreML export, and torch.compile, then trace how device-specific frequency tables are selected. Done means the three pipeline families use the requested real-valued path where needed and Qwen-Image 1.0 registers NPU support; no test file is named in the issue.

由索引模型根据 Issue 内容生成。

描述

Describe the bug

Two pipeline families in the Qwen-Image lineage still build their RoPE table with torch.polar and then index/multiply it as complex64 on the device, with no real-valued fallback:

  • transformer_qwenimage21.py — QwenImage21Rope.forward hardcodes use_real=False
  • transformer_z_image.py — torch.polar + view_as_complex

transformer_qwenimage.py (Qwen-Image 1.0) already has ROPE_PER_DEVICE from the AWS Neuron work in #13718, but only "neuron" is registered, so other accelerators keep taking the complex path.

On backends without complex64 kernels this is a hard failure rather than a slowdown, and it affects three targets, not one:

  1. Ascend NPU (torch_npu) — the op-plugin has no complex64 index/mul. QwenImage21Rope.forward raises aclnnIndex EZ1001 as soon as the complex frequency table is indexed on device. Qwen-Image 2.1 and Z-Image currently cannot run on Ascend NPUs without an out-of-tree monkey-patch.
  2. CoreML export — torch.polar has no MIL equivalent (apple/coremltools#2258).
  3. torch.compile — inductor cannot codegen complex ops (#12668), which matches this same code path.

CUDA eager is unaffected, which is presumably why this has stayed low-pressure upstream.

I understand #12668 already tracks the complex-op problem from the torch.compile angle; opening this as a separate issue because the backend failure mode and the fix scope are different (see "Describe the solution you'd like"). Happy to close this if you would rather consolidate into #12668.

Reproduction

The failure is device-specific, but the code path is easy to isolate. On an Ascend NPU with a from-source diffusers checkout:

import torch
from diffusers.models.transformers.transformer_qwenimage21 import QwenImage21Rope

# Any small instantiation of the rope module is enough; the failure happens at the
# first on-device indexing of the complex frequency table.
rope = QwenImage21Rope(...).to("npu")          # or .to(torch.device("npu"))
rope(x, freqs_cis)                              # -> aclnnIndex EZ1001

A backend-agnostic sanity check for the same assumption (runs anywhere, no model weights needed):

import torch
freqs = torch.polar(torch.ones(8, 4), torch.rand(8, 4)).to("npu")   # complex64 on device
freqs[torch.arange(8).to("npu")]                                    # index -> unsupported on NPU

Describe the solution you'd like

Extend the existing #13718 pattern instead of adding a new one:

  • port ROPE_PER_DEVICE + the real-valued apply_rotary_emb_qwen_neuron path into transformer_qwenimage21.py and transformer_z_image.py;
  • keep the frequency table and its indexing on CPU, crossing the device boundary as reals (_get_device_freqs already does this for Neuron);
  • register "npu" in ROPE_PER_DEVICE for Qwen-Image 1.0.

Context

I hit this while bringing up image models on a single Ascend 910B2; Qwen-Image 2.1 was the only pipeline in the set that could not run at all without patching.

Disclosure, and a note on scope: I opened PR #14905 for exactly this in September and it was auto-closed because it did not link an issue — that is on me, not on the change. Before re-opening anything I would like an explicit ack here, as the AI-assisted contribution section asks. I am also happy to split the work into one PR per pipeline instead of one size/L PR covering both, if that is easier to review.

Logs

RuntimeError: The inner error is reported as above. The process exits for this inner error. (Inner error: EZ1001: ... aclnnIndex ...)
  File "diffusers/models/transformers/transformer_qwenimage21.py", line ..., in forward
    ...

(The exact op-level message from aclnnIndex is the one that names the unsupported complex64 dtype.)

System Info

- diffusers: main (from-source checkout at the time of report; PR #14905 was opened 2026-09-29)
- platform: Linux aarch64
- Python: 3.12.13
- torch: 2.10.0+cpu
- torch_npu: 2.10.0.post4
- CANN: 9.1.0
- accelerator: 1x Ascend 910B2 (HBM 64 GB)

Who can help?

Anyone with familiarity with the Qwen-Image RoPE implementations, or with Apple/Neuron backends where the same real-valued path already exists.

主要语言
Python
星标
34.9k
派生
7.5k
平均合并
3 天 16 小时
30 天内合并 PR
65

环境准备

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

huggingface/diffusers 的其他 Issue

查看 huggingface/diffusers 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。