Qualcomm: bmm over a flattened 4D input returns ~1/W of the right value when followed by unsqueeze (HTP fp16)

未关闭
  • 哪里不对？
#24,502 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
48/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
python, pytorch
领域
embedded-iot, machine-learning

调研方向

Start with the issue's minimal model and run it on CPU and the SM8850 HTP using the shown ExecuTorch lowering utilities; compare outputs with and without the final unsqueeze and with mask supplied as a graph input. Done means the HTP output matches the CPU result for the reported bmm pattern, rather than producing values about 1/W as large.

由索引模型根据 Issue 内容生成。

描述

🐛 Describe the bug

On the HTP, bmm over a flattened 4D input returns wrong values when the same input also feeds a conv and the result is unsqueezed back to 4D. Every output element is ~1/W of the right value (W=24 here), as if only one spatial row were summed. Found in ZipDepth's GlobalContextBlock.

import torch, torch.nn.functional as F
from executorch.backends.qualcomm.serialization.qc_schema import QcomChipset
from executorch.backends.qualcomm.utils.utils import (
    generate_htp_compiler_spec, generate_qnn_executorch_compiler_spec, to_edge_transform_and_lower_to_qnn)

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = torch.nn.Conv2d(192, 1, 1)

    def forward(self, x):
        B, C, H, W = x.shape
        mask = F.softmax(self.conv(x).view(B, 1, H * W), -1)
        return torch.bmm(x.view(B, C, H * W), mask.transpose(1, 2)).unsqueeze(-1)

x = torch.rand(1, 192, 24, 24) * 0.5
cs = generate_qnn_executorch_compiler_spec(
    soc_model=QcomChipset.SM8850, backend_options=generate_htp_compiler_spec(use_fp16=True))
edge = to_edge_transform_and_lower_to_qnn(M().eval(), (x,), cs)

On an S26 Ultra (SM8850, QAIRT 2.47, fp16), main and 1.4.1: output sum 2.05 vs 48.04 on CPU, ratio ~0.043 for every element.

  • Without the unsqueeze, or with mask as a graph input: exact.
  • (x.view(B, C, H * W) * mask).sum(-1, keepdim=True).unsqueeze(-1): exact (our workaround).
Versions

ExecuTorch main (a489dcac3b) and 1.4.1, QAIRT 2.47.0.260601, torch 2.14.

主要语言
Python
星标
5.2k
派生
1.2k
平均合并
1 天 22 小时
30 天内合并 PR
675

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

pytorch/executorch 的其他 Issue

查看 pytorch/executorch 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。