Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[QNN] 16a8w: gathered tensor read wrong by a consumer that only feeds a graph output

Abierto
  • ¿Algo no está bien?
#24,125 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python, pytorch
Área
backend, machine-learning

Línea de trabajo

Start with the provided reproducer and run rope_gather.pte with the three raw inputs, comparing output 2 with expected_out2.npy; the report says the QDQ host output is accurate while the deployed QNN output is wrong. Trace how gathered tensors reach their last consumer when that consumer feeds only a graph output, and compare with output 0, which also feeds a bmm. Done when the graph output matches the expected result and the unaffected cases remain correct.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

🐛 Describe the bug

In a 16a8w HTP graph, a tensor gathered in-graph (table[pos]) is read wrong by its last consumer when that consumer's result only feeds a graph output. Below, cos/sin are gathered once and used by two RoPEs: output 0 (which also feeds a bmm) is correct, output 2 is not (rel err 0.77 vs 0.0017 for the QDQ module on host). Position p gets row p % 40 for most rows.

Not affected: the same graph in fp16, passing cos/sin as inputs instead of gathering them, or a single RoPE. We hit it in a TTS model where the last layer's keys are a method output; returning one more tensor computed from them works around it.

import numpy as np
import torch
from executorch.backends.qualcomm.export_utils import make_quantizer
from executorch.backends.qualcomm.quantizer.quantizer import QuantDtype
from executorch.backends.qualcomm.serialization.qc_schema import QcomChipset
from executorch.backends.qualcomm.utils.utils import (
    generate_htp_compiler_spec, generate_qnn_executorch_compiler_spec, to_edge_transform_and_lower_to_qnn)
from torchao.quantization.pt2e.quantize_pt2e import convert_pt2e, prepare_pt2e

H, N, D = 6, 320, 128


def rope(x, cos, sin):
    return x * cos + torch.cat([-x[..., D // 2:], x[..., :D // 2]], -1) * sin


class M(torch.nn.Module):
    def __init__(self):
        super().__init__()
        ang = torch.arange(2048).float()[:, None] / 1e6 ** (torch.arange(0, D, 2).float() / D)
        self.register_buffer("cos_t", torch.cat([ang.cos(), ang.cos()], -1))
        self.register_buffer("sin_t", torch.cat([ang.sin(), ang.sin()], -1))

    def forward(self, x0, x1, pos):
        cos, sin = self.cos_t[pos.long()], self.sin_t[pos.long()]
        k0 = rope(x0, cos, sin)
        # k0 also feeds a bmm; k1 feeds only the graph output and comes out wrong
        return k0, torch.bmm(k0, x0.transpose(1, 2)), rope(x1, cos, sin)


torch.manual_seed(0)
sample = lambda: (torch.randn(H, N, D) * 3, torch.randn(H, N, D) * 3, torch.arange(N, dtype=torch.int32))  # noqa: E731
m, ex = M().eval(), sample()
q = prepare_pt2e(torch.export.export(m, ex).module(), make_quantizer(quant_dtype=QuantDtype.use_16a8w, soc_model="SM8850"))
for _ in range(8):
    q(*sample())
q = convert_pt2e(q)
specs = generate_qnn_executorch_compiler_spec(soc_model=QcomChipset.SM8850,
                                              backend_options=generate_htp_compiler_spec(use_fp16=False))
with open("rope_gather.pte", "wb") as f:
    f.write(to_edge_transform_and_lower_to_qnn(q, ex, specs).to_executorch().buffer)
for i, t in enumerate(ex):
    t.numpy().tofile(f"input_{i}.raw")
np.save("expected_out2.npy", m(*ex)[2].numpy())
print("QDQ rel err of output 2:", float((q(*ex)[2] - m(*ex)[2]).norm() / m(*ex)[2].norm()))

Run rope_gather.pte with the three inputs and compare output 2 with expected_out2.npy.

Versions

ExecuTorch 1.4.1 (our fork of v1.4.1, not yet checked on main), QAIRT 2.47.0.260601, SM8850 (Galaxy S26 Ultra, HTP v81).

Lenguaje dominante
Python
Estrellas
5.1k
Forks
1.2k
Merge medio
1 d 16 h
PR fusionados (30 d)
621

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de pytorch/executorch

Todos los issues de pytorch/executorch

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.