Qualcomm AI Engine Direct: a delegated mutable buffer starts from leftover memory

Abierto
  • ¿Algo no está bien?
#24,749 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
python
Área
embedded-iot, machine-learning

Línea de trabajo

Start by reading the QNN partitioner logic that decides whether mutable buffers are delegated, including the int64 special case, and trace how delegated buffers are initialized before their first run. Reproduce the counter example with the QNN SDK 2.51 x86 simulator and compare delegated results with CPU results. Done means delegated buffers begin with the registered value across dtypes, and the int64 special case can be removed.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

module: qnn partner: qualcomm

When the QNN partitioner delegates a mutable buffer (the default,
skip_mutable_buffer=False), the buffer lives in memory that the QNN delegate
owns. Nothing sets that memory before the first run, so the buffer starts from
whatever the memory held. The result is wrong with no error.

Example: a counter that should start at zero.

class Int32Counter(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.register_buffer("count", torch.zeros(2, dtype=torch.int32))

    def forward(self, x):
        self.count.add_(x.argmax(-1).to(torch.int32))
        return x * 2, self.count + 0

With x = [[0.1, 0.9, 0.2], [0.7, 0.1, 0.3]] the first run should return
count = [1, 0]. Lowered for HTP float16 and run on the QNN SDK 2.51 x86
simulator, it returns values like [-1486893823, 32716], different on every
run.

Int64 buffers already stay on CPU for this reason, and the same counter with
an int64 buffer comes out right. Buffers of other dtypes,
including float buffers such as KV caches, are still delegated.

Expected: a delegated buffer starts from the same value as a CPU one, so the
int64 special case in the partitioner can go.

cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @qti-horodnic @cbilgin @abhinaykukkadapu @psiddh

Lenguaje dominante
Python
Estrellas
5.2k
Forks
1.2k
Merge medio
1 d 23 h
PR fusionados (30 d)
696

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de pytorch/executorch

Todos los issues de pytorch/executorch

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.