Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

test_backend_cross_device: MoeRouterTopK flakes intermittently on gfx1200

Abierto
  • ¿Algo no está bien?
#3,121 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
cpp
Área
testing-qa

Línea de trabajo

Empieza con tests/vt/test_backend_cross_device.cpp:2304, la prueba MoeRouterTopK, y compara ejecuciones aisladas con ejecuciones completas repetidas del binario test_backend_cross_device en gfx1200. Investiga las discrepancias intermitentes de NMSE y de índices de expertos, incluido si el estado del allocator o de la queue relaciona este caso con los otros ROCm flakes reportados. Se considerará terminado cuando se identifique la causa y se documente una solución o una conclusión práctica.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Row: BACKEND-ROCM

tests/vt/test_backend_cross_device.cpp:2304, TEST_CASE("MoeRouterTopK matches the CPU oracle (f32 and bf16 logits)"), failed once during work on a fresh implementer's row (row/KERNEL-QUANT-CIQ-GEMM-ROCM-IQUANT-w1, unrelated to MoE routing), on isravale (RX 9060 XT, gfx1200, ROCm 7.2.3):

tests/vt/test_backend_cross_device.cpp:2351: ERROR: CHECK( Nmse(ref_w, dw.Download()) <= kNmseTol ) is NOT correct!
  values: CHECK( 0.214825 <= 0.0005 )

plus mismatched expert indices in the same case.

Reproduction rate is low and I want to record it honestly rather than overstate it. The implementer who found it saw 1 failure in 12 full-test_backend_cross_device-binary runs with their (unrelated) change applied, and 1 in 12 with that change stashed at the pristine base commit c6e1092dffc5f3484d9c8d24ad3b4e3f7b7531cc — ruling their own diff out as the cause. I (operator) then ran the isolated case 10/10 clean and the full binary 6/6 clean, so 0 reproductions in 16 of my own runs. Net: 2 failures observed in 28 total runs across two sessions (~7%), never in isolation from either session — it may require the full binary's accumulated device/allocator state to trigger, which is why isolated single-case runs may under-sample it.

Confirmed unrelated to any in-flight PR: the test case and vt::MoeRouterTopK's kernel source are both absent from row/KERNEL-QUANT-CIQ-GEMM-ROCM-IQUANT-w1's diff, and the test case is confirmed present, unmodified, at that branch's pre-reconciliation head 22800690b (predates all work on that row).

Distinct from #3119 (causal conv1d, a different test case, much higher ~37% isolated reproduction rate) and from #1954 (MoeSiluMul). Given this project's position that no device-side ASan exists for RDNA/gfx12, and that this session alone has found two independent classes of adjacent-but-different intermittent ROCm numerical flakes, this may be worth checking whether all three share a root cause (a common allocator/queue-state issue) rather than being investigated as three unrelated kernels — but that is a hypothesis, not a finding, and needs someone to actually look.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Lenguaje dominante
C++
Estrellas
423
Forks
53
Merge medio
1 d 7 h
PR fusionados (30 d)
382

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de mudler/vllm.cpp

Todos los issues de mudler/vllm.cpp

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.