Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

fix: investigate Gemma 4 12B long-context MTP parity on Metal

Cerrado
  • ¿Algo no está bien?
#1,986 3 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Error
Claridad
Necesita aclaración
Estado de actividad
Activo
Stack tecnológico
rust
Área
machine-learning, performance

Línea de trabajo

Start by reproducing the 12B pair with --draft-kind mtp, --draft-block-size 4, --parallel 1, and the 1,056-token buffered diagnostic probe from #1983. Compare rotating-ring ordering, mask/dispatch geometry, and prefix state against classic M=1 decode, then measure teacher-forced, decided-position, and free-generation disagreements. Done means verified causes are corrected, the 12B startup guard is extended with measured cost, and 31B, Qwen MTP, performance, and untested backends are rechecked.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

area:inference platform:macos priority:high status:done type:bug

Problem / Background

While investigating #1983 on M5 Max (128 GiB, Metal, test-fast), an experimental long-context startup probe for models/gemma-4-12b-it-4bit with models/gemma-4-12b-it-assistant-4bit found a block-versus-chain mismatch after a 1,056-token prefill with speculative rotating-cache buffering. At block width 4, position 0 differs in 275,832/524,288 logit bytes; failure localization first reports layer 0 sliding-attention output. Disabling qmv_wide does not resolve it. This diagnostic result does not establish free-running error severity or its first introducing commit.

The new long buffered probe in #1983 is limited to its measured 31B geometry; 12B retains its existing short startup checks. Passing those short checks must not be interpreted as long-context parity. The 31B correction must not be copied to 12B solely because both have 512-wide global heads.

Proposed Solution

Reproduce with the 12B pair above, --draft-kind mtp --draft-block-size 4 --parallel 1, no inexact override, and a diagnostic probe that adds the 1,056-token buffered draw used during #1983. Compare physical rotating-ring ordering, mask/dispatch geometry, and prefix state against classic M=1 decode; quantify the mismatch before selecting a correction. Keep the default startup policy change separate until evidence supports it.

Acceptance Criteria

  • Reproduce on the current baseline and record provenance, actual buffered cache layout, and first divergent sub-operation.
  • Report teacher-forced top-1 and decided-position disagreements at realistic context and actual verify widths, plus three free-generation comparisons.
  • Correct verified causes and extend the 12B startup guard with measured startup cost; retain fail-closed behavior without an inexact override.
  • Recheck 31B and Qwen MTP and measure classic/MTP performance; document untested backends.
Lenguaje dominante
Rust
Estrellas
482
Forks
70
Merge medio
7 h 32 min
PR fusionados (30 d)
345

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de lablup/mlxcel

Todos los issues de lablup/mlxcel

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.