Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

fix: investigate Gemma 4 12B long-context MTP parity on Metal

Chiusa
  • Qualcosa non va?
#1,986 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
rust
Ambito
machine-learning, performance

Direzione di ricerca

Start by reproducing the 12B pair with --draft-kind mtp, --draft-block-size 4, --parallel 1, and the 1,056-token buffered diagnostic probe from #1983. Compare rotating-ring ordering, mask/dispatch geometry, and prefix state against classic M=1 decode, then measure teacher-forced, decided-position, and free-generation disagreements. Done means verified causes are corrected, the 12B startup guard is extended with measured cost, and 31B, Qwen MTP, performance, and untested backends are rechecked.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area:inference platform:macos priority:high status:done type:bug

Problem / Background

While investigating #1983 on M5 Max (128 GiB, Metal, test-fast), an experimental long-context startup probe for models/gemma-4-12b-it-4bit with models/gemma-4-12b-it-assistant-4bit found a block-versus-chain mismatch after a 1,056-token prefill with speculative rotating-cache buffering. At block width 4, position 0 differs in 275,832/524,288 logit bytes; failure localization first reports layer 0 sliding-attention output. Disabling qmv_wide does not resolve it. This diagnostic result does not establish free-running error severity or its first introducing commit.

The new long buffered probe in #1983 is limited to its measured 31B geometry; 12B retains its existing short startup checks. Passing those short checks must not be interpreted as long-context parity. The 31B correction must not be copied to 12B solely because both have 512-wide global heads.

Proposed Solution

Reproduce with the 12B pair above, --draft-kind mtp --draft-block-size 4 --parallel 1, no inexact override, and a diagnostic probe that adds the 1,056-token buffered draw used during #1983. Compare physical rotating-ring ordering, mask/dispatch geometry, and prefix state against classic M=1 decode; quantify the mismatch before selecting a correction. Keep the default startup policy change separate until evidence supports it.

Acceptance Criteria

  • Reproduce on the current baseline and record provenance, actual buffered cache layout, and first divergent sub-operation.
  • Report teacher-forced top-1 and decided-position disagreements at realistic context and actual verify widths, plus three free-generation comparisons.
  • Correct verified causes and extend the 12B startup guard with measured startup cost; retain fail-closed behavior without an inexact override.
  • Recheck 31B and Qwen MTP and measure classic/MTP performance; document untested backends.
Lingua principale
Rust
Stelle
482
Fork
70
Merge medio
7h 2m
PR unite (30g)
367

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di lablup/mlxcel

Tutte le issue di lablup/mlxcel

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.