[Benchmark] Arc Pro B70 / Qwen3.8-Flash-Next UD-Q4_K_XL / 262K: MTP vs DFlash2 (and why Auto never picked DFlash2 on SYCL)

Chiusa Adatta ai principianti
  • Qualcosa non va?
#5,138 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 4 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
62/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
docker, python, shell
Ambito
backend, machine-learning

Direzione di ricerca

Inizia da serve/server.py, dove engine_args() passa engine=exe a drafting.apply(), e guarda server.engine_binary(), che risolve già il launcher SYCL nel suo binario. Passa il binario risolto a drafting.engine_form() e verifica che la riga di avvio non segnali più DFlash2 come non disponibile su Intel. Il secondo bug, la risoluzione del percorso /work in drafting.mtp_ready() e data_dir_of(), è separato e dovrebbe essere una modifica a sé stante.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary
  • On 0.2.10, Auto did not pick DFlash2 on this Arc Pro B70. The startup line was Drafting: MTP (DFlash2 isn't available in this mode).
    • Bug 1: the drafter check reads sycl/serve/strata-sycl.sh instead of the engine binary. I think this affects every SYCL install.
    • Bug 2: /work paths are checked outside the engine container. This one applies to my setup, where the server itself runs in Docker.
  • With both fixed, DFlash2 runs. On short prompts it decodes about as fast as MTP, or slightly faster: about 41 tok/s. It accepts more of its drafts: 83-85% on short answers, against 60-73% for MTP.
  • At 140K-235K context the two drafters are equal, within noise: 28.7-30.5 tok/s against 29.0-29.8 tok/s.
  • Prompt reading is the same with either drafter, about 1,300 tok/s.
Setup
GPU Intel Arc Pro B70 32 GB, xe driver, PCIe host to device 23-26 GB/s (engine probe)
Host Ryzen 9 7950X3D, 93 GB RAM, Debian 13, kernel 7.1.8
Runtime oneAPI 2026.1, compute-runtime 26.35.39758.10-0, level-zero 1.32.0, AOT bmg-g31
Strata v0.2.10 (ffa80d73), engine 0.1.56
Model Unsloth UD-Q4_K_XL, context 262144, --kv fp16 --kv-resident 32768 --prefill 8192 --stream-experts --vram-reserve-mib 1536, vision off, one request at a time
Expert cache MTP: 7,791 slots (22.7 GiB). DFlash2: 7,866 slots (22.9 GiB). The rest of the experts (about 51 GiB) sit in the pinned host mirror.
MTP --mtp data/mtp/rt --spec 4 --spec-min-p 0.5
DFlash2 Q4_0 pack from strata drafting fetch, Auto flags: --draft dflash2 --draft-fallback none --draft-conf 0.6, --spec 4 from my user_args

Requests were greedy (temperature 0), streamed through the OpenAI endpoint. Decode speed is (completion tokens - 1) / (last token - first token). Acceptance figures come from the engine's drafts accepted N of M lines.

Long context: same script, same config, only the drafter changes

Each prompt is the same document set repeated to length, and the model writes 300 tokens.

Prompt tokens MTP decode MTP accepted DFlash2 decode DFlash2 accepted Prompt read (MTP / DFlash2)
140,765 29.0 tok/s 47% (129/276) 28.7 tok/s 52% (105/202) 516* / 1,312 tok/s
187,667 29.8 tok/s 51% (132/259) 30.5 tok/s 59% (112/191) 1,300 / 1,306 tok/s
234,565 29.2 tok/s 50% (138/274) 28.7 tok/s 50% (108/214) 1,267 / 1,276 tok/s

* The first request after engine start, with a cold page cache.

Short prompts

DFlash2, measured with the server otherwise idle:

DFlash2
Short answer (~175 tokens), 2 runs 41.1 / 40.8 tok/s, 83-85% accepted
Long answer (800 tokens), 2 runs 33.8 / 35.3 tok/s, 63-70% accepted
Prompt 5.6K / 18K / 28K 1,182 / 1,168 / 1,309 tok/s

For comparison, MTP on the same day, but not from the same script:

  • 131K-context config, similar short coding prompts: 31.1-40.6 tok/s, 60-74% accepted.
  • 262K config, about 100 real agent turns (OpenCode supervisor), from the engine log, grouped by prompt size and weighted by tokens:
Prompt size Turns MTP decode MTP accepted
2K-50K 40 34.3 tok/s (31.2-43.6) 63%
50K-100K 45 32.4 tok/s (27.1-37.2) 60%
100K-150K 13 33.3 tok/s (28.9-35.7) 62%

DFlash2 on the same agent session at about 115K context: 33.1 and 33.9 tok/s, 68-70% accepted. That is two turns only.

The short-prompt MTP figures are not a clean A/B with the DFlash2 run. Treat the short-prompt difference as "DFlash2 is not slower", not as a measured percentage.

Bug 1: the drafter check reads the launcher script, not the engine binary

In serve/server.py, engine_args() calls drafting.apply(..., engine=exe). On Intel, exe is sycl/serve/strata-sycl.sh.

  1. drafting.engine_form() searches the file it is given for --draft-fallback and finds nothing in the shell script.
  2. It therefore returns FORM_NONE, which apply() treats as blocked.
  3. The start then prints Drafting: MTP (DFlash2 isn't available in this mode), even though the binary supports DFlash2. engine_form("build-sycl-aot/strata") returns mtp-dir.

server.engine_binary(exe, cfg) already resolves the script to its binary for the effort_position check. Using it here fixes the problem:

-                              slots=parallel_n if isinstance(parallel_n, int) and not isinstance(parallel_n, bool) else 0, engine=exe)
+                              slots=parallel_n if isinstance(parallel_n, int) and not isinstance(parallel_n, bool) else 0,
+                              engine=engine_binary(exe, cfg) if exe else exe)

Since b1f4c929 (Auto picks DFlash2 on SYCL), I would expect every Intel start through strata-sycl.sh to fall back to MTP like this.

Bug 2: /work paths are checked outside the engine container (only when the server runs in Docker)

The engine container mounts STRATA_ROOT as /work, so the run config's --mtp, --pack and drafter paths are /work/.... In my setup the server also runs in a container, where STRATA_ROOT is mounted at its real path. As a result:

  • drafting.mtp_ready() found no MTP folder and printed Drafting: off (no drafter is downloaded yet), although the MTP layer was on disk.
  • drafting.data_dir_of() returned /work/data, so pack_ready() did not see the DFlash2 pack.

My workaround is to mount STRATA_ROOT at /work in the server container as well. This probably does not affect a server that runs directly on the host. It might be worth one sentence in INTEL_ARC.md for container setups.

A smaller point: strata drafting fetch needs numpy for the packing step. In a container without numpy it fails with The DFlash2 drafter could not be prepared on this computer, and the actual cause is hidden behind the plain message.

Raw data
Engine log lines (DFlash2, idle server) 
strata dflash2: 106299-token draft head (275.7 MiB), drafter 869 MiB of VRAM (Q4_0 weights)
strata serve: DFlash2 drafter on (/work/data/dflash2/qwen3.8-flash-next-dflash2.q4_0.pack, graphs captured in 15 ms); windows up to 6 tokens; no MTP layer is loaded
strata serve: prompt 84 tokens = 0 reused + 84 read in 719 ms (116.9 tok/s), 175 generated in 4268 ms (41.0 tok/s), drafts accepted 112 of 132, 1 checkpoints
strata serve: prompt 87 tokens = 0 reused + 87 read in 747 ms (116.4 tok/s), 178 generated in 4373 ms (40.7 tok/s), drafts accepted 113 of 136, 1 checkpoints
strata serve: prompt 92 tokens = 0 reused + 92 read in 728 ms (126.4 tok/s), 800 generated in 23699 ms (33.8 tok/s), drafts accepted 333 of 526, 1 checkpoints
strata serve: prompt 92 tokens = 0 reused + 92 read in 756 ms (121.7 tok/s), 800 generated in 22648 ms (35.3 tok/s), drafts accepted 369 of 579, 1 checkpoints
strata serve: prompt 5590 tokens = 0 reused + 5590 read in 4694 ms (1190.9 tok/s), 16 generated in 490 ms (32.7 tok/s), drafts accepted 6 of 12, 1 checkpoints
strata serve: prompt 18013 tokens = 0 reused + 18013 read in 15375 ms (1171.6 tok/s), 16 generated in 488 ms (32.8 tok/s), drafts accepted 7 of 11, 2 checkpoints
strata serve: prompt 28262 tokens = 0 reused + 28262 read in 21548 ms (1311.6 tok/s), 16 generated in 428 ms (37.3 tok/s), drafts accepted 9 of 13, 2 checkpoints
strata serve: prompt 140768 tokens = 0 reused + 140768 read in 107300 ms (1311.8 tok/s), 300 generated in 10467 ms (28.7 tok/s), drafts accepted 105 of 202, 6 checkpoints
strata serve: prompt 187666 tokens = 0 reused + 187666 read in 143700 ms (1305.8 tok/s), 300 generated in 9842 ms (30.5 tok/s), drafts accepted 112 of 191, 6 checkpoints
strata serve: prompt 234566 tokens = 0 reused + 234566 read in 183758 ms (1276.5 tok/s), 300 generated in 10451 ms (28.7 tok/s), drafts accepted 108 of 214, 6 checkpoints
Engine log lines (MTP, same 262K config) 
strata mtp: draft layer loaded, 866 MiB of VRAM (experts 675, dense 111), files read in 0.43 s (1831 MiB/s)
strata serve: prompt 140765 tokens = 0 reused + 140765 read in 272874 ms (515.9 tok/s), 300 generated in 10351 ms (29.0 tok/s), drafts accepted 129 of 276, 6 checkpoints
strata serve: prompt 187667 tokens = 0 reused + 187667 read in 144379 ms (1299.8 tok/s), 300 generated in 10075 ms (29.8 tok/s), drafts accepted 132 of 259, 6 checkpoints
strata serve: prompt 234565 tokens = 0 reused + 234565 read in 185073 ms (1267.4 tok/s), 300 generated in 10270 ms (29.2 tok/s), drafts accepted 138 of 274, 6 checkpoints
Lingua principale
C++
Stelle
11.6k
Fork
1k
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Niko1221/Strata

Tutte le issue di Niko1221/Strata

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.