[Benchmark] 2x Intel Arc Pro B70, Qwen3.8-Flash-Next IQ3_S / IQ3_XXS, Strata 0.2.10 (engine 0.1.56 SYCL): drafting window sweep, slots, and the setups that failed

The issue provides a benchmark report and links a ZIP containing the data; no source files or tests are named. Start by reading the report’s method, results, and limitations, then inspect the linked data to confirm the recorded measurements. The work is to add this community benchmark to the project’s benchmark information while preserving that it is not protocol-comparable with #4810; done when the results and caveats are available to readers.

strata-2x-b70-data.zip

Community benchmark on 2x Intel Arc Pro B70

Measured 2026-11-03 and 2026-11-06 on one Linux machine with two Arc Pro B70 cards. Tested: Qwen3.8-Flash-Next IQ3_S
and IQ3_XXS split across both cards at 128K context, with and without thinking, the MTP drafting window (--spec 2 to 5),
4 and 8 slots, Strata's own tuner, --pipeline-windows 0, and one card. Main limitation: this is not the standard
benchmark. I used my own 21-task coding and reasoning suite with short prompts (168 to 1,071 tokens), so the numbers
are not protocol-comparable with #4810. strata bench run was not run.

This follows #4810 (same cards and quant family, closed for the benchmarks page with an invitation to measure again).

Hardware and software

• GPU: 2x Intel Arc Pro B70 32 GB, xe driver, GuC 70.65.0. Both on PCIe Gen5 x8 (root ports at 32.0 GT/s x8, measured 2026-10-09, not re-read for these runs). Engine PCIe probe 27.9 / 27.6 GB/s host to device. No P2P between the cards (the kernel logs cannot be used for peer-to-peer DMA).
• CPU: Intel Core i7-12700K, no AVX-512 (engine: "the expert kernels run on AVX-2"). 11 pool workers.
• RAM: 64 GB. MemAvailable during two-card runs: 48.8 to 57.3 GiB.
• Storage of the model shards: Samsung 990 EVO Plus NVMe (2026-11-03); a second NVMe, WD_BLACK SN7100 4 TB, ext4 (2026-11-06). A rotating disk (WD40EFZZ) only in the two aborted passes listed under Failures.
• OS: Ubuntu 26.04.1, kernel 7.0.0-42-generic. compute-runtime 26.31.39395.14, Level Zero 1.32.0.
• Strata: release v0.2.10, Terminal installed with install.sh (checksums verified), engine 0.1.56 SYCL release package, build 06698c674673...-oneapi2026.1.1 (JIT from SPIR-V), installed with strata components install-file --backend sycl because update --check --backend intel refused the feed's sycl package ("is a sycl engine, this PC needs intel").
• Background: the machine's other GPU services were stopped for every run; the cards showed 75 / 42 MiB used after they stopped. Power limits and clocks not measured. Point readings during decode: card 0 about 120 W (2026-11-03); card 1 about 216 W and card 0 about 93 W (2026-11-06).

Model and configuration

• Model: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF, IQ3_S and IQ3_XXS, 2 shards each. Repository revision and file hashes: not measured.
• Expert profile: the one shipped with Terminal 0.1.56 (24,576 pairs). MTP draft layer: mtp/rt from an earlier Strata data dir (revision not recorded); the engine logs draft layer loaded, 951 MiB and gathers the draft head from the planar Q6_K output head.
• Vision off. Context 131,072 (32,768 and 40,960 for the slot tests). --kv int8, --stream-experts, --expert-cache auto, --prefill auto, --vram-reserve-mib 2048, --layer-split auto.
• Layer split auto chose K=25 for IQ3_S (card 0 layers 0-24, card 1 layers 25-47), K=26 for IQ3_XXS. 100% of the 24,576 experts resident in VRAM (12,800 + 11,776 slots), no host arena. --pipeline-windows 2 on by default.
• Env: STRATA_VERIFY_NO_HOST=1 STRATA_STAGE_TRIM=1 SYCL_CACHE_PERSISTENT=0 ONEAPI_DEVICE_SELECTOR=level_zero:*. On 2026-11-03 only, every run config also carried STRATA_PREFILL_CPU_SHARE=0.4 (I meant to drop it and did not; I have not checked whether the engine reads it on this path). 2026-11-06 runs do not set it.
• Calibration / tuner: --tune skip, except one arm with --tune quick. Experimental speed projection: not used.

strata launch --backend intel --model-path <shards> --data-dir <data> --gpu 0,1 --context 128k --parallel 1 \
  --drafting mtp --vision off --tune skip --access local --port 8099 --no-tui --yes --restart \
  --set STRATA_VERIFY_NO_HOST=1 --set STRATA_STAGE_TRIM=1 --set SYCL_CACHE_PERSISTENT=0

# engine start line it produced (paths shortened):
strata --serve --pack <pack> --native <shard1> --ple-gguf <shard2> --expert-profile expert-profile.bin \
  --expert-cache auto --prefill auto --spec 2 --spec-min-p 0.5 --mtp <data>/mtp/rt --max-context 131072 \
  --kv int8 --stream-experts --vram-reserve-mib 2048 --layer-split auto

Changed per arm (each read back from the engine's own start line): --spec 3|4|5, --lookup-chain 4,
--pipeline-windows 0, --parallel 4 (engine --batch 4), --parallel 8, --gpu 0 (one card).

Method

• Workload: 21 tasks (13 coding tasks and 8 reasoning tasks, each scored by automatic checks; unit tests for the code), 3 runs each with seeds 1000-1002, one request at a time over /v1/chat/completions, streamed. Prompts are private and not attached; task names, token counts and per-task scores are.
• Sampling: temperature 0.7, top_p 0.8. No thinking: max_tokens 4096, chat_template_kwargs.enable_thinking=false. Thinking: max_tokens 8192, reasoning_effort: low, reasoning_budget_tokens: 2048.
• Each engine start: one 1-token probe, then two 920-token warm-up prompts (not scored). The suite then starts warm. Model loading is not in any timing.
• Prompt reuse: the engine reports 0 reused tokens on every suite request; every prompt was read fresh.
• Speeds below are the engine's own strata serve: prompt ... tok/s, N generated ... tok/s lines, matched to each request by prompt and generated token counts. Decode counts reasoning tokens. Prompt medians use requests with 0 reused and at least 128 read tokens. TTFT is the client clock from dispatch to the first content or reasoning delta.
• With thinking, when the reasoning hits the 2,048-token budget the engine writes a second line that re-reads the reasoning as a prompt (about 2,071 tokens read at a median 1,050 tok/s) and then writes the answer. For those requests (about 34 of 63 per think arm) decode below is (g1 + g2) / (ms1 + ms2); the re-read is not counted as decode.
• Windows: 2026-11-03 19:43 to 22:27 CST (window 8); 2026-11-06 00:37 to 09:11 CST (windows 9 to 15). The spec sweep on 2026-11-06 is compared with the --spec 2 arms from the same morning.

Results

Speed, one stream (engine timings; median and range over the 63 requests of an arm)

Configuration	Date	Prompt tokens (fresh)	Reused	Generated (median, range)	Runs	Prompt tok/s	Decode tok/s	TTFT s	Draft accept
IQ3_S, no thinking, spec 2	11-03	168-1,043	0	449 (49-3,856)	3	596 (273-803)	77.4 (55.9-95.2)	0.89 (0.55-1.74)	87%
IQ3_S, thinking, spec 2	11-03	196-1,071	0	2,197 (944-6,799)	3	637 (336-783)	109.2 (85.7-128.5)	0.92 (0.56-1.47)	87%
IQ3_XXS, no thinking, spec 2	11-03	168-1,043	0	436 (31-4,096)	3	702 (318-967)	76.8 (61.1-91.3)	0.80 (0.54-1.13)	86%
IQ3_XXS, thinking, spec 2	11-03	196-1,071	0	2,173 (852-6,654)	3	729 (360-990)	104.2 (83.4-127.7)	0.82 (0.56-1.12)	86%
IQ3_S, no thinking, spec 2	11-06	168-1,043	0	494 (49-4,096)	3	618 (314-805)	78.7 (72.8-106.6)	0.86 (0.55-1.40)	83%
IQ3_S, no thinking, spec 2, --tune quick	11-06	168-1,043	0	494 (49-4,096)	3	611 (314-801)	78.7 (72.7-106.2)	0.86 (0.55-1.41)	83%
IQ3_S, no thinking, spec 2 + --lookup-chain 4	11-06	168-1,043	0	494 (49-4,096)	3	621 (321-806)	79.2 (71.9-115.3)	0.86 (0.55-1.39)	77%
IQ3_S, no thinking, spec 3	11-06	168-1,043	0	494 (49-4,096)	3	612 (316-806)	93.8 (81.6-111.0)	0.87 (0.55-1.43)	84%
IQ3_S, no thinking, spec 4	11-06	168-1,043	0	494 (49-4,096)	3	613 (313-792)	102.5 (85.5-117.3)	0.87 (0.55-1.43)	80%
IQ3_S, no thinking, spec 4, --pipeline-windows 0	11-06	168-1,043	0	517 (49-2,356)	1	622 (316-794)	102.4 (85.7-111.8)	0.87 (0.56-1.40)	81%
IQ3_S, thinking, spec 2	11-06	196-1,071	0	2,360 (944-4,867)	3	634 (338-794)	110.0 (87.1-128.7)	0.92 (0.56-1.46)	87%
IQ3_S, thinking, spec 3	11-06	196-1,071	0	2,360 (944-4,867)	3	631 (333-789)	115.6 (87.8-143.5)	0.94 (0.57-1.48)	81%
IQ3_S, thinking, spec 4	11-06	196-1,071	0	2,360 (944-5,032)	3	629 (330-811)	115.3 (86.8-149.1)	0.93 (0.56-1.48)	75%
IQ3_S, thinking, spec 5	11-06	196-1,071	0	2,360 (944-4,867)	3	631 (328-790)	109.0 (83.7-145.1)	0.92 (0.57-1.48)	72%
IQ3_XXS, no thinking, spec 2	11-06	168-1,043	0	436 (31-3,269)	3	704 (319-968)	76.6 (70.2-100.5)	0.81 (0.54-1.12)	82%
IQ3_XXS, thinking, spec 2	11-06	196-1,071	0	2,173 (852-8,192)	3	724 (355-986)	105.8 (82.3-127.9)	0.82 (0.56-1.13)	86%

What changed between rows, and nothing else did: the date rows differ in the shard drive and in
STRATA_PREFILL_CPU_SHARE=0.4 (11-03 only); the 11-06 rows differ only in the flags named in the row.

• Drafting window is the one setting that moved decode. Without thinking, --spec 4 gave 102.5 tok/s against 78.7 at the default --spec 2 (same morning, same engine build), and the arm took 8.50 s per task against 10.22. With thinking the gain is smaller (115.3 against 110.0) and --spec 5 is slower than 4. Draft acceptance falls as the window grows (thinking: 87% at spec 2, 72% at spec 5).
• The answers were identical across drafting settings: the same quality score to four decimals in every IQ3_S arm of a mode (table below), consistent with exact verification.
• --lookup-chain 4, --pipeline-windows 0 and Strata's tuner changed nothing measurable. The tuner (--tune quick) reported "Your current settings are already the fastest we found", base decode 77.9 tok/s, prompt 1,308 tok/s, long prompt 3,188 tok/s (its prompt lengths are not in its result file).
• These prompts are short. Prompt speed rises with length (Strata's tuner and #4810 both measured 1,270 to 3,190 tok/s on longer prompts), so the prompt column is not a 4K or 16K figure.
• Engine load (launch to ready): 42 s from the 990 EVO Plus; 28 to 31 s from the SN7100 (43 to 49 GB read), 13 to 16 s when the files were still in page cache. During the suites the shard drive read 1 to 4 MiB/s.

Slots (no thinking, IQ3_S, 21 requests sent as 4 parallel streams)

Configuration	Result	Per-stream decode (client)	Aggregate	Prompt tok/s (engine)
--parallel 4, 40K context (11-03)	21/21 answered, no stall	23.8 (19.7-24.9)	78.0 tok/s	608 (330-781)
--parallel 4, 40K context (11-06)	21/21 answered	24.4 (20.0-25.5)	79.6 tok/s	612 (332-784)
--parallel 4, 128K context	refused at load (below)
--parallel 8, 32K context	refused at load (below)

With --batch the engine logs each prompt with 1 generated, so per-request engine decode is not available; the
per-stream figure is the client's completion tokens / (wall - TTFT). Four slots served four streams at about the
single-stream rate in aggregate.

Correctness (my suite; pass = mean test score)

Configuration	Rows	Score [95% CI over tasks]	s per task	Capped	Empty	Errors
IQ3_S no thinking, spec 2 (11-03 / 11-06)	63 / 63	0.876 [0.80, 0.94] / 0.858 [0.78, 0.93]	10.65 / 10.21	0 / 1	0	0
IQ3_S no thinking, spec 3, spec 4, lookup-chain 4, tuner	63 each	0.858 (all four)	9.08 / 8.50 / 10.10 / 10.22	1 each	0	0
IQ3_S thinking, spec 2 (11-03 / 11-06)	63 / 63	0.961 [0.92, 0.99] / 0.959 [0.92, 0.99]	25.06 / 24.43	0	0	0
IQ3_S thinking, spec 3, 4, 5	63 each	0.959 (all three)	23.36 / 23.42 / 24.60	0	0	0
IQ3_XXS no thinking (11-03 / 11-06)	63 / 63	0.816 [0.71, 0.91] / 0.839 [0.74, 0.92]	10.55 / 9.95	1 / 0	0	0
IQ3_XXS thinking (11-03 / 11-06)	63 / 63	0.887 [0.79, 0.97] / 0.900 [0.83, 0.96]	26.22 / 25.81	0 / 1	0	0
IQ3_S 4 slots 40K (11-03 / 11-06)	21 / 21	0.821 [0.70, 0.93] both	27.0 / 26.5 per stream	0	0	0

Garbage check: no reply from any Strata arm on either date, including the failed runs below, contains a run of 8 or
more ! or the word NaN: 0 of 1,168 rows. These are short prompts only; I did not test the long-prompt case of
#5160 on two cards.

Same-night reference on the same machine (not Strata, for scale only): llama.cpp SYCL serving Qwen3.6-35B-A3B
UD-Q4_K_M on one card, thinking on with no budget and max_tokens 16000, scored 0.746 [0.59, 0.88] at 124 s per
task over 2 runs; 9 of 42 answers were empty because the reasoning reached the cap.

Failures, refusals and poor results

1. Shards on a rotating disk (11-03, two passes, aborted by hand): prompt 23 to 39 tok/s, decode 15 to 35 tok/s (engine lines: prompt median 31 to 32, decode median 18), engine load 570 to 756 s, cards at 23 to 44 W. Same flags from the NVMe: prompt about 600, decode about 78. The engine reads the PLE rows from shard 2 every token (--ple-io direct), so the shard drive matters. Both passes carried STRATA_PREFILL_CPU_SHARE=0.4. Rows kept in the data (13 requests).
2. --parallel 4 at 128K refused: REFUSED: 988 experts of the later layer-split stages are neither in VRAM nor mirrored (the host mirror covers the first stage only); with STRATA_VERIFY_NO_HOST the device plan cannot run them. One xe engine reset (card 1, copy engine) at the teardown of this refused start.
3. --parallel 8 at 32K refused: same message, 186 experts. Two xe engine resets (compute and copy engine on one card) at this refused load. The next engine start and its probe answered normally. I did not turn STRATA_VERIFY_NO_HOST off, and I did not try an explicit --layer-split N, which the refusal message suggests together with STRATA_STAGE_TRIM=1.
4. One B70 (--gpu 0, IQ3_S, spec 4, 128K): the engine loaded (12,000 slots, 13,776 experts in a 26.35 GiB pinned mirror, MemAvailable 28.2 GiB during the arm, PCIe probe 13.2 GB/s) but every request generated 1 to 3 tokens at 0.5 tok/s engine-side; 16 of 21 answers were empty, the rest were fragments such as </think> or <|im_start|>; score 0.004. No !!!!, no resets. 60 tiny calls (~400 tokens in, 11 out) took 5.37 s each (p50). One run only.
5. Tiny calls on two cards: 1.21 s p50 / 1.22 s p95 per call (60 calls, ~400 tokens in, max 11 out, spec 4), against 0.38 s for a 4B model on one card under llama.cpp on the same machine.

Not run

• strata bench run (standard protocol), BENCH.bat/bench.sh, tools/needle_bench.py.
• 4K, 16K, 32K and 128K prompts on the two-card setup.
• DFlash2 (#5138, #5286); STRATA_PREFILL_MMQ=0; STRATA_VERIFY_NO_HOST=0 on two cards; --batch-groups; explicit --layer-split N.
• compute-runtime 26.40 or later; Strata 0.2.11 / engine 0.1.57.
• Power, clock and temperature logging.

Related issues

• #5395 (MTP layer dropped since 0.2.10): on this host install of the packaged 0.2.10 Terminal (strata launch --backend intel --drafting mtp, data dir on the host), --mtp stayed in the engine arguments and the draft layer loaded in every session; acceptance 72 to 87%.
• #5160: comment with these runs and an older one-card NVMe result posted there.

Data

data/ (attached as a zip):

• arm-summary.csv - one line per configuration: quality, CI, s/task, engine and client decode, prompt, draft acceptance, caps, resets.
• requests-client-and-engine.csv - every request: task, run, prompt and generated tokens, finish, client wall and TTFT, score, and the matched engine timing line.
• engine-timing-lines.csv - every engine timing line of these sessions, including probes and warm-ups.
• engine-logs/ - the engine log of each session (paths replaced), including the two refused starts.
• engine-start-and-load-lines.txt - start line, layer split, cache and MTP lines per session.
• configs/ - run configs and launch choices as written by the Terminal (credentials removed), and the tuner's result.
• refusals-launch-output.txt, window-log-excerpts.txt (resets, load times, disk reads, scores per arm), tiny-calls.json.
• scripts/ - the arm wrappers (w8_arm.sh for 11-03, w9_arm.sh for 11-06) and the tiny-call script.
• 2026-10-11-one-b70-ssd-experts-0.1.42/ - the older one-card NVMe result referenced in #5160.

Home folders, user and host names and addresses are replaced with placeholders.

An AI helped write the comment. They were measured on B70 cards.