[Docs] 0.2.16 migration: PF_FUSED=0 passages still say "keeps MMQ"; layout move breaks old-path run configs at restart

Aperta Adatta ai principianti
#6,251 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 5 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
82/100
Tipo di issue
Documentazione
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
documentation

Direzione di ricerca

Leggi docs/DETAILS.md, docs/RADEON_780M.md e docs/STRIX_HALO.md per aggiornare le indicazioni su STRATA_PF_FUSED=0, incluso il gate principale PF_FUSED. Consulta la sezione Updating delle note di rilascio 0.2.16 e TROUBLESHOOTING per indicazioni sui percorsi di clone spostati, come data/ e serve/. Il lavoro è completato quando la documentazione spiega il nuovo comportamento di fallback e dice agli utenti di aggiornare i vecchi percorsi.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

needs-format

Area: [Docs]
Desktop app, Terminal or source? Source — manual clone built with CUDA 13.4 (-DSTRATA_GEOMETRY=flash_next, arch 75;86;120), served under systemd.

What happened

Two related migration gaps when updating a manual-clone install from v0.2.15 (engine 0.1.61) to v0.2.16 (engine 0.1.62). Both were observed on the chassis below; both are documentation/guidance gaps rather than kernel bugs.

Finding 1 — STRATA_PF_FUSED=0 no longer "keeps MMQ", and the docs still say it does.

  • Problem: three doc passages describe the pre-0.2.16 semantics in present tense — docs/DETAILS.md, docs/RADEON_780M.md, and docs/STRIX_HALO.md all say "STRATA_PF_FUSED=0 keeps MMQ."
  • Expected (per those docs): =0 keeps the MMQ prompt path.
  • Actual: v0.2.16 has no MMQ. A user-set =0 selects the native int8 prompt fallback (start log: "every chunk runs 48 of 48 layers on the int8 kernels"), while unset gets the fused kernels via the build defaults. A config carried over from the MMQ era — where =0 was a meaningful "keep the old path" choice — silently lands on a different path after the update, measured 7–9% slower at 7k–20k-token prompts (table below). Related: STRATA_PF_FUSED_SM75=1 alone is inert — the master PF_FUSED gates it (the start log stays "int8 kernels" with the master at 0).

Finding 2 — the layout move invalidates old tree paths in run configs, at restart rather than at update.

  • Problem: data/expert-profile.bin moved to terminal/data/expert-profile.bin (and serve/server.py → terminal/serve/server.py).
  • Expected: updating a manual clone either keeps documented config paths working or says they must be re-pointed.
  • Actual: a config with "--expert-profile", "<clone>/data/expert-profile.bin" keeps serving until the next restart (the running engine never re-reads the path); the new engine then exits 1 with read_expert_profile: cannot open <clone>/data/expert-profile.bin and systemd loops. Nothing in the release notes' Updating section or TROUBLESHOOTING mentions re-pointing old paths.
Smallest steps or command that reproduce it

Finding 1:

  1. Build v0.2.16 (no-MMQ) and serve an IQ3_XXS Flash-Next pack with STRATA_PF_FUSED=0 in the server config's env.
  2. The start log reads "prompt kernels: every chunk runs 48 of 48 layers on the int8 kernels."
  3. Measure prefill at 7k–20k-token prompts; set STRATA_PF_FUSED=1 (+ STRATA_PF_FUSED_SM75=1 for the sm_75 stage) and restart: the log reads "chunks of 1024 tokens or more run 48 of 48 layers on the fused kernels", and prefill rises 7–9% at those lengths.

Finding 2:

  1. On v0.2.15, serve with "--expert-profile", "<clone>/data/expert-profile.bin" (the tracked location at that tag).
  2. Update the clone to v0.2.16 (the file is now at terminal/data/expert-profile.bin) and restart the service.
  3. The engine exits 1 with read_expert_profile: cannot open <clone>/data/expert-profile.bin; systemd restart-loops until the config path is fixed.
Reproduction status

Reproduced locally — both findings, on the chassis below. Finding 1 measured across four arms × three passes; finding 2 caused an actual restart loop during our update.

Strata version

Strata 0.2.16 · engine 0.1.62 (upstream tag v0.2.16 plus two local one-line carries, #6084 and #6107, neither of which touches the prompt kernels). Baseline comparisons: v0.2.15 · engine 0.1.61. Doc passages quoted from origin/main at 64524fb9f.

Graphics card, memory and system

RTX 2080 Ti 11 GB (sm_75) + RTX 5060 Ti 16 GB (sm_120) + RTX 3080 Ti 12 GB (sm_86), PCIe, no P2P · 188 GB RAM · Ubuntu 26.04.1 LTS, kernel 7.3.4 · CUDA 13.4.

Model, size and settings

Qwen3.8-Flash-Next UD-IQ3_XXS · --max-context 524288 · --layer-split 18,33 · --pipeline-windows 2 · --kv k8v4 · vision on · thinking-on serving (temp 0 for the measurements).

Logs (sanitized)

Finding 1, the two start-log states:

strata: prompt kernels: every chunk runs 48 of 48 layers on the int8 kernels.
strata: prompt kernels: chunks of 1024 tokens or more run 48 of 48 layers on the fused kernels; shorter chunks run 48 on the staged fused entry.

Finding 2, the restart failure (path generalized):

read_expert_profile: cannot open <clone>/data/expert-profile.bin

Engine exit 1, systemd restart loop; fixed by pointing the config at the new location.

Measurements (evidence for finding 1)

Prefill tok/s, medians of three passes per arm, randomized passages, ten-minute settles, identical prompts across arms:

prompt tokens int8 (PF_FUSED=0) fused (PF_FUSED=1 + PF_FUSED_SM75=1) v0.2.15 MMQ+SM75
~7k 1,042 1,123 1,140
~9.3k 975 1,071 1,069
~11.6k 1,210 1,323 1,316
~15.5k 1,569 1,748 1,727
~20k 1,635 1,803 1,765

Quality check: paired 180-item battery (MMLU / GSM8K / ARC-Easy, temp 0, thinking-on), int8 vs fused on 0.1.62 — 175/180 on both arms, 0 discordant pairs (McNemar exact p=1.0), zero truncations or request errors. Short prompts (~1.6k) pay ~4% on fused (staged entry), so the fallback retains a niche — the point is that =0 should not silently mean something new.

Suggested fixes (scope for agreement)
  1. Update the three doc passages to the no-MMQ semantics ("=0 selects the int8 prompt fallback; unset for the fused kernels"), and note the PF_FUSED master gate on PF_FUSED_SM75.
  2. Optionally, a start-log note when a user-set PF_FUSED=0 selects the fallback on a no-MMQ build, in the spirit of the existing "STRATA_PF_GROUP_SCRATCH=1 is ignored: the kernels no longer read it" line.
  3. A line in the release notes' Updating section / TROUBLESHOOTING: after the 0.2.16 layout move, re-point run configs that reference data/, serve/, or src/ paths in the clone; the engine fails at start with "cannot open".

No default changes proposed. Happy to send the doc PRs if the scope looks right.

Lingua principale
C++
Stelle
11.6k
Fork
1k
Merge medio
18g 13h
PR unite (30g)
4

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Niko1221/Strata

Tutte le issue di Niko1221/Strata

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.