[Docs] 0.2.16 migration: PF_FUSED=0 passages still say "keeps MMQ"; layout move breaks old-path run configs at restart
Maintainers usually reply within 5 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
- Issue type
- Documentation
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- cpp
- Domain
- documentation
Research direction
Read docs/DETAILS.md, docs/RADEON_780M.md, and docs/STRIX_HALO.md to update the STRATA_PF_FUSED=0 guidance, including the PF_FUSED master gate. Check the 0.2.16 release notes' Updating section and TROUBLESHOOTING for guidance on moved clone paths such as data/ and serve/. Done when the docs explain the new fallback behavior and tell users to update old paths.
Written by the indexing model from the issue text.
Description
Area: [Docs]
Desktop app, Terminal or source? Source — manual clone built with CUDA 13.4 (-DSTRATA_GEOMETRY=flash_next, arch 75;86;120), served under systemd.
What happened
Two related migration gaps when updating a manual-clone install from v0.2.15 (engine 0.1.61) to v0.2.16 (engine 0.1.62). Both were observed on the chassis below; both are documentation/guidance gaps rather than kernel bugs.
Finding 1 — STRATA_PF_FUSED=0 no longer "keeps MMQ", and the docs still say it does.
- Problem: three doc passages describe the pre-0.2.16 semantics in present tense —
docs/DETAILS.md,docs/RADEON_780M.md, anddocs/STRIX_HALO.mdall say "STRATA_PF_FUSED=0keeps MMQ." - Expected (per those docs):
=0keeps the MMQ prompt path. - Actual: v0.2.16 has no MMQ. A user-set
=0selects the native int8 prompt fallback (start log: "every chunk runs 48 of 48 layers on the int8 kernels"), while unset gets the fused kernels via the build defaults. A config carried over from the MMQ era — where=0was a meaningful "keep the old path" choice — silently lands on a different path after the update, measured 7–9% slower at 7k–20k-token prompts (table below). Related:STRATA_PF_FUSED_SM75=1alone is inert — the masterPF_FUSEDgates it (the start log stays "int8 kernels" with the master at 0).
Finding 2 — the layout move invalidates old tree paths in run configs, at restart rather than at update.
- Problem:
data/expert-profile.binmoved toterminal/data/expert-profile.bin(andserve/server.py→terminal/serve/server.py). - Expected: updating a manual clone either keeps documented config paths working or says they must be re-pointed.
- Actual: a config with
"--expert-profile", "<clone>/data/expert-profile.bin"keeps serving until the next restart (the running engine never re-reads the path); the new engine then exits 1 withread_expert_profile: cannot open <clone>/data/expert-profile.binand systemd loops. Nothing in the release notes' Updating section or TROUBLESHOOTING mentions re-pointing old paths.
Smallest steps or command that reproduce it
Finding 1:
- Build v0.2.16 (no-MMQ) and serve an IQ3_XXS Flash-Next pack with
STRATA_PF_FUSED=0in the server config's env. - The start log reads "prompt kernels: every chunk runs 48 of 48 layers on the int8 kernels."
- Measure prefill at 7k–20k-token prompts; set
STRATA_PF_FUSED=1(+STRATA_PF_FUSED_SM75=1for the sm_75 stage) and restart: the log reads "chunks of 1024 tokens or more run 48 of 48 layers on the fused kernels", and prefill rises 7–9% at those lengths.
Finding 2:
- On v0.2.15, serve with
"--expert-profile", "<clone>/data/expert-profile.bin"(the tracked location at that tag). - Update the clone to v0.2.16 (the file is now at
terminal/data/expert-profile.bin) and restart the service. - The engine exits 1 with
read_expert_profile: cannot open <clone>/data/expert-profile.bin; systemd restart-loops until the config path is fixed.
Reproduction status
Reproduced locally — both findings, on the chassis below. Finding 1 measured across four arms × three passes; finding 2 caused an actual restart loop during our update.
Strata version
Strata 0.2.16 · engine 0.1.62 (upstream tag v0.2.16 plus two local one-line carries, #6084 and #6107, neither of which touches the prompt kernels). Baseline comparisons: v0.2.15 · engine 0.1.61. Doc passages quoted from origin/main at 64524fb9f.
Graphics card, memory and system
RTX 2080 Ti 11 GB (sm_75) + RTX 5060 Ti 16 GB (sm_120) + RTX 3080 Ti 12 GB (sm_86), PCIe, no P2P · 188 GB RAM · Ubuntu 26.04.1 LTS, kernel 7.3.4 · CUDA 13.4.
Model, size and settings
Qwen3.8-Flash-Next UD-IQ3_XXS · --max-context 524288 · --layer-split 18,33 · --pipeline-windows 2 · --kv k8v4 · vision on · thinking-on serving (temp 0 for the measurements).
Logs (sanitized)
Finding 1, the two start-log states:
strata: prompt kernels: every chunk runs 48 of 48 layers on the int8 kernels.
strata: prompt kernels: chunks of 1024 tokens or more run 48 of 48 layers on the fused kernels; shorter chunks run 48 on the staged fused entry.
Finding 2, the restart failure (path generalized):
read_expert_profile: cannot open <clone>/data/expert-profile.bin
Engine exit 1, systemd restart loop; fixed by pointing the config at the new location.
Measurements (evidence for finding 1)
Prefill tok/s, medians of three passes per arm, randomized passages, ten-minute settles, identical prompts across arms:
| prompt tokens | int8 (PF_FUSED=0) |
fused (PF_FUSED=1 + PF_FUSED_SM75=1) |
v0.2.15 MMQ+SM75 |
|---|---|---|---|
| ~7k | 1,042 | 1,123 | 1,140 |
| ~9.3k | 975 | 1,071 | 1,069 |
| ~11.6k | 1,210 | 1,323 | 1,316 |
| ~15.5k | 1,569 | 1,748 | 1,727 |
| ~20k | 1,635 | 1,803 | 1,765 |
Quality check: paired 180-item battery (MMLU / GSM8K / ARC-Easy, temp 0, thinking-on), int8 vs fused on 0.1.62 — 175/180 on both arms, 0 discordant pairs (McNemar exact p=1.0), zero truncations or request errors. Short prompts (~1.6k) pay ~4% on fused (staged entry), so the fallback retains a niche — the point is that =0 should not silently mean something new.
Suggested fixes (scope for agreement)
- Update the three doc passages to the no-MMQ semantics ("
=0selects the int8 prompt fallback; unset for the fused kernels"), and note thePF_FUSEDmaster gate onPF_FUSED_SM75. - Optionally, a start-log note when a user-set
PF_FUSED=0selects the fallback on a no-MMQ build, in the spirit of the existing "STRATA_PF_GROUP_SCRATCH=1is ignored: the kernels no longer read it" line. - A line in the release notes' Updating section / TROUBLESHOOTING: after the 0.2.16 layout move, re-point run configs that reference
data/,serve/, orsrc/paths in the clone; the engine fails at start with "cannot open".
No default changes proposed. Happy to send the doc PRs if the scope looks right.
- Dominant language
- C++
- Stars
- 11.6k
- Forks
- 1k
- Avg merge
- 18d 13h
- Merged PRs (30d)
- 4
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Niko1221/Strata
-
[TUI] A leftover run config marks a model installed: the list checks the file name, not the filesOpen
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 5 days
-
needs-format
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Niko1221/Strata#6229 · 1 comment ·
Maintainers usually reply within 5 days
-
[Engine] qsa_rows_bench: --ctx below 32768 makes the dense step read outside the page table (Xid 31)Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 5 days
-
[Engine] explicit --layer-split reloads CUDA0 dense weights a second time at startup (0 MiB freed)Possibly taken @wronglebowsk claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 5 days
-
[Other] Terminal: shell tab completion for the strata commandPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Niko1221/Strata#6086 · 2 comments ·
Maintainers usually reply within 5 days
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
metasequoiaime/msime#11666 ·
Maintainers usually reply within 1 day
-
DAE2 crashes on call.without.effects with a non ref.func expressionPossibly taken @stevenfontanella claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
WebAssembly/binaryen#9404 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 65/100
CodeZombie/goodluckOS#178 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
CVO-Org/immersive-cigs-rewrite#313 ·
Maintainers usually reply within 1 day
-
windows-latest (20261116.315.1) ships with incompatible MSVC (18.12) and pinned LLVM/Clang (20)Openbug report
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
actions/runner-images#15367 · 4 comments ·
Maintainers usually reply within 2 days