[Docs] 0.2.16 migration: PF_FUSED=0 passages still say "keeps MMQ"; layout move breaks old-path run configs at restart

Open Beginner friendly
#6,251 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 5 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
82/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
cpp
Domain
documentation

Research direction

Read docs/DETAILS.md, docs/RADEON_780M.md, and docs/STRIX_HALO.md to update the STRATA_PF_FUSED=0 guidance, including the PF_FUSED master gate. Check the 0.2.16 release notes' Updating section and TROUBLESHOOTING for guidance on moved clone paths such as data/ and serve/. Done when the docs explain the new fallback behavior and tell users to update old paths.

Written by the indexing model from the issue text.

Description

needs-format

Area: [Docs]
Desktop app, Terminal or source? Source — manual clone built with CUDA 13.4 (-DSTRATA_GEOMETRY=flash_next, arch 75;86;120), served under systemd.

What happened

Two related migration gaps when updating a manual-clone install from v0.2.15 (engine 0.1.61) to v0.2.16 (engine 0.1.62). Both were observed on the chassis below; both are documentation/guidance gaps rather than kernel bugs.

Finding 1 — STRATA_PF_FUSED=0 no longer "keeps MMQ", and the docs still say it does.

  • Problem: three doc passages describe the pre-0.2.16 semantics in present tense — docs/DETAILS.md, docs/RADEON_780M.md, and docs/STRIX_HALO.md all say "STRATA_PF_FUSED=0 keeps MMQ."
  • Expected (per those docs): =0 keeps the MMQ prompt path.
  • Actual: v0.2.16 has no MMQ. A user-set =0 selects the native int8 prompt fallback (start log: "every chunk runs 48 of 48 layers on the int8 kernels"), while unset gets the fused kernels via the build defaults. A config carried over from the MMQ era — where =0 was a meaningful "keep the old path" choice — silently lands on a different path after the update, measured 7–9% slower at 7k–20k-token prompts (table below). Related: STRATA_PF_FUSED_SM75=1 alone is inert — the master PF_FUSED gates it (the start log stays "int8 kernels" with the master at 0).

Finding 2 — the layout move invalidates old tree paths in run configs, at restart rather than at update.

  • Problem: data/expert-profile.bin moved to terminal/data/expert-profile.bin (and serve/server.py → terminal/serve/server.py).
  • Expected: updating a manual clone either keeps documented config paths working or says they must be re-pointed.
  • Actual: a config with "--expert-profile", "<clone>/data/expert-profile.bin" keeps serving until the next restart (the running engine never re-reads the path); the new engine then exits 1 with read_expert_profile: cannot open <clone>/data/expert-profile.bin and systemd loops. Nothing in the release notes' Updating section or TROUBLESHOOTING mentions re-pointing old paths.
Smallest steps or command that reproduce it

Finding 1:

  1. Build v0.2.16 (no-MMQ) and serve an IQ3_XXS Flash-Next pack with STRATA_PF_FUSED=0 in the server config's env.
  2. The start log reads "prompt kernels: every chunk runs 48 of 48 layers on the int8 kernels."
  3. Measure prefill at 7k–20k-token prompts; set STRATA_PF_FUSED=1 (+ STRATA_PF_FUSED_SM75=1 for the sm_75 stage) and restart: the log reads "chunks of 1024 tokens or more run 48 of 48 layers on the fused kernels", and prefill rises 7–9% at those lengths.

Finding 2:

  1. On v0.2.15, serve with "--expert-profile", "<clone>/data/expert-profile.bin" (the tracked location at that tag).
  2. Update the clone to v0.2.16 (the file is now at terminal/data/expert-profile.bin) and restart the service.
  3. The engine exits 1 with read_expert_profile: cannot open <clone>/data/expert-profile.bin; systemd restart-loops until the config path is fixed.
Reproduction status

Reproduced locally — both findings, on the chassis below. Finding 1 measured across four arms × three passes; finding 2 caused an actual restart loop during our update.

Strata version

Strata 0.2.16 · engine 0.1.62 (upstream tag v0.2.16 plus two local one-line carries, #6084 and #6107, neither of which touches the prompt kernels). Baseline comparisons: v0.2.15 · engine 0.1.61. Doc passages quoted from origin/main at 64524fb9f.

Graphics card, memory and system

RTX 2080 Ti 11 GB (sm_75) + RTX 5060 Ti 16 GB (sm_120) + RTX 3080 Ti 12 GB (sm_86), PCIe, no P2P · 188 GB RAM · Ubuntu 26.04.1 LTS, kernel 7.3.4 · CUDA 13.4.

Model, size and settings

Qwen3.8-Flash-Next UD-IQ3_XXS · --max-context 524288 · --layer-split 18,33 · --pipeline-windows 2 · --kv k8v4 · vision on · thinking-on serving (temp 0 for the measurements).

Logs (sanitized)

Finding 1, the two start-log states:

strata: prompt kernels: every chunk runs 48 of 48 layers on the int8 kernels.
strata: prompt kernels: chunks of 1024 tokens or more run 48 of 48 layers on the fused kernels; shorter chunks run 48 on the staged fused entry.

Finding 2, the restart failure (path generalized):

read_expert_profile: cannot open <clone>/data/expert-profile.bin

Engine exit 1, systemd restart loop; fixed by pointing the config at the new location.

Measurements (evidence for finding 1)

Prefill tok/s, medians of three passes per arm, randomized passages, ten-minute settles, identical prompts across arms:

prompt tokens int8 (PF_FUSED=0) fused (PF_FUSED=1 + PF_FUSED_SM75=1) v0.2.15 MMQ+SM75
~7k 1,042 1,123 1,140
~9.3k 975 1,071 1,069
~11.6k 1,210 1,323 1,316
~15.5k 1,569 1,748 1,727
~20k 1,635 1,803 1,765

Quality check: paired 180-item battery (MMLU / GSM8K / ARC-Easy, temp 0, thinking-on), int8 vs fused on 0.1.62 — 175/180 on both arms, 0 discordant pairs (McNemar exact p=1.0), zero truncations or request errors. Short prompts (~1.6k) pay ~4% on fused (staged entry), so the fallback retains a niche — the point is that =0 should not silently mean something new.

Suggested fixes (scope for agreement)
  1. Update the three doc passages to the no-MMQ semantics ("=0 selects the int8 prompt fallback; unset for the fused kernels"), and note the PF_FUSED master gate on PF_FUSED_SM75.
  2. Optionally, a start-log note when a user-set PF_FUSED=0 selects the fallback on a no-MMQ build, in the spirit of the existing "STRATA_PF_GROUP_SCRATCH=1 is ignored: the kernels no longer read it" line.
  3. A line in the release notes' Updating section / TROUBLESHOOTING: after the 0.2.16 layout move, re-point run configs that reference data/, serve/, or src/ paths in the clone; the engine fails at start with "cannot open".

No default changes proposed. Happy to send the doc PRs if the scope looks right.

Dominant language
C++
Stars
11.6k
Forks
1k
Avg merge
18d 13h
Merged PRs (30d)
4

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Niko1221/Strata

All issues in Niko1221/Strata

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.