[Engine] STRATA_HC_REQ8=1 with STRATA_HC_PERSIST=1 (HIP): the persistent hc read takes the int8 codes as bf16

Open Beginner friendly
  • Is something wrong?
#5,560 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 4 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
cpp
Domain
backend, performance

Research direction

Read fused_gr.cu around fused_gr_read_persist at line 3769 and compare its eligibility checks with the non-persistent path around line 1600, which handles s_down. Trace how verify.cpp passes s_down and s_up into the read. Done means the persistent read declines when either scale pointer is present, allowing the default read to handle the int8 rows; the issue proposes this check explicitly.

Written by the indexing model from the issue text.

Description

Area

[Engine] (speed, kernels, models, the server and API)

Desktop app, Terminal or source?

Source (START-HERE.bat, setup.sh or a build)

What happened

With STRATA_HC_REQ8=1, w_down / w_up of each hc read hold int8 codes and q8_scale holds their scales (weights.cpp), and the verifier passes them as a.s_down / a.s_up (verify.cpp:1601). fused_gr_read_persist (fused_gr.cu:3769) only declines for q8_down / q8_up (STRATA_HC_Q8) and q8_mixed (STRATA_QFUSE). It does not check s_down, so with STRATA_HC_PERSIST=1 it launches the bf16 persistent kernel on int8 bytes. The non-persistent path handles this (q8w = m.a[0].s_down != nullptr, fused_gr.cu:1600).

Expected: the persistent read declines and the default read (which knows the int8 rows) runs, as it does for STRATA_HC_Q8.

Suggested fix, one line next to the HC_Q8 check in fused_gr_read_persist:

if (a[0].s_down != nullptr || a[0].s_up != nullptr) return false;   // STRATA_HC_REQ8: the default read
Smallest steps or command that reproduce it

Not run. By the code: one HIP card (STRATA_HC_REQ8 refuses a layer split), STRATA_HC_REQ8=1 STRATA_HC_PERSIST=1, with STRATA_QFUSE / STRATA_QFUSE_HC off (the default off gfx12, e.g. gfx103x). Nothing at start-up turns HC_PERSIST off when HC_REQ8 is on (generate.cpp:5116). With QFUSE on, q8_mixed is set and the persistent read already declines, which hides it.

Reproduction status

Unconfirmed static analysis (I read code or ran a scanner; I did not see it fail)

Strata version

main 146df3c4 (source)

Graphics card, memory and system

Found while working on 2x RX 6900 XT 16 GB (gfx1030) · 128 GB RAM · Ubuntu 26.04, ROCm 10.0; not run on this combination (our cards run a layer split)

Model, size and settings

Any model whose hc projections are BF16 in the pack (seen on Qwen3.8-Flash-Next IQ3_S)

Logs

No response

Found with an AI coding assistant by reading the code; not reproduced at runtime.

Dominant language
C++
Stars
11.6k
Forks
1k
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Niko1221/Strata

All issues in Niko1221/Strata

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.