[Engine] Two prompt block scorers enabled at once: WMMA silently wins and no log line is printed

Open Beginner friendly
  • Is something wrong?
#5,693 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 4 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
68/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Active
Tech stack
cpp
Domain
backend, performance

Research direction

Start by searching the engine for STRATA_SELECT_WMMA, STRATA_SELECT_SIMT, and STRATA_SCORES_THREAD to find the scorer dispatch and its existing logging or documentation entry points. Trace how those options are checked to confirm which scorer wins when several are enabled. Done means a log line identifies the selected scorer when options conflict and the precedence is documented.

Written by the indexing model from the issue text.

Description

Environment (anonymized): RX 7900 XTX 24 GB (gfx1100), Ryzen 9 5900X, 62 GiB RAM, Debian 13 (kernel 7.2.12+deb13-amd64),
ROCm 10.1 (HIP 7.16), engine 0.1.59-linux-x86_64-hip-rx7900 from the signed feed (release v0.2.13).
Model Swift-1.5 IQ3_S (qwen4exp: 48 layers, 512 experts / 10 active, native ctx 262144).
Args: --parallel 2 --max-context 262144 --kv int8 --kv-resident 32768 --prefill auto --ple-io direct --expert-cache auto.

What I saw. With both STRATA_SELECT_WMMA=1 (the block scorer on the matrix cores, as the RDNA3 tuning note
suggests) and STRATA_SCORES_THREAD=1 (the exact four-lanes-per-query scorer) set at startup, the engine uses the WMMA
path and the thread scorer never runs. Nothing is logged — not even with STRATA_SCORES_THREAD_TRACE=1. The variables
are present in the engine's /proc/<pid>/environ.

Measurements — one fresh prompt of ~123K unique random tokens per arm, cached_tokens=0, max_tokens 8,
temperature 0, one engine start per arm (the engine's own prompt N tokens ... read in X ms line):

STRATA_SELECT_WMMA STRATA_SCORES_THREAD engagement line? prefill
1 0 — 3174.6 tok/s
1 1 no (masked) 3175.7 tok/s
0 1 qsa exact thread: ... nq=256 active_blocks=16385 capacity=65538 3048.4 tok/s
0 0 — (warp scorer) 1747.2 tok/s

Why it matters. The thread scorer does work on gfx1100 under ROCm 10.1: when WMMA is off it engages and is
1.74x faster than the warp scorer (3048 vs 1747 tok/s) and about 4% slower than WMMA. With WMMA on, the option looks
dead — my first A/B measured a 0.03% difference and I nearly reported the kernel as broken. Others will hit the same.

The dispatch gate itself is fine: max_blocks = max_context/4 + 2 = 65538, active_blocks = 16385 at the scorer's
first window, nq = 256, so active_blocks >= 16385 && active_blocks <= max_blocks && nq >= 64 holds.

Requests. (1) Print one line when more than one prompt block scorer is enabled, naming the one that won (like the
prefill gemm: hipBLASLt tuning enabled (...) line does). (2) Document the precedence between
STRATA_SELECT_WMMA, STRATA_SELECT_SIMT and STRATA_SCORES_THREAD.

Dominant language
C++
Stars
11.6k
Forks
1k
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Niko1221/Strata

All issues in Niko1221/Strata

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.