[Benchmark] DeepSeek-V4.1-Flash, 2x Quadro RTX 8000 (sm_75 source build), 21.6 tok/s

Read docs/OLDER_GPUS.md and the DeepSeek hardware table in MODELS.md; the issue supplies a tested sm_75 source build recipe and explains where it fits. Add guidance that DeepSeek on Turing needs a source build, then check that the wording agrees with the release package support described in MODELS.md.

What did you measure, or what went wrong

A deepseek41 speed measurement on compute capability 7.5, which needed a source build. I could not
find another DeepSeek-V4.1 report for sm_75 in bench/results or in the community report list, so this
may be the first one. Measured 2026-10-27 with Terminal 0.1.51 and the engine built from main at
that time. I re-read the 0.2.11 release notes while writing this (2026-11-07): the DeepSeek packages are
still Linux CUDA 13 / Windows CUDA 13 only, so the source-build note below still applies.

Hardware: 2 x NVIDIA Quadro RTX 8000, 48 GiB each (96 GiB total), no NVLink bridges
(nvidia-smi nvlink --status: "all links are inActive"). 2 x Intel Xeon Gold 6254, 36 cores /
72 threads, 251.5 GiB RAM. Ubuntu 22.04.5, kernel 6.8.0-40-generic, driver 595.99.02 (CUDA 13.2).
Both model sets on SATA SSDs: UD-Q8_K_XL (482 GB) on /dev/sdb, UD-IQ3_S (395 GB) on /dev/sdd3.

Model: unsloth/DeepSeek-V4.1-Flash-1017-GGUF, UD-Q8_K_XL, 508.4 GB in 9 shards (all SHA-256
verified). Context 8192, --vision off, thinking off, temperature 0.

Command: the Terminal's own configuration, unchanged -
strata launch --model-path /srv/models/deepseek-v4.1/UD-Q8_K_XL --model ud-q8_k_xl --family deepseek41 --context 8k --vision off --tune skip --access lan --port 8080 --no-tui --yes
with STRATA_DSV41_EXE pointing at a locally built strata-dsv41 (recipe below).

Results, tools/strata_bench.py, medians of 3 runs per test:

Configuration	Decode tok/s	4K prompt tok/s	4K TTFT
default	21.10 (20.0-22.0)	79.0 (77.7-79.5)	52.2 s
STRATA_BF16_TC=1 + STRATA_SELECT_SIMT=1	21.60 (20.4-23.6)	83.5 (79.1-84.4)	49.4 s
the above + STRATA_SM75_PERF=1	21.10 (19.6-22.5)	85.1 (80.6-86.9)	48.5 s

Short chat TTFT was 0.36-0.81 s in all three arms.

On the Turing switches. OLDER_GPUS.md reports STRATA_BF16_TC=1 giving +20-34% prompt throughput
on one Quadro RTX 8000. On the DeepSeek engine here it gave +5.7% and no decode change: same
direction, much smaller. That 20-34% figure is from the Qwen engine, so the two are not in conflict, but
this is the DeepSeek-engine number for the same card. STRATA_SM75_PERF=1 changed nothing measurable,
which matches DETAILS.md saying that Turing kernel set belongs to the Qwen engine.

Where the time goes (the engine's own per-token breakdown):

decode per token: layers 12.94 ms (of it: routing waits 0.00, CPU lane 22.99,
                  the rest host launches), head wait 31.67, Engram reads 0.19
CPU lane: 9896 of 23760 expert uses (41.6%) went to the CPU
Engram: 48 rows a token, 90.9% served without a read (row cache), 0.24 ms
disk reads: 0.1 MB a token

Varying --cpu-threads isolates the CPU lane:

CPU threads	decode	CPU lane/token	head wait/token
36	22.27 tok/s	22.99 ms	31.67 ms
2	9.80 tok/s	78.73 ms	83.94 ms

Total token time tracks the CPU lane almost exactly (delta 56.5 ms vs 55.7 ms), so on a machine that
cannot hold the experts, the CPU lane and its drain set the speed, not VRAM bandwidth. The GPU layer
chain alone is 12.94 ms/token; how much of the 31.67 ms head wait would survive without the CPU lane is
not something I could separate, so the GPU-only ceiling on these two cards is somewhere around
60-80 tok/s, not a clean 77.

Not measured: 16K prompt and agent session (context was 8192 in these runs; a later 128K
configuration exists but was not benchmarked), prompt-cache behaviour, concurrency, correctness checks.

Per-request JSON exists in the three bench/results/2026-10-27-community-quadro-rtx-8000-* folders and I
can open a results-only PR with them if that is preferred over an issue.

Reproduction status

Measured locally

---

Observations (secondary to the measurement)

1. UD-IQ3_S cannot be used with the deepseek41 engine (engine 0.1.51). The engine stops with
   maya-ds4: layers.0.ffn.experts.: gate / up / down of types IQ2_S / IQ2_S / MXFP4 (gate and up must match; MXFP4 does not mix with the others). Unsloth's dynamic quants mix expert types per layer, so a
   423.7 GB download is unusable. Only UD-Q8_K_XL and the ISTA-DAS sets are listed for DeepSeek, so
   this may be intended; the "not a model size Strata knows" message only appears after the files are on
   disk, so a line in MODELS.md's DeepSeek section saying the engine needs one expert type per layer
   would prevent the download.

2. The OLDER_GPUS.md Turing row and the DeepSeek package. The general support matrix lists RTX 20
   (Turing) as "supported, the ready-made engine", while MODELS.md's DeepSeek hardware table lists RTX 30
   / 40 / 50 only - the two pages are consistent with each other, but a Turing owner who reads
   OLDER_GPUS.md first will expect the ready-made engine to run DeepSeek. One sentence there saying that
   matrix covers the Qwen engine, and that DeepSeek below RTX 30 needs a source build, would save that
   trip.

3. pack_gguf.py --model <IQ3_S dir> succeeds (the tool knows IQ3_S as a dtype), but
   strata launch --model-path rejects the folder as unrecognized_or_unsupported_model_variant before
   any pack is used. Fine as a deliberate limit; naming the accepted sets in the error would make it
   actionable.

---

Appendix: building the DeepSeek engine for Turing (sm_75)

Offered in case it is useful for docs/OLDER_GPUS.md. The Maxwell (sm_52) section there already does
exactly this, so it may be the natural place.

The released DeepSeek packages contain CUDA 13 code for Ampere and Blackwell (the 0.1.51 package said
sm_86, sm_120; RTX 40's sm_89 was added later), so a Turing card needs a source build. Tested on
2 x Quadro RTX 8000 with Terminal 0.1.51; it produces a strata-dsv41 that the Terminal runs when
STRATA_DSV41_EXE points at it.

Run this from the parent directory of the extracted source (Strata/), on a machine with Docker:

mkdir -p build-out

docker run --rm -v "$PWD":/w -w /w nvidia/cuda:12.8.1-devel-ubuntu22.04 bash -c '
  set -euxo pipefail
  export DEBIAN_FRONTEND=noninteractive
  apt-get update -qq
  apt-get install -y -qq --no-install-recommends build-essential ninja-build git python3-pip ca-certificates
  # Ubuntu 22.04 ships cmake 3.22; the tree wants >= 3.24
  pip3 install -q "cmake>=3.28"

  cmake -S /w/Strata -B /w/build \
    -DCMAKE_BUILD_TYPE=Release \
    -DSTRATA_ENABLE_CUDA=ON \
    -DCMAKE_CUDA_ARCHITECTURES=75 \
    -DSTRATA_BUILD_DSV41=ON \
    -G Ninja
  cmake --build /w/build --target strata-dsv41 -j 32
'

cp build/strata-dsv41 build-out/
ldd build-out/strata-dsv41     # expect only libcudart.so.12 / libcublas.so.12 from CUDA

Notes from building it (about 15-20 minutes on 32 cores, most of it the first configure):

• Ubuntu 22.04 is deliberate. A build on 24.04 links against glibc 2.39, so the resulting binary
  refuses to start on 22.04. Build on the oldest distribution you need to support.
• -DSTRATA_BUILD_DSV41=ON is required. Without it the default strata target builds, which is the
  Qwen engine and rejects the DeepSeek arguments (unknown argument: --eos).
• The first configure clones llama.cpp (FetchContent). On a network where github.com is slow or
  blocked, redirect the clone first, for example:
  git config --global url."https://gh-proxy.com/https://github.com/".insteadOf "https://github.com/".
• pip3 install "cmake>=3.28" may time out against the default index; a local mirror works.
• CUDA 13 should also build for sm_75 (13 dropped Pascal and Volta, not Turing), but 12.8 is what the
  released packages use and what this recipe was tested with.

Running it:

export STRATA_DSV41_EXE=/path/to/strata-dsv41
export LD_LIBRARY_PATH=/path/to/cuda/lib64:$LD_LIBRARY_PATH   # if libcudart.so.12 is not on the default path
strata launch --family deepseek41 --model ud-q8_k_xl --model-path /path/to/UD-Q8_K_XL ...