[Benchmark] DeepSeek-V4.1-Flash, 2x Quadro RTX 8000 (sm_75 source build), 21.6 tok/s

Leggi docs/OLDER_GPUS.md e la tabella hardware di DeepSeek in MODELS.md; l’issue fornisce una procedura collaudata per compilare dai sorgenti per sm_75 e spiega dove inserirla. Aggiungi un’indicazione che DeepSeek su Turing richiede una compilazione dai sorgenti, quindi verifica che la formulazione sia coerente con il supporto dei pacchetti della release descritto in MODELS.md.

What did you measure, or what went wrong

A deepseek41 speed measurement on compute capability 7.5, which needed a source build. I could not
find another DeepSeek-V4.1 report for sm_75 in bench/results or in the community report list, so this
may be the first one. Measured 2026-10-27 with Terminal 0.1.51 and the engine built from main at
that time. I re-read the 0.2.11 release notes while writing this (2026-11-07): the DeepSeek packages are
still Linux CUDA 13 / Windows CUDA 13 only, so the source-build note below still applies.

Hardware: 2 x NVIDIA Quadro RTX 8000, 48 GiB each (96 GiB total), no NVLink bridges
(nvidia-smi nvlink --status: "all links are inActive"). 2 x Intel Xeon Gold 6254, 36 cores /
72 threads, 251.5 GiB RAM. Ubuntu 22.04.5, kernel 6.8.0-40-generic, driver 595.99.02 (CUDA 13.2).
Both model sets on SATA SSDs: UD-Q8_K_XL (482 GB) on /dev/sdb, UD-IQ3_S (395 GB) on /dev/sdd3.

Model: unsloth/DeepSeek-V4.1-Flash-1017-GGUF, UD-Q8_K_XL, 508.4 GB in 9 shards (all SHA-256
verified). Context 8192, --vision off, thinking off, temperature 0.

Command: the Terminal's own configuration, unchanged -
strata launch --model-path /srv/models/deepseek-v4.1/UD-Q8_K_XL --model ud-q8_k_xl --family deepseek41 --context 8k --vision off --tune skip --access lan --port 8080 --no-tui --yes
with STRATA_DSV41_EXE pointing at a locally built strata-dsv41 (recipe below).

Results, tools/strata_bench.py, medians of 3 runs per test:

Configuration	Decode tok/s	4K prompt tok/s	4K TTFT
default	21.10 (20.0-22.0)	79.0 (77.7-79.5)	52.2 s
STRATA_BF16_TC=1 + STRATA_SELECT_SIMT=1	21.60 (20.4-23.6)	83.5 (79.1-84.4)	49.4 s
the above + STRATA_SM75_PERF=1	21.10 (19.6-22.5)	85.1 (80.6-86.9)	48.5 s

Short chat TTFT was 0.36-0.81 s in all three arms.

On the Turing switches. OLDER_GPUS.md reports STRATA_BF16_TC=1 giving +20-34% prompt throughput
on one Quadro RTX 8000. On the DeepSeek engine here it gave +5.7% and no decode change: same
direction, much smaller. That 20-34% figure is from the Qwen engine, so the two are not in conflict, but
this is the DeepSeek-engine number for the same card. STRATA_SM75_PERF=1 changed nothing measurable,
which matches DETAILS.md saying that Turing kernel set belongs to the Qwen engine.

Where the time goes (the engine's own per-token breakdown):

decode per token: layers 12.94 ms (of it: routing waits 0.00, CPU lane 22.99,
                  the rest host launches), head wait 31.67, Engram reads 0.19
CPU lane: 9896 of 23760 expert uses (41.6%) went to the CPU
Engram: 48 rows a token, 90.9% served without a read (row cache), 0.24 ms
disk reads: 0.1 MB a token

Varying --cpu-threads isolates the CPU lane:

CPU threads	decode	CPU lane/token	head wait/token
36	22.27 tok/s	22.99 ms	31.67 ms
2	9.80 tok/s	78.73 ms	83.94 ms

Total token time tracks the CPU lane almost exactly (delta 56.5 ms vs 55.7 ms), so on a machine that
cannot hold the experts, the CPU lane and its drain set the speed, not VRAM bandwidth. The GPU layer
chain alone is 12.94 ms/token; how much of the 31.67 ms head wait would survive without the CPU lane is
not something I could separate, so the GPU-only ceiling on these two cards is somewhere around
60-80 tok/s, not a clean 77.

Not measured: 16K prompt and agent session (context was 8192 in these runs; a later 128K
configuration exists but was not benchmarked), prompt-cache behaviour, concurrency, correctness checks.

Per-request JSON exists in the three bench/results/2026-10-27-community-quadro-rtx-8000-* folders and I
can open a results-only PR with them if that is preferred over an issue.

Reproduction status

Measured locally

---

Observations (secondary to the measurement)

1. UD-IQ3_S cannot be used with the deepseek41 engine (engine 0.1.51). The engine stops with
   maya-ds4: layers.0.ffn.experts.: gate / up / down of types IQ2_S / IQ2_S / MXFP4 (gate and up must match; MXFP4 does not mix with the others). Unsloth's dynamic quants mix expert types per layer, so a
   423.7 GB download is unusable. Only UD-Q8_K_XL and the ISTA-DAS sets are listed for DeepSeek, so
   this may be intended; the "not a model size Strata knows" message only appears after the files are on
   disk, so a line in MODELS.md's DeepSeek section saying the engine needs one expert type per layer
   would prevent the download.

2. The OLDER_GPUS.md Turing row and the DeepSeek package. The general support matrix lists RTX 20
   (Turing) as "supported, the ready-made engine", while MODELS.md's DeepSeek hardware table lists RTX 30
   / 40 / 50 only - the two pages are consistent with each other, but a Turing owner who reads
   OLDER_GPUS.md first will expect the ready-made engine to run DeepSeek. One sentence there saying that
   matrix covers the Qwen engine, and that DeepSeek below RTX 30 needs a source build, would save that
   trip.

3. pack_gguf.py --model <IQ3_S dir> succeeds (the tool knows IQ3_S as a dtype), but
   strata launch --model-path rejects the folder as unrecognized_or_unsupported_model_variant before
   any pack is used. Fine as a deliberate limit; naming the accepted sets in the error would make it
   actionable.

---

Appendix: building the DeepSeek engine for Turing (sm_75)

Offered in case it is useful for docs/OLDER_GPUS.md. The Maxwell (sm_52) section there already does
exactly this, so it may be the natural place.

The released DeepSeek packages contain CUDA 13 code for Ampere and Blackwell (the 0.1.51 package said
sm_86, sm_120; RTX 40's sm_89 was added later), so a Turing card needs a source build. Tested on
2 x Quadro RTX 8000 with Terminal 0.1.51; it produces a strata-dsv41 that the Terminal runs when
STRATA_DSV41_EXE points at it.

Run this from the parent directory of the extracted source (Strata/), on a machine with Docker:

mkdir -p build-out

docker run --rm -v "$PWD":/w -w /w nvidia/cuda:12.8.1-devel-ubuntu22.04 bash -c '
  set -euxo pipefail
  export DEBIAN_FRONTEND=noninteractive
  apt-get update -qq
  apt-get install -y -qq --no-install-recommends build-essential ninja-build git python3-pip ca-certificates
  # Ubuntu 22.04 ships cmake 3.22; the tree wants >= 3.24
  pip3 install -q "cmake>=3.28"

  cmake -S /w/Strata -B /w/build \
    -DCMAKE_BUILD_TYPE=Release \
    -DSTRATA_ENABLE_CUDA=ON \
    -DCMAKE_CUDA_ARCHITECTURES=75 \
    -DSTRATA_BUILD_DSV41=ON \
    -G Ninja
  cmake --build /w/build --target strata-dsv41 -j 32
'

cp build/strata-dsv41 build-out/
ldd build-out/strata-dsv41     # expect only libcudart.so.12 / libcublas.so.12 from CUDA

Notes from building it (about 15-20 minutes on 32 cores, most of it the first configure):

• Ubuntu 22.04 is deliberate. A build on 24.04 links against glibc 2.39, so the resulting binary
  refuses to start on 22.04. Build on the oldest distribution you need to support.
• -DSTRATA_BUILD_DSV41=ON is required. Without it the default strata target builds, which is the
  Qwen engine and rejects the DeepSeek arguments (unknown argument: --eos).
• The first configure clones llama.cpp (FetchContent). On a network where github.com is slow or
  blocked, redirect the clone first, for example:
  git config --global url."https://gh-proxy.com/https://github.com/".insteadOf "https://github.com/".
• pip3 install "cmake>=3.28" may time out against the default index; a local mirror works.
• CUDA 13 should also build for sm_75 (13 dropped Pascal and Volta, not Turing), but 12.8 is what the
  released packages use and what this recipe was tested with.

Running it:

export STRATA_DSV41_EXE=/path/to/strata-dsv41
export LD_LIBRARY_PATH=/path/to/cuda/lib64:$LD_LIBRARY_PATH   # if libcudart.so.12 is not on the default path
strata launch --family deepseek41 --model ud-q8_k_xl --model-path /path/to/UD-Q8_K_XL ...