[Benchmark] Windows large pages (Lock pages in memory) on RTX 5080 + 96 GB, 0.2.10: arena on 2 MiB pages, no decode or prompt gain (ABAB)

Chiusa Adatta ai principianti
  • Qualcosa non va?
#5,238 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 4 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
1/5
Tempo stimato
1-3 ore
Idoneità per principianti
74/100
Tipo di issue
Documentazione
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp
Ambito
documentation

Direzione di ricerca

Leggi docs/DETAILS.md e i risultati del benchmark in questa issue. Aggiorna la sezione sulla concessione del diritto «Blocca le pagine in memoria» per descrivere il risultato misurato sul PC Windows testato e menzionare STRATA_NO_LARGEPAGES=1 come opzione per disattivarlo. Il lavoro è completato quando la documentazione riflette le misurazioni riportate senza presentarle come una garanzia generale di prestazioni.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What did you measure, or what went wrong

#1412 showed that "Lock pages in memory" moves the expert arena to 2 MiB large pages, and it noted that the effect on tok/s had not been measured. Here is an interleaved A/B of that effect on the 0.2.10 release.

Setup
  • RTX 5080 16 GB (sm_120, PCIe 5.0 x8, the only GPU), Ryzen 9 9950X3D, 96 GB DDR5-6400, Windows 11 Pro 10.0.26200, driver 617.42
  • Strata 0.2.10, Control and engine 0.1.56 windows-x86_64-cuda-cuda13 (release, c5ab5d40)
  • "Lock pages in memory" was granted to the account in secpol.msc, then the PC was rebooted. The test started 2 minutes after logon, with nothing else running. whoami /priv lists SeLockMemoryPrivilege; before the grant, the log said large pages refused ... VirtualAlloc error 1314.
  • Arm A: large pages, the engine's default once the privilege is held. Log: expert arena: cudaHostRegister PORTABLE ok; large pages (2097152 B).
  • Arm B: STRATA_NO_LARGEPAGES=1. Log: large pages skipped (STRATA_NO_LARGEPAGES); using 4 KB pages.
  • Everything else is identical between arms:
    • OrcaRouter Qwen3.8-Flash-Next-Uncensored IQ3_XXS, hand-made run config: 128K, MTP, cyrillic draft subset, --vram-reserve-mib 1472, 49.80 GiB arena, 4161 GPU slots.
    • The setup-made qwen-128k preset: Unsloth UD-IQ4_XS, DFlash2 Auto, 55.43 GiB arena, 2357 slots.
Method
  • Order: Orca A, B, A, B, then qwen-128k A, B. Each arm is a fresh start.
  • Each start runs strata bench run --url ... --skip-model-hash (standard-v1) and then a short Russian probe: 5 prompts × 3 seeds, thinking off, 220–700 tokens. The probe script is in #5107.
  • Values are medians in tok/s from the engine's timings.
Results
Model, pages starts Russian: code / text edit / JSON / prose / short English decode: short / 4K / 16K prompt 4K / 16K arena load sanity
Orca IQ3_XXS, 2 MiB 2 123 / 163 / 137 / 122 / 115 98 / 93 / 99 2159 / 4132 6.73, 6.75 GiB/s 12/12 ×2
Orca IQ3_XXS, 4 KB 2 129 / 165 / 134 / 128 / 120 103 / 99 / 100 2160 / 4123 6.74, 6.75 GiB/s 12/12 ×2
qwen-128k UD-IQ4_XS, 2 MiB 1 109 / 136 / 95 / 87 / 94 84 / 83 / 79 1874 / 3625 5.95 GiB/s 12/12
qwen-128k UD-IQ4_XS, 4 KB 1 113 / 136 / 101 / 87 / 91 85 / 86 / 82 1873 / 3625 6.27 GiB/s 12/12
  • No gain from large pages on decode, prompt or load speed. If anything, the 4 KB arm is 2–6 % faster. On the interleaved Orca pairs it is ahead in 7 of 8 decode metrics, which is within the 3–6 % spread I measured between two starts of the same config, but it points one way.
  • My guess: with most routed experts in the GPU cache and the rest read by the GPU from pinned host memory over PCIe, the CPU's page size is not on the hot path. I have not checked this beyond these numbers.
  • Reliability: all three large-page starts in this boot got their arena on 2 MiB pages, including the third start after two restarts. I did not see the 1450 from #2295 (same 5080 + 96 GB class of PC). That is a short session, not a soak.
Suggestion

Should docs/DETAILS.md, where it describes granting "Lock pages in memory", say that on this kind of PC it was measured with no speed gain? Then nobody grants a user right and logs off for nothing. STRATA_NO_LARGEPAGES=1 worked as the off switch.

The per-request data is kept; I can add it as a results-only PR if useful.

Reproduction status

Measured locally

Lingua principale
C++
Stelle
11.6k
Fork
1k
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Niko1221/Strata

Tutte le issue di Niko1221/Strata

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.