Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Other] Deepseek Question Can it be tuned further?

Aperta
  • Qualcosa non va?
#4,530 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
35/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
cpp, python
Ambito
backend, machine-learning

Direzione di ricerca

The issue includes launch configuration, engine logs, and a GPU memory graph. Start by reading the logs around tensor parallelism, DSpark loading, mmap experts, and GPU layer placement, then compare the graph with the reported VRAM allocation. Determine whether the configuration can improve the reported prompt and generation speeds and why shared GPU memory is absent; document the relevant findings and any reproducible setting changes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Greetings,
Can i squeeze more than pp 600/21 tg ? with my config? And why not showing Shared GPU memory in the graph as other models are?
VRAM 48 GB
RAM 256 GB

@echo off

title Strata deepseek-v4.1-flash-IST-DAS-2.25bpw

cd /d "A:\0_strata"

"a:\0_strata\.venv\Scripts\python.exe" "A:\0_strata\serve\server.py" "--engine" "strata" "--config" "A:\0_strata\strata-deepseek-v4.1-flash-ist-das-2.25bpw.json" "--host" "0.0.0.0" "--port" "8082" "--open"

if errorlevel 1 pause


{
 "exe": "A:\\0_strata\\engine-dsv41\\0.1.52-windows-x86_64-cuda-cuda13\\strata-dsv41.exe",
 "args": [
  "--pack",
  "A:\\Strata-data\\packs\\deepseek-v4.1-flash-ist-das-2.25bpw",
  "--max-context",
  "262144",
  "--experts",
  "auto",
  "--eos",
  "1",
  "--vision",
  "cpu",
  "--dspark",
  "auto",
  "--dspark-conf",
  "0.65"
 ],
 "cwd": "A:\\0_strata",
 "tokenizer": "A:\\Strata-data\\packs\\deepseek-v4.1-flash-ist-das-2.25bpw\\tokenizer",
 "model_name": "deepseek-v4.1-flash",
 "arch": "deepseek41",
 "log": "A:\\0_strata\\strata-deepseek-v4.1-flash-ist-das-2.25bpw.log",
 "lib_dirs": [
  "A:\\0_strata\\engine-dsv41\\0.1.52-windows-x86_64-cuda-cuda13"
 ],
 "port": 8080,
 "engine_ready_s": 3600,
 "reasoning_budget_tokens": 8192,
 "max_tokens": 15000,
 "gpu": 2,
 "gpus_asked": true,
 "args_written": [
  "--pack",
  "A:\\Strata-data\\packs\\deepseek-v4.1-flash-ist-das-2.25bpw",
  "--max-context",
  "262144",
  "--eos",
  "1",
  "--vision",
  "cpu",
  "--dspark",
  "auto",
  "--dspark-conf",
  "0.65"
 ]
}

{"version": "0.1.52", "source": "f-027-rel a42cc3d7d", "commit": "a42cc3d7df70", "archs": [86, 89, 120], "ptx": true, "cuda": "13.0", "portable": true, "dsv41": true}

[strata] 2026-10-28 20:00:43 engine started: A:\0_strata\engine-dsv41\0.1.52-windows-x86_64-cuda-cuda13\strata-dsv41.exe --serve --pack A:\Strata-data\packs\deepseek-v4.1-flash-ist-das-2.25bpw --max-context 262144 --experts auto --eos 1 --vision cpu --dspark auto --dspark-conf 0.65
ds4: tensor parallel 2 -> 1: the experts' 256-value blocks split n_ff 2304 no other way (each GPU its own layers)
ds4: prompts in chunks of 4096 tokens (the experts are copied into RAM, 48 GiB of graphics memory; --prefill-chunk N changes it)
ds4: DSpark not loaded: the routed experts (149 GB) are far more than the GPU holds (49 GB), where its verify rounds read 3.3x a token's bytes and gain nothing (--dspark on loads it anyway)
ds4: the experts VRAM leaves over (~109.5 GB) are read in place from the model's files (--mmap-experts off copies them into RAM) (223.2 GB of RAM free to keep them: the most-routed 109.5 GB read ahead; prompts in chunks of 4096)
ds4: --mmap-experts auto: the CPU lane's experts (110.6 GB) fit the RAM this process can get (137.7 GB beside 9 GB for the rest): copied in at load, no demand store (--mmap-experts on keeps them in place)
ds4: 11319 of the CPU lane's 11319 experts copied into RAM (109.8 GB, 1 NUMA part) in 34.0 s
ds4: prompt path: the chunked pass from 96 tokens, windows of 16 below
ds4: CUDA0 layers [0, 39] + head: weights 5.86 GB, experts 39.60 GB (4041 of 15360 in VRAM), caches 0.24 GB, 573 MiB free
ds4: 11319 routed experts on the CPU lane (109.8 GB): 11319 copied into RAM (1 NUMA parts), the rest in place in the model's files; 24 threads
maya ds4 serve: GEN <max_new> [keys] <ids>; GENI <max_new> [keys] <pictures> <ids>; STOP, QUIT
ds4 slots: folder A:\0_strata\ds4-slots: 3 conversation(s) kept from earlier runs (0.05 GB)
maya ds4 serve: prompt 649 tokens = 0 reused + 649 read in 11607 ms (55.9 tok/s), 760 generated in 28624 ms (26.55 tok/s); CPU lane 10.9% of the experts
ds4 slots: saved a 1408-token conversation (0.01 GB in RAM in 0.0 s); 1 in RAM, 3 in A:\0_strata\ds4-slots
maya ds4 serve: prompt 19113 tokens = 0 reused + 19113 read in 30637 ms (623.8 tok/s), 88 generated in 3802 ms (23.14 tok/s); CPU lane 1.5% of the experts
maya ds4 serve: prompt 19301 tokens = 19200 reused + 101 read in 1890 ms (53.5 tok/s), 83 generated in 3300 ms (25.16 tok/s); CPU lane 1.6% of the experts
maya ds4 serve: prompt 19534 tokens = 19383 reused + 151 read in 2239 ms (67.4 tok/s), 294 generated in 13101 ms (22.44 tok/s); CPU lane 2.4% of the experts
maya ds4 serve: prompt 19927 tokens = 19827 reused + 100 read in 1682 ms (59.5 tok/s), 227 generated in 10167 ms (22.33 tok/s); CPU lane 2.9% of the experts
maya ds4 serve: prompt 20587 tokens = 20153 reused + 434 read in 3088 ms (140.5 tok/s), 269 generated in 10837 ms (24.82 tok/s); CPU lane 3.4% of the experts
maya ds4 serve: prompt 20869 tokens = 20855 reused + 14 read in 318 ms (44.0 tok/s), 196 generated in 7874 ms (24.89 tok/s); CPU lane 3.8% of the experts
maya ds4 serve: prompt 36907 tokens = 21064 reused + 15843 read in 18801 ms (842.7 tok/s), 226 generated in 9215 ms (24.52 tok/s); CPU lane 2.6% of the experts
maya ds4 serve: prompt 37624 tokens = 37132 reused + 492 read in 2867 ms (171.6 tok/s), 157 generated in 6279 ms (25.00 tok/s); CPU lane 2.7% of the experts
maya ds4 serve: prompt 38032 tokens = 37780 reused + 252 read in 2061 ms (122.3 tok/s), 213 generated in 7937 ms (26.84 tok/s); CPU lane 2.8% of the experts
maya ds4 serve: prompt 56778 tokens = 38244 reused + 18534 read in 20235 ms (915.9 tok/s), 408 generated in 16739 ms (24.37 tok/s); CPU lane 2.3% of the experts
maya ds4 serve: prompt 58966 tokens = 57185 reused + 1781 read in 5596 ms (318.2 tok/s), 339 generated in 14236 ms (23.81 tok/s); CPU lane 2.4% of the experts
maya ds4 serve: prompt 76480 tokens = 59304 reused + 17176 read in 20228 ms (849.1 tok/s), 196 generated in 8544 ms (22.94 tok/s); CPU lane 2.0% of the experts
Image
Lingua principale
C++
Stelle
11.6k
Fork
1k
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Niko1221/Strata

Tutte le issue di Niko1221/Strata

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.