DeepSeek V4.1 Flash (gb10): draft acceptance 24.5%/32.8% and draft depth locked to 5 — widths 2-5 reported by inspect but not servable

オープン
  • 何か問題がありますか？
#24 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
35/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python
領域
machine-learning

調査の方向性

Start by reading the flare inspect and --graphs output alongside the serving flags and failure message in the issue; the payload does not name source files or tests. Determine whether reported draft widths are intended to be servable and whether a shallower verification prefix is supported. Done means documenting or fixing the mismatch and clarifying the supported controls for draft depth and sampling.

索引モデルが issue の本文から書いたものです。

説明

area:scheduling bug model:deepseek-v4.1 performance priority:P2 triage:investigate

Environment

  • 4× DGX Spark (GB10 / SM121), TP4 over a RoCE ring (--roce-ring)
  • flare-inference 0.1.0 (PPA build, git 20261022044608.27aba893)
  • Package: local-inference-lab/DeepSeek-V4.1-Flash-1017-FLARE, revision gb10, file deepseek-v41-flash-1017-tp4-c16-dspark5.safetensors
  • Serving flags: --max-concurrency 16 --mtp-draft-tokens 5 --mtp-adaptive-verification --embedding-table-storage disk --embedding-table-resident-scales --embedding-table-cache-size 1GiB

What I measured

Single stream, 200 generated tokens, computed from /metrics deltas:

workload steps speculative steps proposes/round accepted acceptance tok/step ms/step tok/s
prose 150 81 (54%) 5 118 24.5% 1.33 27.1 49.1
code 137 67 (49%) 5 132 32.8% 1.46 24.6 59.3

flare_speculative_rounds_total{width=...} is essentially only width="6" (575 rounds at batch=1, 7 rounds at width=1). Widths 2-5 are always zero.

Two more data points:

  • Disabling speculation entirely (--mtp-draft-tokens 0) gives prose 38.8 tok/s and code 39.2 tok/s, so speculation is a +70% / +178% win. The feature works; acceptance is just low.
  • Per-step cost with and without speculation is nearly identical (27.1 vs 26.9 ms/step on prose), so the 6 verification rows are essentially free at batch=1.

The issue

flare inspect reports:

speculation   mtp · widths 1, 2, 3, 4, 5, 6

but --graphs lists packed graphs only at depth 6:

verification16x6   verification  16 6 96
verification8x6    verification   8 6 48
verification4x6    verification   4 6 24
verification2x6    verification   2 6 12
speculative16x1    speculative   16 1 16

So widths 2-5 are reported but do not appear to be servable. Trying to use them fails:

Error: Execution(Manifest(Failure { message: "packed verification requires adaptive execution at the captured draft depth" }))

--no-mtp-adaptive-verification fails the same way. The only working settings are depth 5 with adaptive verification, or depth 0.

Questions

  1. Does the widths 1, 2, 3, 4, 5, 6 line in flare inspect describe usable serving widths, or only the draft depths the proposal head can produce? If the former, this package looks like it is missing the packed graphs for depths 2-5.

  2. Is there a supported way to serve at a shallower effective depth without re-exporting? --mtp-verification-token-budget documents "adaptive prefix selection at the captured draft depth", which reads like it could verify a shorter prefix inside the x6 graph. Is that its intended use, and does it change how many drafts are proposed, or only how many are verified?

  3. Is draft-head sampling tunable at all? For the same prompt, temperature: 0 and temperature: 1 produce byte-identical output (each run twice), and flare inspect reports temperature 1 fixed, top_p 1 fixed. That suggests the sampler is fixed at serve time. If acceptance is sensitive to how sharp the draft distribution is, is there an export-side flag for it?

  4. Is 24.5% (prose) / 32.8% (code) acceptance in the expected range for the gb10 DeepSeek V4.1 Flash package at depth 5, or does it suggest a preparation issue? For reference, the DSpark implementation of the same model in our SGLang setup exposes a draft temperature, block verification and a verification confidence cap. I have not found equivalents on the Flare side, so I cannot tell whether the gap is configuration or inherent to this package.

Happy to run any specific measurement or diagnostic you would like.

主要言語
Rust
スター
800
フォーク
106
平均マージ
1日 3時間
マージ済み PR（30日）
100

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/hyperflare-inference のほかの issue

NVIDIA/hyperflare-inference の issue をすべて見る

似ている issue

Rust の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。