Qwen3.8 Flash Next DFlash2: mixed prefill rounds, YaRN serving, and served measurements from a fork
メンテナーはふだん 1 日以内に返信
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 20/100
調査の方向性
Start with qwen38/bootstrap.py, components/dflash2.py, and the mixed-round graph signature and binding paths referenced in the issue. Reproduce the reported export failure with the CPU tests, then inspect the fork's mixed speculative package changes and served measurements. Done means the export defect is fixed and maintainers have decided whether mixed DFlash2 rounds and YaRN serving fit the existing contracts.
索引モデルが issue の本文から書いたものです。
説明
d68ed8f5 added shared DFlash2 preparation for Qwen3.8 Flash Next on components/dflash2.py, selected with --speculative-method dflash2. A fork (Raydius/flare-local-inference, branch wt/dflash-s-e2vk) has served Flash Next with the incoai/Qwen3.8-Flash-Next-DFlash2 draft on an RTX PRO 6000 (SM120, TP1) since 2026-10-31 and, after d68ed8f5, the fork's own recipe is redundant except for three things. Before opening PRs for any of them I would like your decision on each, since AGENTS.md asks for an explicit maintainer request and for prior approval of any new captured-graph operation contract.
- Mixed prefill rounds with a DFlash2 draw. Upstream's
qwen38/bootstrap.pyrejects the combination ("DFlash2 requires separate prefill and verification graphs"). The fork prepares mixed prefill+decode rounds whose embedded decode round draws from DFlash2: the same mixed graph family (one prompt row plusbdecode rows at widthw, selected by the manifest's mixed depth): the decode half composes the shared proposal-only DFlash2 round around the packed target, and the prompt half, built the way the MTP absorption is, writes the chunk's captured features into its request's context rings after the round. Exported and served under--mixed-prefill --max-concurrency 8with prompt-cache reuse; no new graph kind, input binding, or Helios policy. Question: is a mixed round that embeds a DFlash2 draw a shape variant of the existing mixed family (an operation-contract-preserving change, PR-able as a recipe change), or a new operation contract that needs your approval first? - YaRN serving for the 1M-token Flash Next context. The fork carries two small patches: a
qwen3.8-flash-yarn4checkpoint declaration (context=1_048_576,RopeScaling(4., 262144), the same weights) and--rope-parameters(frequency-only interpolation without YaRN's attention temperature, applied to any resolved checkpoint and recorded in the plan). With DFlash2 the draft's rotary and its 2048-position rings take the extended positions unscaled; a probe past 262k tokens kept 3.69 accepted tokens per round (acceptance rate 0.618) against 2.11 / 0.699 below it. Question: do you want that as a PR (declaration first, then the flag), or should long context stay out of the Flash Next recipe? - Served measurements, in case they help set the DFlash2 defaults. All from
flare runof exported packages (never debug-serve), 28-case frozen code-edit workload, single trials with about ±8 % repeatability, history copies off:
| arm (serve-time width on an adaptive package) | c1 tok/s | c1 accepted/round | c8 tok/s | c8 accepted/round |
|---|---|---|---|---|
| adaptive | 405.5 | 1.85 | 533.2 | 3.94 |
| fixed-0 (no speculation) | 153.8 | – | 424.5 | – |
| fixed-3 | 360.6 | 1.74 | 459.6 | 3.42 |
| fixed-7 | 537.3 | 2.44 | 351.7 | 3.87 |
A package exported fixed-7 (--no-mtp-adaptive-verification, only fixed-width rounds captured) served 565.5 tok/s at c1 and 417.0 at c8 (acceptance rate 0.587 / 0.606). Single-stream throughput rises with every extra draft up to 7; at eight streams the fixed curve peaks at 3 and the adaptive controller beats every fixed width.
Independently of the above, a defect PR is coming first: since 317c1ecc and bce032c0 no package with mixed rounds that also speculate can be exported (the mixed decoder is built in verification mode, which now aliases sample rows to hidden rows, and GraphSignature forbids draft_tokens on mixed kinds while the binding requires it); upstream's own Qwen3.8 MTP mixed export is affected. Two commits, CPU tests failing on the parent, a mixed speculative package exported and served on SM120.
- 主要言語
- Rust
- スター
- 800
- フォーク
- 106
- 平均マージ
- 1日 3時間
- マージ済み PR（30日）
- 100
環境構築
- Dockerfile または Docker Compose ファイルあり
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
NVIDIA/hyperflare-inference のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
NVIDIA/hyperflare-inference#268 ·
メンテナーはふだん 1 日以内に返信
-
area:caching area:engram enhancement performance triage:design-review
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
NVIDIA/hyperflare-inference#230 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 5/5 1週間以上 初心者へのやさしさ 15/100
NVIDIA/hyperflare-inference#278 ·
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 25/100
NVIDIA/hyperflare-inference#276 ·
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 35/100
NVIDIA/hyperflare-inference#271 ·
メンテナーはふだん 1 日以内に返信
NVIDIA/hyperflare-inference の issue をすべて見る
似ている issue
-
bug UI
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
wardian-app/Wardian#1746 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
jlcodes99/cockpit-tools#3125 ·
メンテナーはふだん 2 日以内に返信
-
bug user-sprint
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
Vondor/KassenSturz#967 ·
メンテナーはふだん 1 日以内に返信
-
area/ci
難易度 1/5 1時間未満 初心者へのやさしさ 62/100
メンテナーはふだん 1 日以内に返信
-
[BUG] Incremental sync warns "no stored highest_uid" on every round for folders with no new mailオープンbug
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
rustmailer/bichon#411 ·