Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Speculative drafts (MTP / DFlash / DSpark / DFlash2): current status and what to strengthen (post-0.3.3)

Open
  • Is something wrong?
#228 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
15/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
rust
Domain
ai, machine-learning

Research direction

Start with gap 1: the draft gates in justfile:1072 and gate_qwen35_dspark.rs, which already compare greedy ids with and without a draft. Read the draft types in crates/models/src/lib.rs:672 and the shared MTP window in crates/app/src/mtp.rs. Done means each drafted gate prints ids identical N and accept_mean=. Gaps 2 to 5 depend on #88 and are not suited to a first contribution.

Written by the indexing model from the issue text.

Description

Summary

Every draft we run today goes through one runtime type, DraftSpec (crates/models/src/lib.rs:672), which has two variants: Block (a separate draft file, general.architecture "dflash", crates/model/src/arch/mod.rs:63) and Mtp (a next-token layer that ships inside the model file). MTP generates on Qwen3.8 and GLM-5.3 and is on by default there (README.md:65). Block drafts (DSpark) generate on V4.1 and have a gate for Qwen3.6. Nothing reads DFlash2. This issue lists what exists today and the gaps worth closing after 0.3.3, before more drafters show up (DFlash2 for Qwen3.8 27B, and possibly Nemotron 3.5 Bolt).

Current status (origin/main eccb1e69)

MTP (NextN) DFlash / DSpark (block draft) DFlash2
Reader DraftSpec::Mtp / MtpDraft (crates/models/src/lib.rs:672-700). Per-family readers: qwen35moe (crates/model/src/arch/qwen35moe/mtp.rs:1058), mimo2 (crates/model/src/arch/mimo2/mtp.rs:553), GLM NextN (NextnInputs, crates/app/src/arch/glm5next/mod.rs:150) DraftSpec::Block / BlockDraft. V4.1: crates/model/src/arch/deepseek41/spec.rs:352. Qwen3.6 RedHatAI speculator (HF config + safetensors) through the dense draft layers (crates/gpu/src/dspark/dense.rs:1) none (no match for dflash2/DFlash2 in the tree)
Kernels qwen3moe draft walk (crates/gpu/src/arch/qwen3moe/program38.rs:99); GLM nextn_walk/nextn_chain (crates/app/src/arch/glm5next/mtp.rs:87,106) shared DsparkBody/DraftLayers (crates/app/src/dspark.rs:35); V4.1 dflash_router (crates/gpu-deepseek41/src/experts_mxfp4.rs:63-97), draft expert kernels (gate_dspark_experts.rs:1608) none: no grouped dynamic depthwise conv, no top-K path selector
Generation path shared MTP window MtpDraft (crates/app/src/mtp.rs:1,388) plus host policy (crates/app/src/mtp/policy.rs:1); MtpBody for qwen3moe (crates/app/src/arch/qwen3moe/mtp.rs) and GLM (crates/app/src/arch/glm5next/mtp.rs:1) CardDspark: the draft runs on a card of its own (crates/app/src/dspark.rs:1,50); V4.1 Ds41Draft (crates/app/src/arch/deepseek41/draft.rs:24); the runtime maps it to Program::Block (crates/runtime/src/speculative.rs:670) none
Models Qwen3.8, GLM-5.3: on by default, width follows measured cost (README.md:65). Off on paged plans (README.md:190; #71) and after a kept prefix (README.md:81). MiMo sidecar MTP not attached (#74). qwen35moe files with a NextN layer: tensors read as Role::Unused, no draft (crates/model/src/arch/qwen35moe/hparams.rs:1508; #30). Qwen3.8 27B dense: per #88 the next-token tensors are unused (not verified on main, #88 is not merged) V4.1 via BLOOMERY_DRAFT=dspark (gate_ds41_serve.rs:2367). Qwen3.6 DSpark: gate only (justfile:961). Whether Qwen3.6 serving uses it by default: unknown (README.md:47 has a "DSpark off" row). MiniCPM-V / Qwen3.6 DSpark tracked in #90 none
Gates gate-gpu-qwen4exp-mtp (justfile:1072; known red in 0.3.2, README.md:76); bloomery_serve_qwen38 draft refusal checks (crates/gpu-gates/src/bin/bloomery_serve_qwen38.rs:84,112); glm5next serve plain vs draft (gate_glm5next_serve.rs:30) gate-gpu-qwen35-dspark: layers vs host f64, plus 256 greedy ids equal to the run with no draft (gate_qwen35_dspark.rs:1-20); gate-gpu-ds41-draft (justfile:1795); ds41-dspark-loop (justfile:1816); dspark-hc / dspark-experts (justfile:2230,2235) none
Measured numbers throughput with MTP on, e.g. Spark 41.6 tok/s (README.md:203). Acceptance length: not in docs (unknown). GLM on/off measurement still open (#76) throughput row with DSpark off only (README.md:47). Acceptance length: not in docs (unknown) external only: Inco reports a mean acceptance length on Qwen3.8-27B of MTP 4.54 / DFlash 4.92 / DSpark 5.49 / DFlash2 5.97

External reference: DFlash is a one-pass block diffusion drafter conditioned on the target's hidden states (arXiv 2602.06036). DFlash2 adds a grouped dynamic depthwise conv and a low-rank top-K path selector. Its config arch is DFlash2DraftModel, with block_size, target_layer_ids, mask_token_id, conv_kernel_size, conv_group_size, selector_rank and selector_top_k. A public drafter exists: z-lab/Qwen3.8-27B-DFlash2.

Gaps to strengthen (in priority order)

  1. Acceptance metrics and a common draft gate contract. Every draft gate should check that the output is lossless: N greedy ids equal to the run with no draft, as gate_qwen35_dspark already does. It should also record the mean acceptance length and the draft's share of step time.
    Done when each drafted gate (qwen4exp-mtp, glm5next serve, ds41-draft, qwen35-dspark) prints ids identical N and accept_mean= lines. A docs table then carries acceptance per model and draft. This absorbs #76.
  2. DFlash2 for Qwen3.8 27B dense, after #88 lands. Read DFlash2DraftModel configs as a new BlockDraft shape, or as a third DraftSpec variant. Add the grouped dynamic depthwise conv and the low-rank top-K selector over the target head's candidates. Reuse DsparkBody/DraftLayers for the rest.
    Done when a gate holds the conv and selector against a host f64 reference, 256 greedy ids match the run with no draft, and acceptance on Qwen3.8-27B is measured next to the external 5.97.
    Open question: does a dense target accept a block draft today? #88 says dense drafts are refused there. This is unverified on main.
  3. MTP where the tensors already ship. Covers qwen35moe NextN (#30, Role::Unused at hparams.rs:1508), MiMo sidecar (#74), and the Qwen3.8 27B next-token tensors (#88).
    Done when each one is attached through the shared MtpBody, passes a lossless greedy check, and has acceptance recorded.
  4. ctx/VRAM accounting for drafts. A draft's bytes compete with ctx and slots. Examples: the qwen4exp-mtp stage card loses about 2.84 GB (README.md:76); paged plans turn MTP off (#71); CardDspark reports resident and taken bytes separately (crates/app/src/dspark.rs:65-67).
    Done when the ctx-first planner (#155) accounts for the draft as one term, and the load line states plainly when a default draft is dropped and why.
  5. Nemotron 3.5 Bolt 74B-A9B check (released 10/15). Whether it ships MTP or a draft is unknown.
    Done when config and tensor names are inventoried with gguf-inventory's mtp_nextn group (crates/gguf/src/bin/gguf-inventory.rs:105), and a note says which of the paths above it would use. No weights are downloaded until disk space allows.

Out of scope / ordering

  • None of this is 0.3.3 scope. 0.3.3 ships N7 (#156), M2a and the open fixes first. This work starts after the 0.3.3 tag.
  • Not here: new drafter training, and lookup/n-gram drafts (#32 covers verify cost).
  • Related: #30, #32, #71, #74, #76, #88, #90, #155, #156.

(Issue written by the bloomery AI assistant.)

Dominant language
Rust
Stars
21
Forks
4
Avg merge
3h 27m
Merged PRs (30d)
194

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from midagedev/bloomery

All issues in midagedev/bloomery

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.