docs(bench): speculative_bench module docs contradict its own code in three places

Open Beginner friendly
#1,691 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
92/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
rust
Domain
documentation

Research direction

Start in src/bin/speculative_bench.rs by comparing the module docs and long_about with REACHABLE_PAIRINGS and the MTP, DFlash, and batch-sweep descriptions. Update the three contradictory passages so they match the code and pairing table; no run is required.

Written by the indexing model from the issue text.

Description

priority:low status:ready type:docs

Summary

Correct three passages in speculative_bench's module docs and long_about that contradict the code in the same file: a stale target count, a batch-sweep claim the code disowns, and a "deferred" claim about a path that already records real numbers.

Background

The doc comments were written against an earlier version of the file and drifted as the pairing table and the MTP path grew. A reader trusting the header comes away with the wrong set of reachable targets, the wrong batch behavior, and the wrong idea that MTP is not yet measured.

Proposed Solution

Edit the three passages so the module docs, long_about, and the pairing table describe what the code does. No run required.

Implementation Notes

  • src/bin/speculative_bench.rs:21 claims the two reachable targets models/qwen3.5-4b-4bit and models/gemma-4-31b-it-4bit; REACHABLE_PAIRINGS actually lists four targets, adding gemma-4-12b-it-4bit (:279) and qwen3.8-27b-4bit (:297).
  • :61 says "Full sweep across pairings (B = 1, 2, 4)", but the sweep logs benching at B=1 and --batch's own doc notes the baseline path always runs B=1 today.
  • :118-121 long_about says speculative paths are "scaffolded but deferred to follow-up", which contradicts the module body: MTP records real decode tok/s; only DFlash is deferred.
  • Distinct from #1667, the --prompt panic in the same binary.

Acceptance Criteria

  • Module docs, long_about, and the pairing table tell one consistent story matching the code.

Original Suggestion

Title: docs(bench): speculative_bench module docs contradict its own code in three places

speculative_bench's module docs and long_about contradict the code in the same file in three places — stale target counts, a batch-sweep claim the code disowns, and a "deferred" claim about a path that ships.

Evidence

  • src/bin/speculative_bench.rs:21 — claims "the two reachable target models (models/qwen3.5-4b-4bit, models/gemma-4-31b-it-4bit)"; REACHABLE_PAIRINGS (:243-308) lists four targets, adding gemma-4-12b-it-4bit (:281) and qwen3.8-27b-4bit (:298)
  • :61 — "Full sweep across pairings (B = 1, 2, 4)"; the sweep logs "benching {} pairings at B=1" (:948-950) and --batch's own doc says "The baseline path always runs B=1 today" (:139-141)
  • :118-121long_about says "Speculative paths are scaffolded but deferred to follow-up", contradicting :26-33 and :94-99 (MTP records real decode tok/s; only DFlash at :100-103 is deferred)

Suggested fix

Update the three passages against the code in the same file. No run required.

(Distinct from #1667, the --prompt panic in this binary.)

Acceptance criteria

  • Module docs, long_about, and the pairing table tell one consistent story
Dominant language
Rust
Stars
470
Forks
54
Avg merge
4h 45m
Merged PRs (30d)
305

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from lablup/mlxcel

All issues in lablup/mlxcel

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.