Share benchmark results from your GPU

Open Beginner friendly
  • Is something wrong?
#11 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Documentation
Clarity
Clearly specified
Activity status
Active
Tech stack
python, pytorch
Domain
machine-learning

Research direction

Run the two benchmark commands in the issue on a GPU, then add the resulting JSON files under results/ and list the GPUs in the README. Start by checking the benchmark scripts and README for the expected result format and listing style. Done when results from at least two additional GPUs are included and listed.

Written by the indexing model from the issue text.

Description

area: benchmarks good first issue help wanted

Kernel and model results currently come from one RTX 3090. Numbers from other GPUs show
where the fused INT4 kernel helps and where it does not.

Task: run

python scripts/bench_kernels.py --json kernels.json
bitsmith hf --model Qwen/Qwen2.5-0.5B --device cuda --dtype bfloat16 --eval-tokens 0 --json model.json

and open a pull request adding the files under results/ (for example
results/kernels_rtx4090.json), or post them in a comment here.

Done when at least two more GPUs are in results/ and listed in the README.

Dominant language
Python
Stars
0
Forks
0
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from yashwanthdevavarapu/bitsmith

All issues in yashwanthdevavarapu/bitsmith

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.