Inference latency benchmark
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 66/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
Research direction
Work in src/complexity.py and add latency_ms(model, input_shape=(1,3,64,64), runs=200), which needs 10 warm-up runs, model.eval() and torch.no_grad(), and should record the CPU/device name. Issue #16 is listed as a dependency, so check whether it is merged and what model object it provides before wiring this up. Done when the function returns a positive float in milliseconds.
Written by the indexing model from the issue text.
Description
Batch-1 CPU latency.
Files: src/complexity.py
Tasks
-
latency_ms(model, input_shape=(1,3,64,64), runs=200)with 10 warm-up runs - Use
model.eval()andtorch.no_grad() - Record the device/CPU name in the output
Done when
Function returns a positive float in milliseconds.
Depends on
#16
Close with a commit or PR message containing Closes #<this issue>.
- Dominant language
- Jupyter Notebook
- Stars
- 0
- Forks
- 0
- Avg merge
- 1m
- Merged PRs (30d)
- 3
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from VisionForgeCSE/CNN-For-Image-Classification
-
evaluation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
evaluation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
training
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Optimizer factoryOpentraining
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
training
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
All issues in VisionForgeCSE/CNN-For-Image-Classification
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
vllm-project/vllm#61131 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
sktime/pytorch-forecasting#2459 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
SheffieldML/GPy#1217 ·
Maintainers usually reply within 1 day
-
model evaluation request not-gguf
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
area:docs
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
jejjohnson/geonnax#89 ·
Maintainers usually reply within 1 day