facebookresearch/maskrcnn-benchmark

Inference time increases linearly with batch size

Open

#294 opened on Dec 20, 2018

View on GitHub
 (8 comments) (2 reactions) (0 assignees)Python (9,161 stars) (2,574 forks)batch import
help wanted

Description

❓ Questions and Help

I tested the batch inference，this is result batch size time 1 0.22 2 0.42 4 0.84 6 1.2 8 crash OOM Why big batchsize doesn't save time?

Contributor guide

Tech stack
pythonpytorch
Domain
machine learning
Issue type
performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Basic PyTorch knowledgeUnderstanding of batch inference
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
Investigate why batch size does not reduce per sample inference time. Look at the batch processing code in maskrcnn benchmark/modeling/detector and check for memory bottlenecks or sequential processing. Consider GPU utilization and potential OOM issues. Review comments in the issue for additional context.