MM validation wrappers truncate ground truth when batch_size exceeds one
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- computer-vision, machine-learning
Research direction
Start with the MMDetDataset.len and MMSegDataset.len validation implementations mentioned in the issue. Reproduce the 8-sample, batch_size=2, one-GPU case and verify that the wrapper length matches the full underlying dataset length so all validation predictions and annotations remain aligned.
Written by the indexing model from the issue text.
Description
Summary
Both MMDetDataset.len and MMSegDataset.len return floor(dataset_length / (batch_size * num_gpus)) * num_gpus in validation mode. These wrappers count samples, but the formula uses the number of batches per GPU and omits multiplication by batch_size. For an 8-sample dataset with batch_size=2 and one GPU, len() returns 4. The validation DataLoader wraps the underlying dataset and still returns all samples when drop_last=False, while the evaluator uses the MMDet/MMSeg wrapper length to load annotations or ground-truth masks. The remaining predictions are therefore omitted or misaligned during evaluation.
Expected behavior
Validation wrapper length should match the number of examples whose predictions are evaluated. Since drop_last is configurable separately and not passed into these wrappers, their sample count should remain the full underlying dataset length.
- Dominant language
- C++
- Stars
- 9.2k
- Forks
- 725
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from activeloopai/deeplake
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
activeloopai/deeplake#3260 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
activeloopai/deeplake#3259 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
activeloopai/deeplake#3256 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
activeloopai/deeplake#3254 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
activeloopai/deeplake#3247 · 1 comment ·
Maintainers usually reply within 1 day
All issues in activeloopai/deeplake
Similar issues
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
iOS: hidden scale bar invalidates its intrinsic content size on every layout pass of MLNMapViewOpen
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
maplibre/maplibre-native#4723 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
HarbourMasters/Shipwright#7320 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Make Catch2 optional when `RDK_BUILD_CPP_TESTS=OFF`Possibly taken @pechersky claimed this today. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 2 days