[Bug]: MeanAverageRecall returns 0.0 instead of -1 when there are no targets (whole run or empty size bucket)

Open
  • Is something wrong?
#2,964 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@MaiAditya is already working on this.

Since Nov 12, 2026.

  • #2965 by @MaiAditya — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
20/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
python
Domain
analytics

Research direction

The bug is in mean_average_recall.py, in the no-target paths that return 0.0 where the sentinel should be -1. Compare them with the -1 handling in MeanAveragePrecisionResult in the mean average precision module, which the issue names as the reference. A fix is already open as pull request #2965, so check that before starting. Done means an empty run and an empty size bucket both return -1 for mAR, matching pycocotools.

Written by the indexing model from the issue text.

Description

Search before asking
  • I have searched the issues and discussions and found no similar bug report.
Bug

sv.metrics.MeanAverageRecall returns 0.0 when there are no targets to recall: for a whole run without targets, and for an empty small / medium / large size bucket. sv.metrics.MeanAveragePrecision and pycocotools return -1 in both cases, so with mAR an empty bucket looks like a model that found none of its objects.

sv mAP small: -1 | sv mAR@100 small: 0.0
sv mAP medium: -1 | sv mAR@100 medium: 0.0
pycocotools AR@100 small: -1.0 medium: -1.0
no targets at all -> sv mAP: -1 | sv mAR@100: 0.0

Comparing mAR with pycocotools on 40 random inputs, 9 differed, all of them on an empty bucket; mAP and non-empty mAR matched.

Environment

supervision 0.31.7 (same code on develop 215e06ae), pycocotools 2.0.11, Python 3.12.7, macOS.

Minimal Reproducible Example
import numpy as np
import supervision as sv
from supervision.metrics import MeanAveragePrecision, MeanAverageRecall

gt = sv.Detections(xyxy=np.array([[0, 0, 200, 200]], float), class_id=np.array([0]))  # one large target
pred = sv.Detections(xyxy=np.array([[0, 0, 200, 200]], float), class_id=np.array([0]), confidence=np.array([0.9]))
mp = MeanAveragePrecision().update(pred, gt).compute()
mr = MeanAverageRecall().update(pred, gt).compute()
print("sv mAP small:", mp.small_objects.map50_95, "| sv mAR@100 small:", mr.small_objects.mAR_at_100)
print("no targets:", MeanAverageRecall().update(pred, sv.Detections.empty()).compute().mAR_at_100)
Are you willing to submit a PR?
  • Yes I'd like to help by submitting a PR!

I have a fix with tests ready (return the -1 sentinel in the no-target paths of mean_average_recall.py, as MeanAveragePrecisionResult does) and will open a PR linked to this issue.

Dominant language
Python
Stars
51.6k
Forks
5k
Avg merge
1d 17h
Merged PRs (30d)
191

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from roboflow/supervision

All issues in roboflow/supervision

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.