[Bug]: Soft-NMS decays a detection by a duplicate that was already decayed below it

Open
  • Is something wrong?
#2,950 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
numpy, python
Domain
computer-vision, machine-learning

Research direction

Read _soft_nms_decay_from_iou_matrix and _nms_loop_from_iou_matrix, then inspect tests/detection/utils/test_iou_and_nms.py and tests/detection/test_core.py. The issue reports a proposed implementation and tests through all four entry points, but asks maintainers to decide between matching the paper and preserving current behavior, so first resolve that design question. Done means the chosen behavior and its performance tradeoff are documented and covered by tests.

Written by the indexing model from the issue text.

Description

Search before asking
  • I have searched the Supervision issues and found no similar bug report. Searched issues and PRs for "soft nms", "with_soft_nms" and "soft_non_max_suppression"; only #1624 and #2790 came up.
Bug

sv.Detections.with_soft_nms, sv.box_soft_non_max_suppression and sv.mask_soft_non_max_suppression decay scores in the order of the input scores. Soft-NMS (Bodla et al., ICCV 2017, Figure 2) selects m ← argmax S in every iteration, on scores that earlier iterations have already decayed. The two differ as soon as a decay moves a detection below one that came after it: a duplicate that was already decayed still lowers the score of its neighbour.

Expected, following Figure 2 for A [0, 0, 10, 10] 0.90, B [0, 4, 10, 14] 0.80 and C [0, 10, 10, 20] 0.75 with sigma=0.5 (IoU(A, B) = 60/140, IoU(B, C) = 40/160, IoU(A, C) = 0):

  1. Pick A. B becomes 0.80 * exp(-(60/140)**2 / 0.5) = 0.554055. C keeps 0.75.
  2. Pick C, because 0.75 > 0.554055. B becomes 0.554055 * exp(-(40/160)**2 / 0.5) = 0.488952.
  3. Pick B. Result [0.9, 0.488952, 0.75].

Actual on develop (5b57e10): [0.9, 0.554055, 0.661873]. C is decayed by B although B is already below it, so score_threshold=0.7 drops C, a separate object that hard NMS (with_nms(threshold=0.2)) keeps.

The gap grows with the number of duplicates, which is the input Soft-NMS is meant for. One object with 8 exact duplicates and a neighbour at IoU 1/3 scored 0.78: the paper's loop gives 0.78 * exp(-(1/3)**2 / 0.5) = 0.6246, develop gives 0.1056, because each duplicate decays the neighbour again. On 2,000 random scenes of 2 to 8 boxes, 334 differ from the paper's loop by more than 1e-6, by up to 0.25.

The authors' reference code (cpu_soft_nms) and mmcv's softnms_cpu both search the remaining boxes for the highest current score before each decay step.

A question before any PR. This looks like a deliberate choice: the docstring of _soft_nms_decay_from_iou_matrix describes the current form as the single-pass, no re-sort loop computed as one vectorized closed form, and AGENTS.md asks for no Python loops in hot paths. Selecting by current score cannot be written as one sum over a fixed order, so matching the paper means a loop like _nms_loop_from_iou_matrix (one pass per detection, NumPy work inside). In a local version of that change, box_soft_non_max_suppression including the IoU matrix goes from 0.09 to 0.44 ms at 97 boxes, 7.3 to 15.7 ms at 1,000 and 49 to 72 ms at 3,000, and no existing expected value in tests/detection/utils/test_iou_and_nms.py or tests/detection/test_core.py changes.

Which would you prefer: (a) match the paper, with that cost, or (b) keep the current behaviour and state in the docstrings that it differs from the paper when a decay reorders detections? I have (a) ready with tests through all four entry points and can send either.

Environment
  • Supervision: develop at 5b57e10
  • Python: 3.13
  • OS: macOS (arm64)
Minimal Reproducible Example
import numpy as np
import supervision as sv

detections = sv.Detections(
    xyxy=np.array([[0, 0, 10, 10], [0, 4, 10, 14], [0, 10, 10, 20]], dtype=np.float32),
    confidence=np.array([0.90, 0.80, 0.75]),
    class_id=np.array([0, 0, 0]),
)
print(np.round(detections.with_soft_nms(sigma=0.5).confidence, 6).tolist())
print(len(detections.with_soft_nms(sigma=0.5, score_threshold=0.7)))
# develop: [0.9, 0.554055, 0.661873] and 1
# paper:   [0.9, 0.488952, 0.75]     and 2
Additional

No response

Are you willing to submit a PR?
  • Yes I'd like to help by submitting a PR!
Dominant language
Python
Stars
51.6k
Forks
5k
Avg merge
1d 17h
Merged PRs (30d)
191

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from roboflow/supervision

All issues in roboflow/supervision

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.