Question about the COCO-Object result

Open
#2 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python

Research direction

Start by reproducing the COCO-Object evaluation with the released implementation and the paper's settings, using the standard COCO-Object annotations and background-inclusive evaluation described in the issue. Compare the procedure with the reported Table 1 result, focusing on annotation conversion, prompts, dataset-specific parameters, post-processing, and the decoding bucket size; done means identifying and documenting the source of the 4.3 mIoU discrepancy or confirming the exact evaluation command and data-preparation procedure.

Written by the indexing model from the issue text.

Description

Hi, thank you for releasing the code.
I tried to reproduce the ActiveSAM result on COCO-Object using the released implementation and the settings provided in the paper. The results on VOC21 and PC60 are consistent with those reported in the paper, but the COCO-Object result is about 4.3 mIoU lower. I will attach my reproduction result below.
I used the standard COCO-Object annotations converted from COCO-Stuff-164K following the commonly used GroupViT/SCLIP protocol, and the evaluation includes the background class. I also verified that the decoding bucket size does not explain the difference.
Could you please clarify whether the reported COCO-Object result used different annotation files, conversion rules, prompts, dataset-specific parameters, or post-processing steps? If possible, could you also provide the exact evaluation command or data-preparation procedure used to obtain the result in Table 1?
Thank you for your help!

Image
Dominant language
Python
Stars
18
Forks
0
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from VILA-Lab/ActiveSAM

All issues in VILA-Lab/ActiveSAM

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.