Verify evals on Papers with Code
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 48/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- huggingface
- Domain
- documentation
Research direction
Start with the linked Papers with Code paper page and its CALLHOME Part2 and CALLHOME Part2 Two-Speaker evaluation pages. Compare the displayed scores, model name, benchmark protocol, and openness metadata with the paper, then check the repository README if the leaderboard badges are relevant. Done means reporting any discrepancies or confirming that the entries are correct.
Written by the indexing model from the issue text.
Description
Hi,
Niels here from the open-source team at Hugging Face. Congratulations on your work!
I've made the paper and 2 paper-native evaluations available on Papers with Code.
The paper is part of the Speaker Diarization task page.
The TOLD (EEND-OLA + SOAP) results currently rank first on CALLHOME Part2 (diarization) and CALLHOME Part2 Two-Speaker (diarization).
Would it be possible to verify these results and let me know if any score, model name, benchmark protocol, or openness metadata should be corrected?
You can also edit the task, methods, project page, and GitHub URL directly from the paper page using your Hugging Face account.
If you'd like to showcase the results in your repository README, you can copy these live leaderboard badges (or use the “Copy PwC badge” button in the Results section):
Kind regards,
Niels
- Dominant language
- Python
- Stars
- 20.4k
- Forks
- 2k
- Avg merge
- 6h 31m
- Merged PRs (30d)
- 169
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from modelscope/FunASR
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
modelscope/FunASR#3704 · 1 comment ·
-
bug needs feedback
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
modelscope/FunASR#3401 · 2 comments ·
-
bug needs triage
Difficulty 4/5 3-5 days Newbie friendliness 30/100
modelscope/FunASR#3727 ·
-
needs triage question
Difficulty 4/5 3-5 days Newbie friendliness 30/100
modelscope/FunASR#3718 · 3 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
modelscope/FunASR#3715 · 1 comment ·
All issues in modelscope/FunASR
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
xinnan-tech/xiaozhi-fde-talk#263 ·
-
rules
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
huggingface/Repo2RLEnv#163 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 95/100
huggingface/sentence-transformers#4074 ·
-
comp/dashboard invalid P3
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
NousResearch/hermes-agent#121143 ·