EvalScope integration: one-command evaluation support for PerceptionBench
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- documentation
Research direction
Start by checking the README and existing documentation to find the appropriate location for an evaluation workflow link. Add a concise EvalScope section with the supplied command and guide URL, then verify that the example and benchmark-specific setup notes are accurate.
Written by the indexing model from the issue text.
Description
Hi MoonshotAI/PerceptionBench maintainers,
EvalScope now supports PerceptionBench. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.
This may be useful for users who find this repository while setting up PerceptionBench and want a quick path to evaluate a hosted or locally served model.
Example:
evalscope eval \
--model YOUR_MODEL \
--api-url OPENAI_API_COMPAT_URL \
--api-key EMPTY_TOKEN \
--datasets perception_bench \
--limit 10
Docs: https://evalscope.readthedocs.io/en/latest/benchmarks/perception_bench.html
EvalScope repo: https://github.com/modelscope/evalscope
Some benchmarks may require additional assets, media files, or judge-model configuration for a full run; the EvalScope guide above includes the benchmark-specific setup notes.
If this is useful for your community, we can follow up with a small README/Docs PR linking to the EvalScope usage guide.
- Dominant language
- Python
- Stars
- 208
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from MoonshotAI/PerceptionBench
-
Test set Open
Difficulty 5/5 Over a week Newbie friendliness 35/100
MoonshotAI/PerceptionBench#2 · 1 reaction ·
-
PBench & VisRes Open
Difficulty 4/5 3-5 days Newbie friendliness 35/100
All issues in MoonshotAI/PerceptionBench
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100