Please attach which harness was used with every published entry in the data
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 62/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- javascript, react
- Domain
- documentation, testing-qa
Research direction
Start by locating the data source and publishing path for ReactBench's published entries; the issue does not name specific files or tests. Trace how each entry is represented, then add the harness used for every published result and verify that the published data displays it consistently.
Written by the indexing model from the issue text.
Description
I've been playing with Pi a lot recently with multiple agent configurations (which is just to say, presets with model + reasoning effort + subagents all baked in, much like what Amp does). Part of this also means naturally customising the prompts and tools for each agent to maximise each model's performance - and although benchmarks are always not super representative of real-world use, it's super helpful to go look at the harness that does the best job with each model. I was super excited to see ReactBench since I've been writing a lot of React with side-projects recently, and it'd be super helpful to understand which harness eventually gave the scores that you have published. NextJS evals and Artificial Analysis do this too.
- Dominant language
- JavaScript
- Stars
- 271
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from millionco/reactbench
-
astra? Open
Difficulty 5/5 Over a week Newbie friendliness 20/100
millionco/reactbench#5 · 2 comments · 4 reactions ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
millionco/reactbench#4 ·
All issues in millionco/reactbench
Similar issues
-
curation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
amponce/archive-movie-browser#186 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
clerk/javascript#9852 ·
-
bug p1 tools
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
HarperFast/skills#96 ·
-
factory-active factory-automatic task-bug-reproduction-cannot-reproduce task-identify-harness-labels-done task-identify-issue-type-done
Difficulty 2/5 1-3 hours Newbie friendliness 84/100