Support more Coding Benchmark?
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Domain
- testing-qa
Research direction
No files, tests, or entry points are named. Start by locating the existing SWE-bench support and its benchmark interface, then determine what would be required for an additional benchmark such as Tau-bench; the issue is complete only when the supported benchmark scope and implementation requirements are defined.
Written by the indexing model from the issue text.
Description
Thanks for your great work! Do you currently support, or plan to support, additional coding benchmarks beyond SWE-bench, such as Tau-bench?
- Dominant language
- Python
- Stars
- 857
- Forks
- 96
- Avg merge
- 6h 52m
- Merged PRs (30d)
- 1
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA-NeMo/ProRL-Agent-Server
-
opencode harness captures zero completions — config doesn't include gateway baseURLMay be free again @billxbf claimed this 133 days ago, and no pull request is open. Openbug
NVIDIA-NeMo/ProRL-Agent-Server#39 · 1 comment · 1 assignee ·
All issues in NVIDIA-NeMo/ProRL-Agent-Server
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
UKGovernmentBEIS/control-arena#917 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
debpalash/VoiceStudio#2937 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
aws-samples/sample-per-model-bedrock#261 ·
Maintainers usually reply within 1 day