Request for model trajectories
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- data, machine-learning
Research direction
No files, tests, or entry points are named in the issue. Start by checking the repository's existing trajectory outputs and release documentation; done would mean confirming whether model trajectories are accessible or documenting a release plan.
Written by the indexing model from the issue text.
Description
Hello!
I wanted to ask if there is any way for users to review the trajectories of different models tested on ProgramBench problems, like how we can currently access the Docent links for trajectories on SWE-Bench. If not, are there currently any plans to release these trajectories in the near future? Thank you for your help!
- Dominant language
- Python
- Stars
- 928
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from facebookresearch/ProgramBench
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Difficulty 4/5 3-5 days Newbie friendliness 55/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 4/5 3-5 days Newbie friendliness 58/100
All issues in facebookresearch/ProgramBench
Similar issues
-
bug ci good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
documentation
Difficulty 2/5 Half a day Newbie friendliness 62/100
inmanta/inmanta-core#10835 ·
-
sponsored
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Diaoul/subliminal#1382 ·