Add GEM Response Generation to Full Benchmark
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 15/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning, testing-qa
Research direction
No files, tests, or entry points are identified in the issue. Start by locating the full benchmark flow and its existing Schema-Guided Dialog evaluation, including the shuffle challenge set. Done means response generation is included in the complete benchmark and its results are evaluated consistently.
Written by the indexing model from the issue text.
Description
Response generation in Schema-Guided Dialog (including shuffle challenge set)
- Dominant language
- Python
- Stars
- 42
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bigscience-workshop/evaluation
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Refactor task template to merge multilingual.json and english.jsonMay be free again @wilsonyhlee claimed this 1861 days ago, and no pull request is open. Open
bigscience-workshop/evaluation#64 · 1 assignee ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
bigscience-workshop/evaluation#61 · 2 comments · 3 reactions ·
-
Setup testingMay be free again @tttyuntian claimed this 1876 days ago, and no pull request is open. Openengineering
bigscience-workshop/evaluation#57 · 2 comments · 3 assignees ·
-
Start overleaf for benchmark tech reportMay be free again @arunraja-hub claimed this 1872 days ago, and no pull request is open. Opendocumentation
bigscience-workshop/evaluation#54 · 1 comment · 3 reactions · 2 assignees ·
All issues in bigscience-workshop/evaluation
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
rpm-software-management/mock#1824 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
jpata/particleflow#520 ·
Maintainers usually reply within 1 day
-
bug good first issue hacktoberfest
Difficulty 1/5 Under an hour Newbie friendliness 78/100
gridhead/gi-loadouts#699 ·
Maintainers usually reply within 13 days
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
FinanceFlash/unvibecode#206 ·
Maintainers usually reply within 1 day