Add GEM E2E to Full Benchmark
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
No files or tests are named. Start by locating the full-benchmark entry point and existing dataset integrations, then determine how the GEM E2E task and its shuffle challenge set are represented. Done means both GEM E2E evaluation variants are included in the full benchmark and covered by the repository's relevant checks.
Written by the indexing model from the issue text.
Description
E2E NLG (+shuffle challenge set)
- Dominant language
- Python
- Stars
- 42
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bigscience-workshop/evaluation
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Refactor task template to merge multilingual.json and english.jsonMay be free again @wilsonyhlee claimed this 1855 days ago, and no pull request is open. Open
bigscience-workshop/evaluation#64 · 1 assignee ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
bigscience-workshop/evaluation#61 · 2 comments · 3 reactions ·
-
Setup testingMay be free again @tttyuntian claimed this 1870 days ago, and no pull request is open. Openengineering
bigscience-workshop/evaluation#57 · 2 comments · 3 assignees ·
-
Start overleaf for benchmark tech reportMay be free again @arunraja-hub claimed this 1866 days ago, and no pull request is open. Opendocumentation
bigscience-workshop/evaluation#54 · 1 comment · 3 reactions · 2 assignees ·
All issues in bigscience-workshop/evaluation
Similar issues
-
New InternshipOpennew_internship
Difficulty 1/5 Under an hour Newbie friendliness 70/100
-
[BUG] Reports tab: "Unban" button tooltip shows raw `{{ip}}` placeholder instead of the IP addressOpenbug javascript ui
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
bunkerity/bunkerweb#4001 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 92/100
PedestrianDynamics/pyFDS-Evac#476 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
google/differential-privacy#516 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
adobe-fonts/source-serif#153 ·