Add GEM MLSum to Full Benchmark

Assessment

This issue has not been assessed yet.

Description

Getting set up

First steps

1. Read the whole issue, then the project's contributing guide.
2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
3. Fork the repository and make your change on a branch.
4. Open a pull request that references the issue number.

More from bigscience-workshop/evaluation

• Add MKQA to Full Benchmark

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 4/5 3-5 days Newbie friendliness 25/100

  bigscience-workshop/evaluation#73 · Opened Feb 7, 2022

• Refactor task template to merge multilingual.json and english.json

  May be free again @wilsonyhlee claimed this 1854 days ago, and no pull request is open. Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  bigscience-workshop/evaluation#64 · 1 assignee · Opened Sep 2, 2021

• Wrap evaluation benchmark using HF-trainer

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 5/5 Over a week Newbie friendliness 20/100

  bigscience-workshop/evaluation#61 · 2 comments · 3 reactions · Opened Aug 25, 2021

• Setup testing

  May be free again @tttyuntian claimed this 1870 days ago, and no pull request is open. Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  engineering

  bigscience-workshop/evaluation#57 · 2 comments · 3 assignees · Opened Aug 19, 2021

• Start overleaf for benchmark tech report

  May be free again @arunraja-hub claimed this 1866 days ago, and no pull request is open. Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  documentation

  bigscience-workshop/evaluation#54 · 1 comment · 3 reactions · 2 assignees · Opened Aug 16, 2021

All issues in bigscience-workshop/evaluation

Similar issues

• ADR: R3's household baseline takes the lower of the charger reading and the last set current (narrows ADR-0039)

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  adr

  Difficulty 2/5 1-3 hours Newbie friendliness 72/100

  kristofdegrave/homeassistant-smart-charging#1607 · Opened Oct 2, 2026

  Maintainers usually reply within 1 day

• Claiming namespace sf-local-architect

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  namespace operations

  Difficulty 2/5 1-3 hours Newbie friendliness 64/100

  EclipseFdn/open-vsx.org#13665 · Opened Oct 2, 2026

  Maintainers usually reply within 1 day

• Revise contributing page first impression

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  doc good first issue help wanted

  Difficulty 2/5 1-3 hours Newbie friendliness 68/100

  collective/icalendar#1865 · 2 comments · Opened Oct 2, 2026

  Maintainers usually reply within 1 day

• Build the opentelemetry-collector charm for riscv64

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 2/5 1-3 hours Newbie friendliness 82/100

  canonical/opentelemetry-collector-operator#409 · Opened Oct 2, 2026

  Maintainers usually reply within 1 day

• test_more_dropdown_navigates_correctly_tc_id_c95103[Desktop-1-Add-ons for Firefox Android] - should be removed

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 1/5 Under an hour Newbie friendliness 85/100

  mozilla/addons-release-tests#1243 · Opened Oct 2, 2026

  Maintainers usually reply within 1 day

More Python issues