Setup testing
@tttyuntian is already working on this.
Since Aug 19, 2021.
Assessment
This issue has not been assessed yet.
Description
#56 set up a basic unit test, but we have to consider what kind of tests we want to run. This is especially important given that GitHub workflows does not have any GPU support, and will thus take a non-trivial amount of time to complete even a basic simple benchmark run. The proposal is to ideate some ways in which we could make tests modular and reasonably fast.
- Dominant language
- Python
- Stars
- 42
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bigscience-workshop/evaluation
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Refactor task template to merge multilingual.json and english.jsonMay be free again @wilsonyhlee claimed this 1855 days ago, and no pull request is open. Open
bigscience-workshop/evaluation#64 · 1 assignee ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
bigscience-workshop/evaluation#61 · 2 comments · 3 reactions ·
-
Start overleaf for benchmark tech reportMay be free again @arunraja-hub claimed this 1866 days ago, and no pull request is open. Opendocumentation
bigscience-workshop/evaluation#54 · 1 comment · 3 reactions · 2 assignees ·
-
translate validation prompts into all training languagesMay be free again @epavlick claimed this 1877 days ago, and no pull request is open. Openmultilingual simple_benchmark
bigscience-workshop/evaluation#50 · 9 comments · 2 assignees ·
All issues in bigscience-workshop/evaluation
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
raullenchai/Rapid-MLX#4042 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
LearningCircuit/local-deep-research#7067 ·
Maintainers usually reply within 1 day
-
#bug
Difficulty 1/5 Under an hour Newbie friendliness 92/100
apache/superset#44923 · 1 comment ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
lawndoc/stack-back#123 ·