How to get train-splits.txt and valid-splits.txt before training tr11-176B-ml
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 28/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Stale
- Domain
- data, devops, machine-learning
Research direction
Start with line 39 of train/tr11-176B-ml/tr11-176B-ml.slurm and trace how TRAIN_DATA_PATH and VALID_DATA_PATH are consumed. Review the repository's documented training and data setup for the source or generation process for train-splits.txt and valid-splits.txt. Done means the steps clearly explain how to obtain both files before running tr11-176B-ml.
Written by the indexing model from the issue text.
Description
- Big Science version: latest
- Python version: 3.8.8
- Operating System: Ubuntu 20.04.5 LTS
Description
How to get train-splits.txt and valid-splits.txt at Line39 in train/tr11-176B-ml/tr11-176B-ml.slurm. Thx.
TRAIN_DATA_PATH=$MEGATRON_DEEPSPEED_REPO/data/train-splits.txt
VALID_DATA_PATH=$MEGATRON_DEEPSPEED_REPO/data/valid-splits.txt
- Dominant language
- Shell
- Stars
- 1k
- Forks
- 102
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bigscience-workshop/bigscience
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 1/5 Under an hour Newbie friendliness 25/100
-
eval opt-175B Open
Difficulty 4/5 3-5 days Newbie friendliness 20/100
bigscience-workshop/bigscience#63 · 1 comment ·
All issues in bigscience-workshop/bigscience
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
danielmiessler/LifeOS#2218 ·
-
docs(agents): strengthen the no-backslash-escaped-backticks rule with an issue-creation example Open
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
package-update
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
oSoWoSo/vOid_Community_repOsitory#148 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
chore
Difficulty 1/5 Under an hour Newbie friendliness 91/100
alunduil/alunduil-chezmoi#792 ·