mC4 sampling & pre-processing
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 35/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Stale
- Domain
- documentation
Research direction
Start in the data/mc4 documentation linked by the issue and compare the two mC4 storage links with their intended destinations. Verify that the corrected locations are still valid and clarify or update the preprocessing references; done means the documentation links work and the scope around the related processing code is decided.
Written by the indexing model from the issue text.
Description
Hi @TevenLeScao,
I think there are some confusing and broken link in the mC4 data preprocessing section. Can you take a look?
Both of the links are broken here,
The original link should be,
In addition to that, the multinomial data processing code to create the different language splits are in this pull request, https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/9
Here's few things,
- Did you use this data for any one of your experiments?
- If not then I think you can update the doc, https://github.com/bigscience-workshop/bigscience/tree/master/data/mc4
For reference purpose, if you want to keep the code, I'm happy to open a pull request here. If not I'll close the pull request from bigscience/Megatron-Deepspeed repo.
Let me know what do you think.
- Dominant language
- Shell
- Stars
- 1k
- Forks
- 102
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from bigscience-workshop/bigscience
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 1/5 Under an hour Newbie friendliness 25/100
-
Difficulty 3/5 1-2 days Newbie friendliness 28/100
bigscience-workshop/bigscience#66 · 1 comment ·
All issues in bigscience-workshop/bigscience
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
community-scripts/ProxmoxVE#17425 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
danielmiessler/LifeOS#2218 ·
-
docs(agents): strengthen the no-backslash-escaped-backticks rule with an issue-creation example Open
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
technical-debt
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
ll7/robot_sf_ll7#9560 ·
-
package-update
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
oSoWoSo/vOid_Community_repOsitory#148 · 1 comment ·