bigscience-workshop/data-preparation

View on GitHub

Code used for sourcing and cleaning the BigScience ROOTS corpus

Stars: 318
Forks: 42
Open beginner issues: 0
Indexed issues: 10

Dominant language: Jupyter Notebook
License: Apache-2.0
Last GitHub push: Feb 20, 2023
Latest indexed: Sep 14, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

10 open issues indexed

Question about ROOTS corpus: availability & earlier web data Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

bigscience-workshop/data-preparation#45 · Opened Sep 24, 2025
Welcome to try SailCraft - A data cleaning tool built upon this repository Open

Difficulty 5/5 Over a week Newbie friendliness 10/100

bigscience-workshop/data-preparation#44 · Opened Apr 30, 2024
Why stopwords_min_cutoff rather than stopwords_max_cutoff? Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

bigscience-workshop/data-preparation#43 · Opened Apr 18, 2024
Changing parmater values to extreme in parameters_filtering.py doesn't change the no. f documents being removed Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

bigscience-workshop/data-preparation#42 · Opened Feb 14, 2024
author = "bigscience-catalogue-lm-data", there is no this data in Huggingface. Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

bigscience-workshop/data-preparation#41 · Opened Feb 5, 2024
Can't find the Deduplication Report Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

bigscience-workshop/data-preparation#40 · Opened Nov 9, 2023
Extending this codebase Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

bigscience-workshop/data-preparation#39 · 1 reaction · Opened Jun 18, 2023
the version of simhash Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

bigscience-workshop/data-preparation#38 · Opened Jun 9, 2023
Mismatch of the Available Data Quantity on Huggingface Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

bigscience-workshop/data-preparation#37 · Opened May 2, 2023
Write READMEs Open

bigscience-workshop/data-preparation#6 · 1 assignee · Opened Apr 26, 2022

bigscience-workshop/data-preparation

Get new issues in your inbox