train_test_split() silently drops rows when train_size + test_size < 1
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 74/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- data, machine-learning
Research direction
Start with train_test_split() in src/datasets/arrow_dataset.py, especially the size validation around lines 5236–5249 and split slicing around lines 5345–5360. Check the existing tests for train_test_split() and add coverage for sizes whose sum is below 1. Done means the behavior is explicit and tested, so leftover rows are not silently discarded.
Written by the indexing model from the issue text.
Description
Found on main (39fd698d619816099ffa8eb07b5c79b0f4f5f299, datasets 5.2.1.dev0). I reproduced this live against the local checkout.
In src/datasets/arrow_dataset.py:5236-5249, n_train / n_test are computed from the two sizes, and only n_train + n_test > n_samples raises. The < case has no check, no warning, and the leftover rows are never assigned to either split (lines 5345-5360, permutation[n_test:(n_test+n_train)]).
Repro (verified live):
from datasets import Dataset
ds = Dataset.from_dict({"x": list(range(100))})
d = ds.train_test_split(train_size=0.5, test_size=0.2, seed=42)
len(d["train"]), len(d["test"]) # (50, 20) — 30 rows dropped, no warning
train_size=0.5, test_size=0.2 on 100 rows returns train=50, test=20 — 30 rows (30% of the data) silently vanish. A user who supplies both sizes reasonably expects them to partition the dataset (or an error); instead data is lost without a trace.
Why it matters: silent data loss in a data-splitting primitive. The dropped rows are invisible in the returned DatasetDict and can silently shrink training data and skew results.
I searched open issues for train_test_split + drop rows / train_size + test_size sum — no matching open issue (closest is #8425, about empty datasets).
- Dominant language
- Python
- Stars
- 22.2k
- Forks
- 3.7k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 19
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from huggingface/datasets
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
huggingface/datasets#9100 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
huggingface/datasets#9099 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
huggingface/datasets#9097 ·
Maintainers usually reply within 1 day
-
`eval` loader fails when the first log in a folder has no samplesPossibly taken @BahaaMattar claimed this 2 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
huggingface/datasets#9069 ·
Maintainers usually reply within 1 day
-
JSON loader rounds floats to 10 decimals in batches that contain a mixed-type fieldPossibly taken @uvcode01-max claimed this 6 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
huggingface/datasets#9043 ·
Maintainers usually reply within 1 day
All issues in huggingface/datasets
Similar issues
-
[Bug]: reme CLI crashes when the running server used a relative config pathPossibly taken @nurlanmnn claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
agentscope-ai/ReMe#770 ·
Maintainers usually reply within 1 day
-
Pi: paused model calls are reported as "not signed in" when the model's provider comes from an extensionPossibly taken @balcsida claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
microsoft/TokenEconomizer#288 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 67/100
sickn33/agentic-awesome-skills#2437 ·
Maintainers usually reply within 1 day
-
site-bug triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ethereum/consensus-specs#5846 ·
Maintainers usually reply within 1 day