train_test_split() silently drops rows when train_size + test_size < 1

Open Beginner friendly
#9,098 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
74/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
numpy, pandas, python

Research direction

Start with train_test_split() in src/datasets/arrow_dataset.py, especially the size validation around lines 5236–5249 and split slicing around lines 5345–5360. Check the existing tests for train_test_split() and add coverage for sizes whose sum is below 1. Done means the behavior is explicit and tested, so leftover rows are not silently discarded.

Written by the indexing model from the issue text.

Description

Found on main (39fd698d619816099ffa8eb07b5c79b0f4f5f299, datasets 5.2.1.dev0). I reproduced this live against the local checkout.

In src/datasets/arrow_dataset.py:5236-5249, n_train / n_test are computed from the two sizes, and only n_train + n_test > n_samples raises. The < case has no check, no warning, and the leftover rows are never assigned to either split (lines 5345-5360, permutation[n_test:(n_test+n_train)]).

Repro (verified live):

from datasets import Dataset
ds = Dataset.from_dict({"x": list(range(100))})
d = ds.train_test_split(train_size=0.5, test_size=0.2, seed=42)
len(d["train"]), len(d["test"])  # (50, 20) — 30 rows dropped, no warning

train_size=0.5, test_size=0.2 on 100 rows returns train=50, test=20 — 30 rows (30% of the data) silently vanish. A user who supplies both sizes reasonably expects them to partition the dataset (or an error); instead data is lost without a trace.

Why it matters: silent data loss in a data-splitting primitive. The dropped rows are invisible in the returned DatasetDict and can silently shrink training data and skew results.

I searched open issues for train_test_split + drop rows / train_size + test_size sum — no matching open issue (closest is #8425, about empty datasets).

Dominant language
Python
Stars
22.2k
Forks
3.7k
Avg merge
2d 19h
Merged PRs (30d)
19

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/datasets

All issues in huggingface/datasets

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.