Dataset.from_list silently drops rows when records are empty dictionaries
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in the Dataset.from_list path, which builds the table from the list of dicts, and trace where an empty dict yields no schema/rows — likely the inferred features or the Arrow writer treat zero columns as zero rows. The issue names tests/test_dataset_list.py as the regression test; run it, add the [{}, {}] case there, and confirm len(ds) == 2 and ds.to_list() == [{}, {}]. Done means the empty-dict rows survive with no columns and the existing from_list tests still pass.
Written by the indexing model from the issue text.
Description
Dataset.from_list silently drops rows when records are empty dictionaries
Describe the bug
Dataset.from_list silently returns zero rows when the input contains empty dictionaries. The input records are lost even though they are valid rows with no fields.
Steps to reproduce
from datasets import Dataset
ds = Dataset.from_list([{}, {}])
print(len(ds)) # 0 on the affected version, should be 2
print(ds.to_list()) # [] on the affected version, should be [{}, {}]
The regression test is in tests/test_dataset_list.py.
Expected behavior
The dataset should have two rows and no columns: len(ds) == 2 and ds.to_list() == [{}, {}].
Actual behavior
The dataset has zero rows and no columns: len(ds) == 0 and ds.to_list() == [].
Environment
- Python 3.12.3
- datasets 5.2.1.dev0
- Linux-7.0.0-44-generic-x86_64-with-glibc2.39
- Dominant language
- Python
- Stars
- 22.2k
- Forks
- 3.7k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 18
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from huggingface/datasets
-
resolve_pattern() allowed_extensions filter accepts intermediate suffixes — c.parquet.tmp and b.csv.bak pass as parquet/CSV data filesPossibly taken @codingfeng-fufu claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
huggingface/datasets#9100 ·
Maintainers usually reply within 1 day
-
infer_module_for_data_files_list() counts every dotted filename component as an extension — data.tar.gz is inferred as webdataset, and the tie-break is reverse-alphabetical, contradicting the docstringPossibly taken @zhengdaodeguL claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
huggingface/datasets#9099 ·
Maintainers usually reply within 1 day
-
train_test_split() silently drops rows when train_size + test_size < 1Possibly taken @kevkoa2106 claimed this 2 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
huggingface/datasets#9098 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
Dataset.shuffle() / train_test_split() with seed=None are deterministic given the global NumPy seed and mutate global RNG state, contradicting the docsPossibly taken @zhengdaodeguL claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
huggingface/datasets#9097 ·
Maintainers usually reply within 1 day
-
`eval` loader fails when the first log in a folder has no samplesPossibly taken @BahaaMattar claimed this 4 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
huggingface/datasets#9069 ·
Maintainers usually reply within 1 day
All issues in huggingface/datasets
Similar issues
-
priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
OpenHands/extensions#975 ·
Maintainers usually reply within 1 day
-
Function/dict state handlers can't supply provider-state injected values (`generate.provider_state`)Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
pact-foundation/pact-python#1789 ·
Maintainers usually reply within 9 days
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
juandresrodca/AdminForge#66 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
backend/local comp/tools P2 tool/terminal type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NousResearch/hermes-agent#158788 ·
Maintainers usually reply within 1 day