Dataset.from_list silently drops rows when records are empty dictionaries

Open Beginner friendly
#9,128 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
data

Research direction

Start in the Dataset.from_list path, which builds the table from the list of dicts, and trace where an empty dict yields no schema/rows — likely the inferred features or the Arrow writer treat zero columns as zero rows. The issue names tests/test_dataset_list.py as the regression test; run it, add the [{}, {}] case there, and confirm len(ds) == 2 and ds.to_list() == [{}, {}]. Done means the empty-dict rows survive with no columns and the existing from_list tests still pass.

Written by the indexing model from the issue text.

Description

Dataset.from_list silently drops rows when records are empty dictionaries

Describe the bug

Dataset.from_list silently returns zero rows when the input contains empty dictionaries. The input records are lost even though they are valid rows with no fields.

Steps to reproduce

from datasets import Dataset

ds = Dataset.from_list([{}, {}])
print(len(ds))        # 0 on the affected version, should be 2
print(ds.to_list())   # [] on the affected version, should be [{}, {}]

The regression test is in tests/test_dataset_list.py.

Expected behavior

The dataset should have two rows and no columns: len(ds) == 2 and ds.to_list() == [{}, {}].

Actual behavior

The dataset has zero rows and no columns: len(ds) == 0 and ds.to_list() == [].

Environment

  • Python 3.12.3
  • datasets 5.2.1.dev0
  • Linux-7.0.0-44-generic-x86_64-with-glibc2.39
Dominant language
Python
Stars
22.2k
Forks
3.7k
Avg merge
2d 22h
Merged PRs (30d)
18

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/datasets

All issues in huggingface/datasets

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.