pola-rs/polars
在 GitHub 查看Confusing error message when reading CSV with duplicate column name
Open
#28,310 创建于 2026年7月9日
A-io-csvP-mediumacceptedbuggood first issuepython
仓库指标
- Star
- (38,496 star)
- PR 合并指标
- (平均合并 8天 3小时) (30 天内合并 156 个 PR)
描述
Checks
- I have checked that this issue has not already been reported.
- I have confirmed this bug exists on the latest version of Polars.
Reproducible example
import io
import polars as pl
data = "a,a_duplicated_0,a\n1,2,3"
print(pl.read_csv(io.StringIO(data)))
Log output
Traceback (most recent call last):
File "/home/elt/code/polars-test/csv_dup.py", line 5, in <module>
print(pl.read_csv(io.StringIO(data)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/io/csv/functions.py", line 581, in read_csv
df = _read_csv_impl(
^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/io/csv/functions.py", line 733, in _read_csv_impl
pydf = PyDataFrame.read_csv(
^^^^^^^^^^^^^^^^^^^^^
polars.exceptions.ComputeError: found more fields than defined in 'Schema'
Consider setting 'truncate_ragged_lines=True'.
Issue description
Duplicate column names in the CSV file are renamed into name_duplicated_N, however a column with this name might already exist in the file. (This might actually happen if someone writes back this pl.DataFrame as a csv file.) When this happens, the previous column in the schema is overwritten, so the schema ends up with fewer fields than the dataframe, leading to an obscure error. It would be nice if this was caught and renamed until all names are unique.
Expected behavior
shape: (1, 3)
┌─────┬────────────────┬────────────────┐
│ a ┆ a_duplicated_0 ┆ a_duplicated_1 │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 │
╞═════╪════════════════╪════════════════╡
│ 1 ┆ 2 ┆ 3 │
└─────┴────────────────┴────────────────┘