pola-rs/polars

Confusing error message when reading CSV with duplicate column name

Open

#28,310 创建于 2026年7月9日

在 GitHub 查看
 (3 评论) (1 反应) (0 负责人)Rust (2,826 fork)batch import
A-io-csvP-mediumacceptedbuggood first issuepython

仓库指标

Star
 (38,496 star)
PR 合并指标
 (平均合并 8天 3小时) (30 天内合并 156 个 PR)

描述

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import io
import polars as pl
data = "a,a_duplicated_0,a\n1,2,3"
print(pl.read_csv(io.StringIO(data)))

Log output

Traceback (most recent call last):
File "/home/elt/code/polars-test/csv_dup.py", line 5, in <module>
print(pl.read_csv(io.StringIO(data)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/_utils/deprecation.py", line 128, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/io/csv/functions.py", line 581, in read_csv
df = _read_csv_impl(
^^^^^^^^^^^^^^^
File "/home/elt/code/polars-test/.venv/lib/python3.12/site-packages/polars/io/csv/functions.py", line 733, in _read_csv_impl
pydf = PyDataFrame.read_csv(
^^^^^^^^^^^^^^^^^^^^^
polars.exceptions.ComputeError: found more fields than defined in 'Schema'
Consider setting 'truncate_ragged_lines=True'.

Issue description

Duplicate column names in the CSV file are renamed into name_duplicated_N, however a column with this name might already exist in the file. (This might actually happen if someone writes back this pl.DataFrame as a csv file.) When this happens, the previous column in the schema is overwritten, so the schema ends up with fewer fields than the dataframe, leading to an obscure error. It would be nice if this was caught and renamed until all names are unique.

Expected behavior

shape: (1, 3)
┌─────┬────────────────┬────────────────┐
│ a ┆ a_duplicated_0 ┆ a_duplicated_1 │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 │
╞═════╪════════════════╪════════════════╡
│ 1 ┆ 2 ┆ 3 │
└─────┴────────────────┴────────────────┘

Installed versions

贡献者指南