overwrite parameter to relation.write_csv and write_parquet seems to do nothing

Open
#156 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
50/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
databases

Research direction

Reproduce the issue with DuckDBPyRelation.write_csv and write_parquet using the two identical write_csv calls shown in the report. Trace the overwrite parameter from those relation methods into the write operation and determine whether overwrite=False should reject an existing target. Done means the parameter has the expected effect, with coverage for both methods, or its behavior is clearly documented if it is intentional.

Written by the indexing model from the issue text.

Description

needs triage
What happens?

The DuckDBPyRelation methods write_csv and write_parquet both have a parameter called overwrite: bool. Setting it to either False or True does nothing as far as I can tell; the target file is always overwritten if it exists.

I expected setting overwrite=False to raise an error if the target already exists. If it's meant to something else, can you please document what it does?

I can't find a matching parameter in duckdb COPY TO, so maybe it's meant for something else and I'm just confused.

To Reproduce
conn.sql('select * from range(10)').write_csv('/tmp/foo.csv', overwrite=False)
conn.sql('select * from range(10)').write_csv('/tmp/foo.csv', overwrite=False)

The file is overwritten two times in a row.

OS:

Linux

DuckDB Package Version:

1.4.1

Python Version:

3.12.12

Full Name:

Daniel Armak

Affiliation:

SparkBeyond

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?
  • Yes, I have
Did you include all relevant configuration to reproduce the issue?
  • Yes, I have
Dominant language
Python
Stars
186
Forks
113
Avg merge
20h 58m
Merged PRs (30d)
11

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from duckdb/duckdb-python

All issues in duckdb/duckdb-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.