Could not write temp file and throw Invalid argument during BIG table sort
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Start by reproducing the DuckDB CLI command against a comparable 56-million-row asset_tree table, sorting by hierarchy_path and observing the temporary files under /db/com.db.tmp. Done means the CREATE TABLE sort completes without the Invalid argument error or reports a confirmed platform or filesystem limitation.
Written by the indexing model from the issue text.
Description
As you can see below, i have a table with 56 million rows, and attemping to perform sorting to "hierarchy_path" column will cause it to fail with invalid argument. i observed 400 + tmp files are being created on and off during the execution and after 5 minutes it goes off with error, i consider it crashed or at least not completing the SQL to create the table.
duckdb "/db/com.db"
v1.2.1 8e52ec4395
Enter ".help" for usage hints.
D select count(*) from asset_tree;
┌─────────────────┐
│ count_star() │
│ int64 │
├─────────────────┤
│ 56452767 │
│ (56.45 million) │
└─────────────────┘
D
duckdb "/db/com.db"
v1.2.1 8e52ec4395
Enter ".help" for usage hints.
D create table eric as
select * from asset_tree order by hierarchy_path;
49% ▕█████████████████████████████▍ ▏ IO Error:
Could not write file "/db/com.db.tmp/duckdb_temp_block-4611686018427598582.block": Invalid argument
D .quit
- Dominant language
- C++
- Stars
- 127
- Forks
- 80
- Avg merge
- 13h 41m
- Merged PRs (30d)
- 44
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from duckdb/duckdb-java
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
duckdb/duckdb-java#832 · 2 comments ·
-
Native SIGSEGV in DuckDB JDBC when repeatedly executing queries with window functions and LIMIT 0 Open
Difficulty 4/5 3-5 days Newbie friendliness 48/100
duckdb/duckdb-java#871 · 4 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
duckdb/duckdb-java#872 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
duckdb/duckdb-java#837 ·
-
executeBatch()` on a prepared INSERT runs s are ~1–2 orders of magnitude slower than the Appender Open
Difficulty 5/5 Over a week Newbie friendliness 42/100
duckdb/duckdb-java#815 · 1 comment ·
All issues in duckdb/duckdb-java
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
bug-unconfirmed
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/cuda-samples#453 ·