Failure to collect operations that join two lazily retrieved Polars lazyframes
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 68/100
Research direction
Start at DuckDBPyRelation.pl(lazy=True) and reproduce the failure with the two relations joined through Polars, then follow the execution path into rel.join(...).collect(). Done means the supplied example collects successfully and returns the joined result without a ComputeError.
Written by the indexing model from the issue text.
Description
What happens?
I have downstream functions that expect multiple polars.LazyFrame objects. In my upstream code, I want to source my data from DuckDB to enable efficient pushdown. However, using DuckDBPyRelation.pl(lazy=True) raises a ComputeError when operating on multiple LazyFrame objects.
To Reproduce
import duckdb
with duckdb.connect() as con:
con.sql("CREATE TABLE tbl (t1 INTEGER)")
con.sql("CREATE TABLE tbl2 (t1 INTEGER)")
# Retrieve the results
rel = con.table("tbl").pl(lazy=True)
rel2 = con.table("tbl2").pl(lazy=True)
# Compute and raise the error
df = rel.join(rel2, on="t1", how="inner").collect()
the block above raises
ComputeError: caught exception during execution of a Python source, exception: InvalidInputException: Invalid Input Error: Attempting to execute an unsuccessful or closed pending query result
OS:
Linux - aarch64
DuckDB Package Version:
1.5.5
Python Version:
3.12
Full Name:
Akshay Gupta
Affiliation:
Capital One
What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.
I have tested with a stable release
Did you include all relevant data sets for reproducing the issue?
Yes
Did you include all code required to reproduce the issue?
- Yes, I have
Did you include all relevant configuration to reproduce the issue?
- Yes, I have
- Dominant language
- Python
- Stars
- 186
- Forks
- 113
- Avg merge
- 22h 45m
- Merged PRs (30d)
- 14
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from duckdb/duckdb-python
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
duckdb/duckdb-python#633 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
duckdb/duckdb-python#627 ·
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
duckdb/duckdb-python#576 · 3 comments ·
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
duckdb/duckdb-python#534 ·
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
duckdb/duckdb-python#386 ·
All issues in duckdb/duckdb-python
Similar issues
-
area: harness bug status: needs-triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Human-Agent-Society/reef#625 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 1/5 Under an hour Newbie friendliness 80/100
learningequality/kolibri#15351 · 2 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Name consistency Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
eellak/triplestore#65 · 1 comment ·