maxRows option for executeStatement is not taken into account
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- sql, typescript
- Domain
- api
Research direction
Start at the session.executeStatement entry point and trace how its maxRows option is passed into the operation. Compare that path with fetchChunk, then run the TypeScript reproduction from the issue and confirm that fetchAll respects maxRows while preserving the documented behavior.
Written by the indexing model from the issue text.
Description
Version
I am using the 1.8.4 version
Description
We want to limit the amount of RAM usage. passing a default maxRows option has no effect, the whole query is returned
How to reproduce
const operation = await this.session.executeStatement(
`WITH numbers AS (
SELECT num
FROM (SELECT EXPLODE(SEQUENCE(1, 100)) AS num) AS seq
)
SELECT num
FROM numbers
ORDER BY RAND()`,
{ maxRows: 10 },
);
const fetched = await operation.fetchAll();
console.log(fetched.length);
You will notice here that fetched has a size of 100, not 10.
Workaround
The workaround is to use fetchChunk instead:
const chunk = await operation.fetchChunk({
maxRows: limit - result.length,
});
Please deprecate or fix maxRows options for the executeStatement method of the session object
- Dominant language
- TypeScript
- Stars
- 36
- Forks
- 50
- Avg merge
- 13h 46m
- Merged PRs (30d)
- 9
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from databricks/databricks-sql-nodejs
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
engineer-bot
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
databricks/databricks-sql-nodejs#274 · 1 comment · 1 reaction ·
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
All issues in databricks/databricks-sql-nodejs
Similar issues
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
dennys-bd/agent-hive#184 ·
-
Add: hunch Open
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
AbdelStark/awesome-typesafe#104 ·
-
ai-observability bug team/ai-observability
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
vicharanashala/fln#563 ·
-
review: medium
Difficulty 1/5 Under an hour Newbie friendliness 93/100
langgenius/dify#42764 · 1 reaction ·