Avoid side effects by cached state
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 55/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- python
- Domain
- performance
Research direction
Start with tpcbench.py and trace how query executions are performed. Check where cached tables persist between queries, then ensure they are cleared before each execution. Done means separate query runs no longer retain cached PySpark tables or affect benchmark results.
Written by the indexing model from the issue text.
Description
The existing TPCH benchmark script, tpcbench.py, retains PySpark's cached tables across different query executions. This can lead to unintended side effects on the benchmark results. Therefore, we should clear the cached tables before each query execution.
- Dominant language
- Python
- Stars
- 23
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/datafusion-benchmarks
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
apache/datafusion-benchmarks#30 · 4 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 42/100
apache/datafusion-benchmarks#22 · 2 comments ·
All issues in apache/datafusion-benchmarks
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100