Implement iterator to lazily go through the paged response
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 65/100
Research direction
Start by reading the paged list-view implementation introduced in PR 3349 and its linked discussion. Trace how pages are currently iterated, then verify that consuming the iterator requests each next page only when needed instead of fetching all pages upfront.
Written by the indexing model from the issue text.
Description
Feature Request / Improvement
Recently the support for paged response was added to the list-view operationr: https://github.com/apache/iceberg-python/pull/3349. Right now it will just iterate over all the pages, instead it would be good to see if we can call the next page lazily to avoid many network calls: https://github.com/apache/iceberg-python/pull/3349#issuecomment-4462173769
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 589
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 70
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg-python
-
kind:bug
Difficulty 1/5 Under an hour Newbie friendliness 92/100
apache/iceberg-python#4006 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3996 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-python#3979 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3885 ·
-
[Bug] PyArrowFileIO fails to propagate s3.ssl.ca-cert to pyarrow.fs.S3FileSystem tls_ca_file_path Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/iceberg-python#3866 · 1 comment ·
All issues in apache/iceberg-python
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
learningequality/ricecooker#747 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
BSData/horus-heresy-3rd-edition#3171 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
run-llama/llama_index#23199 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
KhronosGroup/glTF-Blender-IO#2769 ·