REST: Send empty first-page pageToken when rest-page-size is set
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 82/100
Research direction
Start at the REST catalog pagination code for list_tables, list_views, and list_namespaces, then inspect the SigV4 adapter’s query parsing. Send an empty pageToken on the first request when rest-page-size is set, and preserve blank query values in the adapter. Verify that the request includes pageToken= and that pagination works with a page size of 1.
Written by the indexing model from the issue text.
Description
Apache Iceberg version
0.12.0 (latest release)
Please describe the bug 🐞
When rest-page-size is set, list_tables, list_views and list_namespaces send pageSize without pageToken on the first request:
GET /v1/namespaces/ns/tables?pageSize=100
The REST spec says servers must return all results in one response when pageToken is not set, and that clients start pagination by sending an empty pageToken (PageToken). So compliant servers ignore rest-page-size, and some servers fail on the request. With tabulario/iceberg-rest:1.6.0 and namespaces a, b, c:
RestCatalog("t", uri=uri).list_namespaces()
# [('a',), ('b',), ('c',)]
RestCatalog("t", uri=uri, **{"rest-page-size": "1"}).list_namespaces()
# ServerError: NumberFormatException: Cannot parse null string
GET /v1/namespaces?pageSize=1 -> 500 NumberFormatException
GET /v1/namespaces?pageSize=1&pageToken= -> {"namespaces": [["a"]], "next-page-token": "1"}
The first request should include pageToken= when rest-page-size is set. The Java client always sends pageToken, starting with an empty value. The SigV4 adapter also needs to keep blank query values, since parse_qsl drops them by default.
rest-page-size was added in #3377. The same error in apache/iceberg-rest-fixture was reported in apache/iceberg#13119.
Willingness to contribute
- I can contribute a fix for this bug independently
- I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- I cannot contribute a fix for this bug at this time
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 664
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 131
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg-python
-
StringLiteral to IntegerType/LongType silently truncates fractional strings, changing predicate meaningPossibly taken @Shahzeb-DataEngineer claimed this today. Openkind:bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
apache/iceberg-python#4382 · 2 comments ·
Maintainers usually reply within 1 day
-
`inspect.files()` and the other file metadata tables raise `ArrowInvalid` when a snapshot has no manifestsPossibly taken @YuangGao claimed this 12 days ago. Openkind:bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-python#4269 ·
Maintainers usually reply within 1 day
-
`inspect.entries()` raises TypeError when a data file has no column_sizesPossibly taken @Rahul-cs50 claimed this 16 days ago. Openkind:bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#4186 · 2 comments ·
Maintainers usually reply within 1 day
-
`older_than` reads a naive datetime as UTC and expires snapshots it should keepPossibly taken @Rahul-cs50 claimed this 14 days ago. Openkind:bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-python#4182 · 1 comment ·
Maintainers usually reply within 1 day
-
PyArrowFileIO: every small S3 write is a 3-request multipart upload; expose allow_delayed_openPossibly taken @David-Banquet claimed this 30 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
apache/iceberg-python#4093 · 3 comments ·
Maintainers usually reply within 1 day
All issues in apache/iceberg-python
Similar issues
-
ライブラリを追加Openpriority middle
Difficulty 1/5 Under an hour Newbie friendliness 62/100
KATO-Hiro/AtCoderClans#13015 ·
Maintainers usually reply within 1 day
-
request-theme
Difficulty 2/5 Under an hour Newbie friendliness 62/100
LizardByte/ThemerrDB#10024 · 2 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
btclib-org/btclib-node#2505 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
open-webui/open-webui#34462 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
KLayout/klayout#2554 · 1 comment ·
Maintainers usually reply within 1 day