REST create_table omits partition spec-id; Java-based servers (Polaris) reject it
Maintainers usually reply within 4 days
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 66/100
Research direction
Start in RestCatalog::create_table and at how UnboundPartitionSpec serializes its optional spec id, since the bug is that the field is omitted when unset. Check the serialized CreateTableRequest body for a partition spec with no with_spec_id call. Done when an unset spec id is sent as spec-id: 0 and a test asserts that body, with the Polaris repro from the issue as manual confirmation.
Written by the indexing model from the issue text.
Description
Apache Iceberg Rust version
0.10.1 (also present on main at 3bb8167)
Describe the bug
RestCatalog::create_table serializes the partition spec without spec-id when the UnboundPartitionSpec has none set. Since #2610, unset ids are omitted rather than sent as null.
Servers that parse the request with Iceberg Java's PartitionSpecParser require spec-id. For example, Apache Polaris 1.3.0 rejects the request with a bare HTTP 400 Bad Request (WebApplicationException) and logs nothing that points to the cause. Building the spec with .with_spec_id(0) makes the same request succeed.
#2702 notes the same parser behaviour for TableUpdate::AddSpec on the commit path. The create-table path hits it too.
To Reproduce
- Start Polaris 1.3.0 (in-memory persistence, bootstrap credentials) with an internal catalog.
- Create a table through
RestCatalogwith a partition spec built byUnboundPartitionSpec::builder().add_partition_field(...), withoutwith_spec_id. - Polaris answers 400.
The same request body replayed with curl:
"partition-spec": {"fields": [{"source-id": 27, "name": "receive_day", "transform": "day"}]}returns 400."partition-spec": {"spec-id": 0, "fields": [...]}returns 200.- Adding a
field-idwithoutspec-idstill returns 400.
Expected behavior
A table created through the REST catalog with an unbound partition spec is accepted by Java-based REST servers. One option is to send spec-id: 0 in CreateTableRequest when it is unset, since a new table's first spec is always 0.
Willingness to contribute
Happy to test a fix against Polaris.
AI disclosure: drafted with an AI coding assistant; reproduced and verified by me.
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 604
- Avg merge
- 5d 11h
- Merged PRs (30d)
- 59
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg-rust
-
V1 manifest lists written by Java lose added/existing/deleted file countsPossibly taken @kacper-li-airspace-intelligence claimed this 4 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-rust#3445 ·
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/iceberg-rust#3359 ·
Maintainers usually reply within 4 days
-
Row filter keeps every row for `<` and `<=` on a null column that is missing from the data fileOpen
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/iceberg-rust#3355 · 1 reaction ·
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-rust#3348 ·
Maintainers usually reply within 4 days
-
datafusion
Difficulty 1/5 Under an hour Newbie friendliness 88/100
apache/iceberg-rust#3324 · 1 reaction ·
Maintainers usually reply within 4 days
All issues in apache/iceberg-rust
Similar issues
-
area:ecosystem documentation
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Chelis-Lang/chelis#3866 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
metasequoiaime/msime#8620 ·
Maintainers usually reply within 1 day
-
agent-detection macos p2 triaged
Difficulty 1/5 Under an hour Newbie friendliness 90/100
Maintainers usually reply within 1 day
-
bug documentation sev-0
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Maintainers usually reply within 1 day