Support multi-argument transforms
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 58/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- data-engineering
Research direction
Start by tracing the partition-field model and its serialization, then follow scan-planning handling for unknown transforms. Compare the current single-argument source-id behavior with the V3 specification. Done means multi-argument source-ids load and serialize correctly, are treated as unknown during scan planning, and write-time evaluation remains out of scope.
Written by the indexing model from the issue text.
Description
Feature Request / Improvement
Part of #1818.
The V3 spec generalizes partition and sort fields to multiple source columns via source-ids. PyIceberg currently raises when source-ids has more than one element, so loading such tables fails — but the spec requires v3 readers to read tables with unknown transforms, ignoring them (null partition values, always-true projection in scan planning).
Scope: model source-ids, serialize per spec (source-id for single-argument transforms, source-ids otherwise), and treat multi-argument transforms as unknown in scan planning. Evaluating multi-argument transforms on write is out of scope until the spec defines concrete ones.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 589
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 70
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg-python
-
kind:bug
Difficulty 1/5 Under an hour Newbie friendliness 92/100
apache/iceberg-python#4006 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3996 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-python#3979 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3885 ·
-
[Bug] PyArrowFileIO fails to propagate s3.ssl.ca-cert to pyarrow.fs.S3FileSystem tls_ca_file_path Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/iceberg-python#3866 · 1 comment ·
All issues in apache/iceberg-python
Similar issues
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
canonical/paas-charm#368 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
tech debt
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
StevenBlack/hosts#3256 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
qualcomm/qai-appbuilder#275 ·