add_files support parquet files with field ids

Open
#3,556 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
54/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python
Domain
data-engineering

Research direction

Start by locating the add_files entry point and reading how it currently handles Parquet files with field IDs. The work is done when add_files accepts the field-ID-bearing files described in the report while preserving existing support for files without field IDs.

Written by the indexing model from the issue text.

Description

Feature Request / Improvement

I am using rust to commit to iceberg rest catalogue, however that requires field id's to be written for commit, however when I try to commit those same files using pyIceberg ( In case of re-initialisation ).
This error is shown

fb0b1e42-2c49-434c-b39a-da4586e7ef05-die_info.parquet because it has field IDs. `add_files` only supports addition of files without field_ids

If it is not assigned to someone I believe I can work on this.
Thanks!s

Dominant language
Python
Stars
1.1k
Forks
589
Avg merge
2d 4h
Merged PRs (30d)
72

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg-python

All issues in apache/iceberg-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.