Why non-atomic delete and upsert?
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- data-engineering, databases
Research direction
The issue names no files, tests, or entry points; begin by locating the delete and upsert implementations in PyIceberg and reviewing how they create snapshots. Determine whether the observed multiple-snapshot behavior is intentional, then document the rationale or define the required atomic behavior before making a change.
Written by the indexing model from the issue text.
Description
Question
The implementation of the delete and upsert operation make an deliberate choice to create multiple snapshots, breaking the atomicity expectation.
For example, if I start a delete operation to remove some entries, I do not expect consumers of the data to see a snapshot with some but not all of the entries gone.
Given that the implementation choice looks very deliberate, I'm curious about the rationale for it?
It seems that other clients of the library expect atomic operation as well, for example https://github.com/apache/iceberg-python/issues/3076
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 589
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 72
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3996 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-python#3979 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3885 ·
-
[Bug] PyArrowFileIO fails to propagate s3.ssl.ca-cert to pyarrow.fs.S3FileSystem tls_ca_file_path Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/iceberg-python#3866 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3836 · 1 comment ·
All issues in apache/iceberg-python
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
learningequality/ricecooker#747 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
BSData/horus-heresy-3rd-edition#3171 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
run-llama/llama_index#23199 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
KhronosGroup/glTF-Blender-IO#2769 ·