DELETE operations don't apply custom snapshot properties from session config
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
Research direction
Start in SparkTable.deleteFromRowFilter() and compare its handling with the insert, update, and merge paths. Check how SparkWriteConf.extraSnapshotMetadata() is used, then run the provided Spark session configuration and DELETE reproduction. Done means the DELETE snapshot contains the configured custom property, matching other DML operations.
Written by the indexing model from the issue text.
Description
Apache Iceberg version
main (development)
Query engine
Spark
Please describe the bug 🐞
Description
Custom snapshot properties set via Spark session config (e.g., spark.sql.iceberg.snapshot-property.custom-key) are not applied to DELETE operations, while they work correctly for INSERT, UPDATE, and MERGE operations.
Expected Behavior
DELETE should respect custom snapshot properties from session config, consistent with other DML operations.
Actual Behavior
DELETE operations silently ignore custom snapshot properties.
Steps to Reproduce
spark.conf().set("spark.sql.iceberg.snapshot-property.my-key", "my-value");
// INSERT works - snapshot has my-key ✅
sql("INSERT INTO table VALUES (1, 'a')");
// DELETE doesn't work - snapshot missing my-key ❌
sql("DELETE FROM table WHERE id = 1");
Root Cause
In SparkTable.deleteFromRowFilter(), the code doesn't call SparkWriteConf.extraSnapshotMetadata() to apply session properties, unlike insert/update paths.
Impact
Users relying on custom snapshot properties for tracking (e.g., audit trails, job metadata) will have incomplete information for DELETE operations.
Related
- Feature introduced in #14545
- Affects Spark 4.0 and 4.1
Willingness to contribute
- I can contribute a fix for this bug independently
- I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- I cannot contribute a fix for this bug at this time
- Dominant language
- Java
- Stars
- 9.3k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 143
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
improvement
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100