Commit retry (0.12) always fails on catalogs that expose only the current snapshot (e.g. Nessie REST): "Cannot find starting snapshot", even for appends

Open
  • Is something wrong?
#4,327 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Anurag-S1ngh is already working on this.

Since Nov 10, 2026.

  • #4333 by @Anurag-S1ngh — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
postgresql, python
Domain
backend, databases

Research direction

Read CommitWindow.resolve in pyiceberg/table/update/snapshot.py and the retry introduced by #3320; compare append behavior with operations that require conflict validation. The issue includes a linked open pull request, so check its changes and discussion first. Done means concurrent appends can retry when refreshed metadata lacks the starting snapshot, while validation dependent operations fail with a clear, appropriate error.

Written by the indexing model from the issue text.

Description

kind:bug
Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞
Summary

Since #3320 (0.12.0), a commit that loses the race to a concurrent writer is retried. Before deciding whether any conflict validation is needed, the retry resolves the transaction's starting snapshot (CommitWindow.resolve in pyiceberg/table/update/snapshot.py):

base = metadata.snapshot_by_id(base_id) if base_id is not None else None
if base_id is not None and base is None:
    raise ValidationException(f"Cannot find starting snapshot {base_id}")

Nessie's Iceberg REST catalog exposes only each table's current snapshot. Table history lives in Nessie's own commit log, not in the table metadata. This is intentional (Nessie docs, projectnessie/nessie#9634; see also #1105 and #1368). So after a reload the starting snapshot is never there, and every retry fails with:

pyiceberg.exceptions.ValidationException: Cannot find starting snapshot <id>

This happens even for a plain append, which needs no conflict validation. On such catalogs the retry added in 0.12 can never succeed. Concurrent writers to the same table fail exactly as often as on 0.11; only the exception changes, from CommitFailedException to ValidationException.

Until 0.12, the single-snapshot behaviour only affected history and time-travel questions (#1105, #1368). With the retry, it makes every concurrent write to a table on these catalogs fail.

To reproduce

Nessie Iceberg REST catalog. Two writers append to different partitions of the same table at the same time:

import threading
from pyiceberg.catalog import load_catalog

barrier = threading.Barrier(2)

def writer(batch):
    table = load_catalog("nessie", **props).load_table("ns.tbl")   # REST catalog backed by Nessie
    barrier.wait()        # make both commits race
    table.append(batch)   # the loser raises ValidationException: Cannot find starting snapshot

threads = [threading.Thread(target=writer, args=(b,)) for b in (batch_partition_a, batch_partition_b)]
for t in threads: t.start()
for t in threads: t.join()

Observed (5 rounds of 2 concurrent writers, 200 rows each):

pyiceberg Appends committed Error on the losing writer
0.11.1 5 / 10 CommitFailedException
0.12.0 5 / 10 ValidationException: Cannot find starting snapshot <id>

table.overwrite(df, overwrite_filter=...) behaves the same, whether the filter matches whole partitions or individual rows.

Environment: pyiceberg 0.12.0 (also compared with 0.11.1), Python 3.12, Nessie Iceberg REST catalog (0.108.4), S3-compatible object storage (OCI), Trino 483 using the same catalog.

Expected

For operations that need no conflict validation (append), the retry should not need the starting snapshot. It should reload the table and re-apply on the current snapshot, as Java Iceberg does:

  • SnapshotProducer.commit() (source) retries on CommitFailedException;
  • each attempt's apply() reloads, takes the current latest snapshot as parent, and calls validate(base, parent);
  • that method is a no-op for FastAppend.

Verified against the same Nessie catalog: a deliberately slow Trino INSERT (Java Iceberg) whose starting snapshot was made stale by a concurrent commit to another partition still committed, with all rows present.

For operations that do validate (overwrite, delete), Java walks the history from the current snapshot back to the starting one (MergingSnapshotProducer.validationHistory, source) and fails clearly when it can't: "Cannot determine history between starting snapshot %s and the last known ancestor %s". A Trino row-level DELETE in the same setup failed that way. PyIceberg failing for these operations is reasonable. A message that says the catalog doesn't expose the snapshot history would help users tell this case apart from a real data conflict.

Possible fix

When the starting snapshot is not in the refreshed metadata:

  • Transactions with only appends: don't need the starting snapshot. Rebuild on the refreshed table's current snapshot, as Java's SnapshotProducer does.
  • Transactions that need validation (overwrite, delete): stop retrying and re-raise the original CommitFailedException (the 0.11 behaviour) instead of a ValidationException, so callers can tell a lost race from a real data conflict. These must never continue without validation: skipping the check silently would let them commit over a conflicting change.
Workaround
  • Around the whole write, catch CommitFailedException and ValidationException whose message contains "Cannot find starting snapshot", reload the table, and redo the write from scratch. This works on 0.11 and 0.12 against Nessie for appends and overwrites, with no duplicated rows.
  • Disabling the built-in retry with the table property commit.retry.num-retries=0 is not a good option on shared tables: Java Iceberg writers (e.g. Trino) read the same property and would stop retrying the appends they handle correctly today.
Related
  • #3320: introduced the commit retry and CommitWindow.
  • #4316 / #4317: also in CommitWindow, but about transactions that write several branches; a different cause.
  • #1105, #1368, projectnessie/nessie#9634: Nessie REST exposes only the current snapshot, by design.
Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Dominant language
Python
Stars
1.2k
Forks
664
Avg merge
2d 10h
Merged PRs (30d)
131

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg-python

All issues in apache/iceberg-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.