Commit retry (0.12) always fails on catalogs that expose only the current snapshot (e.g. Nessie REST): "Cannot find starting snapshot", even for appends

Ouverte
  • Quelque chose ne va pas ?
#4,327 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

@Anurag-S1ngh y travaille déjà.

Depuis le 10/11/2026.

  • #4333 par @Anurag-S1ngh — ouverte

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
35/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
À l'abandon
Stack technique
postgresql, python
Domaine
backend, databases

Piste de recherche

Read CommitWindow.resolve in pyiceberg/table/update/snapshot.py and the retry introduced by #3320; compare append behavior with operations that require conflict validation. The issue includes a linked open pull request, so check its changes and discussion first. Done means concurrent appends can retry when refreshed metadata lacks the starting snapshot, while validation dependent operations fail with a clear, appropriate error.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

kind:bug
Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞
Summary

Since #3320 (0.12.0), a commit that loses the race to a concurrent writer is retried. Before deciding whether any conflict validation is needed, the retry resolves the transaction's starting snapshot (CommitWindow.resolve in pyiceberg/table/update/snapshot.py):

base = metadata.snapshot_by_id(base_id) if base_id is not None else None
if base_id is not None and base is None:
    raise ValidationException(f"Cannot find starting snapshot {base_id}")

Nessie's Iceberg REST catalog exposes only each table's current snapshot. Table history lives in Nessie's own commit log, not in the table metadata. This is intentional (Nessie docs, projectnessie/nessie#9634; see also #1105 and #1368). So after a reload the starting snapshot is never there, and every retry fails with:

pyiceberg.exceptions.ValidationException: Cannot find starting snapshot <id>

This happens even for a plain append, which needs no conflict validation. On such catalogs the retry added in 0.12 can never succeed. Concurrent writers to the same table fail exactly as often as on 0.11; only the exception changes, from CommitFailedException to ValidationException.

Until 0.12, the single-snapshot behaviour only affected history and time-travel questions (#1105, #1368). With the retry, it makes every concurrent write to a table on these catalogs fail.

To reproduce

Nessie Iceberg REST catalog. Two writers append to different partitions of the same table at the same time:

import threading
from pyiceberg.catalog import load_catalog

barrier = threading.Barrier(2)

def writer(batch):
    table = load_catalog("nessie", **props).load_table("ns.tbl")   # REST catalog backed by Nessie
    barrier.wait()        # make both commits race
    table.append(batch)   # the loser raises ValidationException: Cannot find starting snapshot

threads = [threading.Thread(target=writer, args=(b,)) for b in (batch_partition_a, batch_partition_b)]
for t in threads: t.start()
for t in threads: t.join()

Observed (5 rounds of 2 concurrent writers, 200 rows each):

pyiceberg Appends committed Error on the losing writer
0.11.1 5 / 10 CommitFailedException
0.12.0 5 / 10 ValidationException: Cannot find starting snapshot <id>

table.overwrite(df, overwrite_filter=...) behaves the same, whether the filter matches whole partitions or individual rows.

Environment: pyiceberg 0.12.0 (also compared with 0.11.1), Python 3.12, Nessie Iceberg REST catalog (0.108.4), S3-compatible object storage (OCI), Trino 483 using the same catalog.

Expected

For operations that need no conflict validation (append), the retry should not need the starting snapshot. It should reload the table and re-apply on the current snapshot, as Java Iceberg does:

  • SnapshotProducer.commit() (source) retries on CommitFailedException;
  • each attempt's apply() reloads, takes the current latest snapshot as parent, and calls validate(base, parent);
  • that method is a no-op for FastAppend.

Verified against the same Nessie catalog: a deliberately slow Trino INSERT (Java Iceberg) whose starting snapshot was made stale by a concurrent commit to another partition still committed, with all rows present.

For operations that do validate (overwrite, delete), Java walks the history from the current snapshot back to the starting one (MergingSnapshotProducer.validationHistory, source) and fails clearly when it can't: "Cannot determine history between starting snapshot %s and the last known ancestor %s". A Trino row-level DELETE in the same setup failed that way. PyIceberg failing for these operations is reasonable. A message that says the catalog doesn't expose the snapshot history would help users tell this case apart from a real data conflict.

Possible fix

When the starting snapshot is not in the refreshed metadata:

  • Transactions with only appends: don't need the starting snapshot. Rebuild on the refreshed table's current snapshot, as Java's SnapshotProducer does.
  • Transactions that need validation (overwrite, delete): stop retrying and re-raise the original CommitFailedException (the 0.11 behaviour) instead of a ValidationException, so callers can tell a lost race from a real data conflict. These must never continue without validation: skipping the check silently would let them commit over a conflicting change.
Workaround
  • Around the whole write, catch CommitFailedException and ValidationException whose message contains "Cannot find starting snapshot", reload the table, and redo the write from scratch. This works on 0.11 and 0.12 against Nessie for appends and overwrites, with no duplicated rows.
  • Disabling the built-in retry with the table property commit.retry.num-retries=0 is not a good option on shared tables: Java Iceberg writers (e.g. Trino) read the same property and would stop retrying the appends they handle correctly today.
Related
  • #3320: introduced the commit retry and CommitWindow.
  • #4316 / #4317: also in CommitWindow, but about transactions that write several branches; a different cause.
  • #1105, #1368, projectnessie/nessie#9634: Nessie REST exposes only the current snapshot, by design.
Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Langage dominant
Python
Étoiles
1.2k
Forks
654
Merge moyen
2 j 11 h
PR mergées (30 j)
122

Préparer son environnement

  • Aucun Dockerfile ni fichier Docker Compose
  • Propose un modèle de pull request
  • Aucun guide de contribution

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de apache/iceberg-python

Toutes les issues de apache/iceberg-python

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.