Commit retry (0.12) always fails on catalogs that expose only the current snapshot (e.g. Nessie REST): "Cannot find starting snapshot", even for appends

Aberta
  • Algo errado?
#4,327 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

@Anurag-S1ngh já está trabalhando nisso.

Desde 10/11/2026.

  • #4333 de @Anurag-S1ngh — aberto

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
35/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Estagnada
Stack de tecnologia
postgresql, python
Domínio
backend, databases

Direção de pesquisa

Read CommitWindow.resolve in pyiceberg/table/update/snapshot.py and the retry introduced by #3320; compare append behavior with operations that require conflict validation. The issue includes a linked open pull request, so check its changes and discussion first. Done means concurrent appends can retry when refreshed metadata lacks the starting snapshot, while validation dependent operations fail with a clear, appropriate error.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

kind:bug
Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞
Summary

Since #3320 (0.12.0), a commit that loses the race to a concurrent writer is retried. Before deciding whether any conflict validation is needed, the retry resolves the transaction's starting snapshot (CommitWindow.resolve in pyiceberg/table/update/snapshot.py):

base = metadata.snapshot_by_id(base_id) if base_id is not None else None
if base_id is not None and base is None:
    raise ValidationException(f"Cannot find starting snapshot {base_id}")

Nessie's Iceberg REST catalog exposes only each table's current snapshot. Table history lives in Nessie's own commit log, not in the table metadata. This is intentional (Nessie docs, projectnessie/nessie#9634; see also #1105 and #1368). So after a reload the starting snapshot is never there, and every retry fails with:

pyiceberg.exceptions.ValidationException: Cannot find starting snapshot <id>

This happens even for a plain append, which needs no conflict validation. On such catalogs the retry added in 0.12 can never succeed. Concurrent writers to the same table fail exactly as often as on 0.11; only the exception changes, from CommitFailedException to ValidationException.

Until 0.12, the single-snapshot behaviour only affected history and time-travel questions (#1105, #1368). With the retry, it makes every concurrent write to a table on these catalogs fail.

To reproduce

Nessie Iceberg REST catalog. Two writers append to different partitions of the same table at the same time:

import threading
from pyiceberg.catalog import load_catalog

barrier = threading.Barrier(2)

def writer(batch):
    table = load_catalog("nessie", **props).load_table("ns.tbl")   # REST catalog backed by Nessie
    barrier.wait()        # make both commits race
    table.append(batch)   # the loser raises ValidationException: Cannot find starting snapshot

threads = [threading.Thread(target=writer, args=(b,)) for b in (batch_partition_a, batch_partition_b)]
for t in threads: t.start()
for t in threads: t.join()

Observed (5 rounds of 2 concurrent writers, 200 rows each):

pyiceberg Appends committed Error on the losing writer
0.11.1 5 / 10 CommitFailedException
0.12.0 5 / 10 ValidationException: Cannot find starting snapshot <id>

table.overwrite(df, overwrite_filter=...) behaves the same, whether the filter matches whole partitions or individual rows.

Environment: pyiceberg 0.12.0 (also compared with 0.11.1), Python 3.12, Nessie Iceberg REST catalog (0.108.4), S3-compatible object storage (OCI), Trino 483 using the same catalog.

Expected

For operations that need no conflict validation (append), the retry should not need the starting snapshot. It should reload the table and re-apply on the current snapshot, as Java Iceberg does:

  • SnapshotProducer.commit() (source) retries on CommitFailedException;
  • each attempt's apply() reloads, takes the current latest snapshot as parent, and calls validate(base, parent);
  • that method is a no-op for FastAppend.

Verified against the same Nessie catalog: a deliberately slow Trino INSERT (Java Iceberg) whose starting snapshot was made stale by a concurrent commit to another partition still committed, with all rows present.

For operations that do validate (overwrite, delete), Java walks the history from the current snapshot back to the starting one (MergingSnapshotProducer.validationHistory, source) and fails clearly when it can't: "Cannot determine history between starting snapshot %s and the last known ancestor %s". A Trino row-level DELETE in the same setup failed that way. PyIceberg failing for these operations is reasonable. A message that says the catalog doesn't expose the snapshot history would help users tell this case apart from a real data conflict.

Possible fix

When the starting snapshot is not in the refreshed metadata:

  • Transactions with only appends: don't need the starting snapshot. Rebuild on the refreshed table's current snapshot, as Java's SnapshotProducer does.
  • Transactions that need validation (overwrite, delete): stop retrying and re-raise the original CommitFailedException (the 0.11 behaviour) instead of a ValidationException, so callers can tell a lost race from a real data conflict. These must never continue without validation: skipping the check silently would let them commit over a conflicting change.
Workaround
  • Around the whole write, catch CommitFailedException and ValidationException whose message contains "Cannot find starting snapshot", reload the table, and redo the write from scratch. This works on 0.11 and 0.12 against Nessie for appends and overwrites, with no duplicated rows.
  • Disabling the built-in retry with the table property commit.retry.num-retries=0 is not a good option on shared tables: Java Iceberg writers (e.g. Trino) read the same property and would stop retrying the appends they handle correctly today.
Related
  • #3320: introduced the commit retry and CommitWindow.
  • #4316 / #4317: also in CommitWindow, but about transactions that write several branches; a different cause.
  • #1105, #1368, projectnessie/nessie#9634: Nessie REST exposes only the current snapshot, by design.
Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Linguagem predominante
Python
Estrelas
1.2k
Forks
654
Merge médio
2d 11h
PRs com merge (30d)
122

Preparar o ambiente

  • Sem Dockerfile nem arquivo Docker Compose
  • Tem um modelo de pull request
  • Sem guia de contribuição

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de apache/iceberg-python

Todas as issues de apache/iceberg-python

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.