Commit retry (0.12) always fails on catalogs that expose only the current snapshot (e.g. Nessie REST): "Cannot find starting snapshot", even for appends

Đang mở
  • Có gì không ổn?
#4,327 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@Anurag-S1ngh đang làm issue này rồi.

Từ ngày 10/11/2026.

  • #4333 của @Anurag-S1ngh — đang mở

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
35/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Đình trệ
Công nghệ
postgresql, python
Lĩnh vực
backend, databases

Hướng nghiên cứu

Read CommitWindow.resolve in pyiceberg/table/update/snapshot.py and the retry introduced by #3320; compare append behavior with operations that require conflict validation. The issue includes a linked open pull request, so check its changes and discussion first. Done means concurrent appends can retry when refreshed metadata lacks the starting snapshot, while validation dependent operations fail with a clear, appropriate error.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

kind:bug
Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞
Summary

Since #3320 (0.12.0), a commit that loses the race to a concurrent writer is retried. Before deciding whether any conflict validation is needed, the retry resolves the transaction's starting snapshot (CommitWindow.resolve in pyiceberg/table/update/snapshot.py):

base = metadata.snapshot_by_id(base_id) if base_id is not None else None
if base_id is not None and base is None:
    raise ValidationException(f"Cannot find starting snapshot {base_id}")

Nessie's Iceberg REST catalog exposes only each table's current snapshot. Table history lives in Nessie's own commit log, not in the table metadata. This is intentional (Nessie docs, projectnessie/nessie#9634; see also #1105 and #1368). So after a reload the starting snapshot is never there, and every retry fails with:

pyiceberg.exceptions.ValidationException: Cannot find starting snapshot <id>

This happens even for a plain append, which needs no conflict validation. On such catalogs the retry added in 0.12 can never succeed. Concurrent writers to the same table fail exactly as often as on 0.11; only the exception changes, from CommitFailedException to ValidationException.

Until 0.12, the single-snapshot behaviour only affected history and time-travel questions (#1105, #1368). With the retry, it makes every concurrent write to a table on these catalogs fail.

To reproduce

Nessie Iceberg REST catalog. Two writers append to different partitions of the same table at the same time:

import threading
from pyiceberg.catalog import load_catalog

barrier = threading.Barrier(2)

def writer(batch):
    table = load_catalog("nessie", **props).load_table("ns.tbl")   # REST catalog backed by Nessie
    barrier.wait()        # make both commits race
    table.append(batch)   # the loser raises ValidationException: Cannot find starting snapshot

threads = [threading.Thread(target=writer, args=(b,)) for b in (batch_partition_a, batch_partition_b)]
for t in threads: t.start()
for t in threads: t.join()

Observed (5 rounds of 2 concurrent writers, 200 rows each):

pyiceberg Appends committed Error on the losing writer
0.11.1 5 / 10 CommitFailedException
0.12.0 5 / 10 ValidationException: Cannot find starting snapshot <id>

table.overwrite(df, overwrite_filter=...) behaves the same, whether the filter matches whole partitions or individual rows.

Environment: pyiceberg 0.12.0 (also compared with 0.11.1), Python 3.12, Nessie Iceberg REST catalog (0.108.4), S3-compatible object storage (OCI), Trino 483 using the same catalog.

Expected

For operations that need no conflict validation (append), the retry should not need the starting snapshot. It should reload the table and re-apply on the current snapshot, as Java Iceberg does:

  • SnapshotProducer.commit() (source) retries on CommitFailedException;
  • each attempt's apply() reloads, takes the current latest snapshot as parent, and calls validate(base, parent);
  • that method is a no-op for FastAppend.

Verified against the same Nessie catalog: a deliberately slow Trino INSERT (Java Iceberg) whose starting snapshot was made stale by a concurrent commit to another partition still committed, with all rows present.

For operations that do validate (overwrite, delete), Java walks the history from the current snapshot back to the starting one (MergingSnapshotProducer.validationHistory, source) and fails clearly when it can't: "Cannot determine history between starting snapshot %s and the last known ancestor %s". A Trino row-level DELETE in the same setup failed that way. PyIceberg failing for these operations is reasonable. A message that says the catalog doesn't expose the snapshot history would help users tell this case apart from a real data conflict.

Possible fix

When the starting snapshot is not in the refreshed metadata:

  • Transactions with only appends: don't need the starting snapshot. Rebuild on the refreshed table's current snapshot, as Java's SnapshotProducer does.
  • Transactions that need validation (overwrite, delete): stop retrying and re-raise the original CommitFailedException (the 0.11 behaviour) instead of a ValidationException, so callers can tell a lost race from a real data conflict. These must never continue without validation: skipping the check silently would let them commit over a conflicting change.
Workaround
  • Around the whole write, catch CommitFailedException and ValidationException whose message contains "Cannot find starting snapshot", reload the table, and redo the write from scratch. This works on 0.11 and 0.12 against Nessie for appends and overwrites, with no duplicated rows.
  • Disabling the built-in retry with the table property commit.retry.num-retries=0 is not a good option on shared tables: Java Iceberg writers (e.g. Trino) read the same property and would stop retrying the appends they handle correctly today.
Related
  • #3320: introduced the commit retry and CommitWindow.
  • #4316 / #4317: also in CommitWindow, but about transactions that write several branches; a different cause.
  • #1105, #1368, projectnessie/nessie#9634: Nessie REST exposes only the current snapshot, by design.
Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Ngôn ngữ chính
Python
Star
1.2k
Fork
654
Merge trung bình
2 ngày 11 giờ
Pull request đã merge (30 ngày)
122

Chuẩn bị môi trường

  • Không có Dockerfile hay tệp Docker Compose
  • Có mẫu pull request
  • Không có hướng dẫn đóng góp

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của apache/iceberg-python

Tất cả issue của apache/iceberg-python

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.