[Bug] Data evolution self-merge can cross rollback snapshot lineage

Empieza ejecutando la reproducción ABA del rollback de Spark 3.5; después, sigue la identidad del snapshot base a través del staging de self-merge de Spark y la detección de conflictos de commit, y a continuación la implementación de PyPaimon Ray. Añade una cobertura de regresión equivalente para los casos latest < base, same-ID/different-UUID y rollback ABA. Se considera terminado cuando las actualizaciones de linajes antiguos fallen de forma segura o dejen sin cambios las filas de reemplazo en ambas implementaciones.

Description

Data evolution self-merge validates staged row-ID partial updates using the base snapshot ID, but not the snapshot UUID or lineage. Both the current Spark implementation and the PyPaimon Ray implementation in #9339 treat snapshot IDs as sufficient identity.

A rollback can delete newer snapshots, after which new commits reuse the same numeric snapshot IDs. A staged update from the old snapshot can therefore be accepted against a different replacement snapshot with the same ID (an ABA problem).

Reproduction

This has been reproduced with Spark 3.5:

1. Enable row tracking and data evolution.
2. Create snapshot 1 containing (id=1, b=10).
3. Create snapshot 2 containing (id=2, b=20).
4. Stage a self-merge partial update b = b + 1 for id=2, based on the original snapshot 2.
5. Roll back the table to snapshot 1.
6. Insert (id=30, b=300), recreating snapshot ID 2 with a different UUID and reusing the relevant row ID.
7. Commit the old staged update.

Expected: (id=30, b=300) remains unchanged, or the old commit fails closed.

Actual: the commit succeeds and produces (id=30, b=21). The old update is silently applied to the replacement row.

Trigger window and impact

The trigger window is narrow: a self-merge must remain staged across a rollback, snapshot IDs must be recreated, and row-ID ranges must overlap. However, the impact is a silent wrong-row update rather than a clean commit failure.

Follow-up scope

Fix Spark and Python together so that their behavior remains aligned:

• fail closed when latest snapshot ID < base snapshot ID;
• detect equal snapshot IDs with different snapshot UUIDs;
• prevent rollback snapshot-ID reuse from accepting staged updates from an old lineage;
• decide how the base snapshot UUID is carried through self-merge staging and commit conflict detection;
• add equivalent Spark and Python regression tests for latest < base, same-ID/different-UUID, and rollback ABA.

PR #9339 intentionally does not address this lineage problem. It remains aligned with the current Spark behavior and is scoped to forward snapshot rebase, conflict retry, and safe abort/temporary-file cleanup.