[RFC] Pluggable pre-commit check in the Java client's OpenHouseTableOperations
Maintainers usually reply within 2 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- java
- Domain
- backend-api-design
Research direction
Start by reading OpenHouseTableOperations, especially doCommit and the existing loadMetadata hook; compare the client proposal with the server-side checks in #831. The issue identifies openhouse-java-itest for tests, but asks maintainers to choose between several API designs and clarify documentation requirements. Done means agreeing on the approach and implementing and testing the client-side pre-commit check.
Written by the indexing model from the issue text.
Description
Motivation
OpenHouseTableOperations.doCommit is the client-side owner of commit routing and mutation. Today there is no way for a catalog built on top of it to run a decision before a commit mutates anything.
Integrators who need one — a policy check, a validation gate, a guard against a class of commit — have to subclass OpenHouseTableOperations and override doCommit. That has a specific cost: doCommit classifies the commit using three locals at :201, :202 and :207, and one of the methods behind them, isMetadataUpdated, is private. A subclass that needs to know the route has to re-implement it.
That copy drifts, and there is already a concrete example: #351 added a fourth clause to isMetadataUpdated (sortOrder), where it previously compared three fields. Any downstream copy written before that change silently classified sortOrder-only commits as no-ops — with no compile error and no test failure. The next such change will do the same thing to whoever is copying it today.
Two pieces of existing work suggest this is a gap worth closing rather than a new direction:
- #641 added an overridable
loadMetadatahook to this same class, so subclasses can transform metadata as it loads without duplicatingdoRefresh. That is the same problem on the read path. The write path has no equivalent. - #831 added pluggable pre-commit checks to the server-side
OpenHouseInternalTableOperations. This proposes the client-side mirror, so a catalog embedded in a Spark or Trino driver has the same extension point.
Proposal
A small capability in the Java client runtime, with no behaviour change when nothing is registered.
Three types, in a new javaclient/precommit package:
@FunctionalInterface
public interface CommitPrecondition {
CommitPrecondition NOOP = context -> { };
/**
* @throws CommitPreconditionException to deny the commit.
* Unchecked by necessity: doCommit declares no throws clause.
*/
void check(CommitContext context);
default CommitPrecondition andThen(CommitPrecondition next) { ... }
}
CommitContext — an immutable value carrying only what doCommit already knows: table identifier, cluster, the three route flags, and the base and proposed TableMetadata. It exposes the route as a derived enum (REPLACE, SNAPSHOT, METADATA, NOOP) rather than storing it, so the enum cannot disagree with the flags it came from.
CommitPreconditionException — unchecked, carrying a closed reason enum. Deliberately not a CommitFailedException: Iceberg's Tasks loop retries those, and a deterministic denial fails the same way every time, so wrapping it there would retry to exhaustion. #831 reached the same conclusion independently on the server side.
Invocation point — after the route is classified, before the first mutation:
boolean metadataUpdated = isMetadataUpdated(base, metadata); // :201
boolean snapshotsUpdated = areSnapshotsUpdated(base, metadata); // :202
boolean replaceCommit = isReplaceTransaction() && base != null; // :207
+ precommit.get().check(context);
try { ... } // :208
Everything happens before any mutation, so a denial can never surface as CommitStateUnknownException.
Composition, not a constructor change. A final field with an initializer is excluded from Lombok's @Builder and @AllArgsConstructor — the same arrangement the file already uses for replaceTransaction at :83:
private final AtomicReference<CommitPrecondition> precommit =
new AtomicReference<>(CommitPrecondition.NOOP);
public OpenHouseTableOperations withPreCommit(CommitPrecondition p) {
precommit.set(Objects.requireNonNull(p));
return this;
}
No signature changes, so nothing downstream recompiles, and existing subclasses that call super(...) with the current argument list keep working.
Contract for implementations, mirroring #831's:
- check only — no mutation, no side effects
- synchronous and bounded
- reads the current base only
- idempotent and safe to repeat during an Iceberg commit retry
- thread-safe: one instance may serve many table operations concurrently
Why the context carries the flags rather than just (base, metadata). This is the part that matters. A hook given only the two metadata objects forces every integrator to re-derive the route, which means re-implementing the private isMetadataUpdated — the exact drift this proposal exists to remove. The flags are already computed three lines above the hook site; passing them costs nothing and is the whole benefit.
Alternatives
1. Make isMetadataUpdated package-private.
One-word change, and it removes the re-implementation. But integrators still have to override doCommit and still duplicate the routing logic around it, so the two layers can still disagree about ordering and about what runs before a mutation. Narrower and cheaper; solves less. A reasonable fallback if the capability itself isn't wanted.
2. Subclass and override doCommit.
What integrators do today. It works, but it puts commit orchestration in two layers where only one owns routing and mutation, and it requires the private-method copy.
3. Reuse the server-side types from #831.
Not reachable from the client. PreCommitCheck and PendingCommit live in a Spring Boot server module, discovery is Spring bean collection, the runner takes a Micrometer MeterRegistry, and the module builds against a different Iceberg version than the client runtime. Beyond packaging, PendingCommit is assembled from commit-only property keys that a client driver never sees, so an extracted type would share a name without sharing a contract.
That said — if maintainers would prefer the same type names on both sides for symmetry (PreCommitCheck / PendingCommit rather than CommitPrecondition / CommitContext), that's an easy change and arguably the better outcome. Happy to follow whichever reads as one pattern rather than two.
4. A protected overridable method, matching #641's loadMetadata.
This is the closest match to existing style in this class, and it would work:
protected void preCommit(TableMetadata base, TableMetadata metadata,
boolean replaceCommit, boolean metadataUpdated,
boolean snapshotsUpdated) { }
It's a smaller diff than the capability and needs no new types. Two tradeoffs, and we'd defer to maintainers on whether they matter:
- It requires subclassing to use, so it only serves integrators who already subclass. #831's server-side equivalent is composable instead.
- Checks don't stack — a subclass gets one override, so two independent concerns have to be manually combined inside it.
If the parameter list feels wide, that's really an argument for grouping the flags into a context object, which lands back at the proposal above. We're genuinely happy with either.
Additional Notes
- Default is
NOOP, and an empty composition returnsNOOPby identity, so a caller that registers nothing pays no virtual call per commit. Behaviour is unchanged unless something opts in. - One edit covers both runtimes.
integrations/java/iceberg-1.5/openhouse-java-runtimecompilesiceberg-1.2's sources, so the change ships to both without duplication. - Tests would go in
openhouse-java-itest, following the existing arrangement. - Question for maintainers:
CONTRIBUTING.mdsays API changes needdocs/specsupdated.docs/specslooks like the REST surface — does a new Java client interface fall under that requirement, or is it out of scope? - Happy to split this into smaller PRs if preferred — the types and the invocation point are separable.
- Dominant language
- Java
- Stars
- 406
- Forks
- 86
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 25
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from linkedin/openhouse
-
[BUG] Retention with backup fails for MONTH and YEAR time-partitioned tablesPossibly taken @0z5a claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 28/100
Maintainers usually reply within 2 days
-
Difficulty 5/5 Over a week Newbie friendliness 18/100
Maintainers usually reply within 2 days
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
Maintainers usually reply within 2 days
-
[BUG] Stale table creates enter update validation instead of returning a version conflictPossibly taken @menarguez claimed this 48 days ago. Open
Difficulty 3/5 1-2 days Newbie friendliness 85/100
Maintainers usually reply within 2 days
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
Maintainers usually reply within 2 days
All issues in linkedin/openhouse
Similar issues
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
molgenis/molgenis-emx2#7059 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
quarkiverse/quarkus-flow#1169 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
eclipse-4diac/4diac-ide#3211 ·
Maintainers usually reply within 1 day
-
needs-umair step-4
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Umairishuman/BuildScan3D#24 ·
-
👀 example 📱 iOS
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
hyochan/react-native-nitro-sound#871 ·
Maintainers usually reply within 2 days