[Bug] RemoteTableQuery keySerializer should not include partition keys

Open Beginner friendly
#8,145 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
java
Domain
databases

Research direction

Start in org.apache.paimon.flink.query.RemoteTableQuery.lookup and trace its use of InternalRowSerializer.toBinaryRow. Reproduce a remote lookup join against a partitioned primary-key table where partition and trimmed key types differ, then verify serialization no longer reads partition fields and the ClassCastException is gone.

Written by the indexing model from the issue text.

Description

bug
Search before asking
  • I searched in the issues and found nothing similar.
Paimon version

When doing a lookup join to the remote query service (for primary key tables), we pass the partition keys, a bucket number, and a row to join to the existing table. Right now, serialization of the row fails because the serializer expected by the remote query service client assumes that the row includes partition fields, but we do not pass them (this is the same in LocalTableQuery, not a bug).

Let's change the key serializer here to avoid looking for partition keys.

Compute Engine

Flink

Minimal reproduce step

Use the remote query service to perform a lookup join from a flink row to a paimon table. The table must be partitioned, and the partitioned key and trimmed key must have different types (e.g. int vs. string). I'll include a test case in the PR that trips it up.

What doesn't meet your expectations?
java.lang.ClassCastException: class org.apache.paimon.data.BinaryString cannot be cast to class java.lang.Integer                                                                                   
        at org.apache.paimon.data.GenericRow.getInt(GenericRow.java:152)
        at org.apache.paimon.data.serializer.InternalRowSerializer.toBinaryRow(InternalRowSerializer.java:123)
        at org.apache.paimon.flink.query.RemoteTableQuery.lookup(RemoteTableQuery.java:73)
Anything else?

No response

Are you willing to submit a PR?
  • I'm willing to submit a PR!
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 14h
Merged PRs (30d)
468

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/paimon

All issues in apache/paimon

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.