Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

InMemorySetState.retract NPEs on an absent secondary-index key in MEMORY-cache lookup join

Open Beginner friendly
  • Is something wrong?
#10,279 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
84/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
java
Domain
databases

Research direction

Start with InMemorySetState.retract and compare its absent-key behavior with LocalKvSetState.retract, then trace the calls from SecondaryIndexLookupTable.refreshRow. Reproduce the MEMORY-cache lookup join with a filtered-out row followed by a retract, and consider the issue done when that sequence no longer throws an NPE and matches the disk-backed behavior.

Written by the indexing model from the issue text.

Description

Search before asking
  • I searched in the issues and found no similar issues.
Paimon version

master (1.5-SNAPSHOT)

Compute Engine

Flink (secondary-index lookup join with lookup.cache = MEMORY).

Minimal reproduce step
  1. Run a lookup join whose join key is not the table's primary key (a secondary-index lookup), with lookup.cache = MEMORY and a lookup filter (predicate).
  2. Feed a changelog where a row that does not pass the predicate is later deleted (a -D / -U for that key).
What doesn't meet your expectations?

The lookup refresh crashes with a NullPointerException. SecondaryIndexLookupTable.refreshRow only adds a row to the index state when the predicate passes (on +I / +U), but retracts unconditionally on -D / -U. So a row that was filtered out was never added, and its later retract reaches InMemorySetState.retract, which does values.get(secKey).remove(...) with no null check and throws. The disk-backed sibling LocalKvSetState.retract already treats an absent key as a no-op, so the same join crashes only in MEMORY cache mode.

Anything else?

This is a behavior inconsistency between the two SetState implementations; MEMORY mode should tolerate the absent-key retract like the disk-backed one.

Are you willing to submit a PR?
  • I'm willing to submit a PR!
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
502

Getting set up

We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/paimon

All issues in apache/paimon

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.