apache/hudi

Limit the amount of partitions considered for GlobalBloomIndex

Open

#14484 opened on Nov 30, 2025

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Java (4,823 stars) (2,431 forks)batch import
area:indexfrom-jiragood first issuepriority:hightype:improvement

Description

Currently, global bloom index will check inputs against files in all partitions.. In lot of cases, the user may know a range of partitions actually impacted from updates clearly (e.g upstream system drops updates older than a year, ... ).. In such a scenario,it may make sense to support an option for Global bloom to control how many partitions you want to match against, to gain performance.

JIRA info

Contributor guide

Tech stack
java
Domain
backenddataperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic understanding of Hudi's indexingJava programming knowledge
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
Investigate GlobalBloomIndex implementation in the Hudi codebase (e.g., hudi client/hudi client common/src/main/java/org/apache/hudi/index/bloom/). Understand how partitions are currently considered and where a configuration option for limiting partitions could be added. Look for existing configuration parameters for BloomIndex. Discuss in the issue if maintainers have suggestions.