apache/pulsar

Make the autoSkipNonRecoverableData can work for cursor ledger

Open

#17,220 opened on Aug 23, 2022

View on GitHub
 (4 comments) (0 reactions) (1 assignee)Java (13,697 stars) (3,485 forks)batch import
Stalehelp wanted

Description

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

After we enabled autoSkipNonRecoverableData, the topic will skip the non-recoverable ledger which will lose some data, but the topic is able to load and continues to provide messaging service. This usually happens when multiple bookie replicas are corrupted, or the Ledger has been deleted This usually happens when multiple bookie copies are corrupted

But the Ledger used by the cursor is not able to be skipped even if autoSkipNonRecoverableData=true

Solution

Make the cursor can skip the non-recoverable Ledger if autoSkipNonRecoverableData=true

Alternatives

No

Anything else?

No

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Contributor guide

Tech stack
java
Domain
backenddatabase
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Understanding of Apache Pulsar cursor managementKnowledge of ledger recovery in Pulsar
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
The issue requires extending the existing autoSkipNonRecoverableData functionality, currently applied to topic ledgers, to cursor ledgers. The implementation likely resides in the managed ledger or cursor components of Apache Pulsar. Developers should examine the code path where autoSkipNonRecoverableData is checked for topic ledgers and replicate similar logic for cursor ledgers. The assignee (if any) or comments may provide additional context. Look for files such as ManagedLedgerImpl.java or similar classes handling cursor recovery.