opensearch-project/OpenSearch

[Remote Store] Remove redundant uploads while restoring data from snapshot

Open

#11,044 opened on Nov 1, 2023

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Java (1,505 forks)batch import
Storage:DurabilityStorage:Remotebuggood first issue

Repository metrics

Stars
 (8,123 stars)
PR merge metrics
 (Avg merge 5d 9h) (266 merged PRs in 30d)

Description

Describe the bug

If we restore a remote store backed index from an interop enabled repository, we are uploading all the data in remote. This is not needed when there is preexisting data in remote store. There can be preexisting data , when the index is closed and we try to restore it back in time from a snapshot.

https://github.com/opensearch-project/OpenSearch/blob/aa0fddb50d6150330718ac0af23b72bd1598ae4a/server/src/main/java/org/opensearch/index/shard/IndexShard.java#L4916-L4922

Given that we have enabled upload during snapshot restore, there is no need to copy the segment files manually to remoteDirectory .

Contributor guide

Research direction
Examine the remote store upload logic during snapshot restore in IndexShard.java around line 4916. Identify the condition where preexisting data exists and ensure uploads are skipped.
Tech stack
java
Domain
backend
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
JavaGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60