opensearch-project/OpenSearch

Skip metadata upload if no new segment files to upload during refresh for remote store clusters

Open

#17,530 建立於 2025年3月6日

在 GitHub 查看
 (10 留言) (0 反應) (1 負責人)Java (8,123 star) (1,505 fork)batch import
Storage:Remoteenhancementgood first issue

描述

Is your feature request related to a problem? Please describe

In the refresh flow of remote store indexes, we determine the files to be uploaded, upload them and then upload the associated metadata. This, however, uploads the metadata even when there is no new files to upload. This can be optimised and metadata upload can be skipped for such cases.

https://github.com/opensearch-project/OpenSearch/blob/3966ed93beb2ff13854d2a6e1ff51dfa32295830/server/src/main/java/org/opensearch/index/shard/RemoteStoreRefreshListener.java#L266C21-L292

Describe the solution you'd like

Optimise the segments upload flow to skip metadata upload when there are no newer segment files.

Related component

No response

Describe alternatives you've considered

No response

Additional context

No response

貢獻者指南

技術棧
javarest api
領域
backendperformance
議題類型
feature
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
1-2 days
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
blocked
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
clear
前置要求
JavaOpenSearch internalsremote store concepts
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
45
研究方向
Examine RemoteStoreRefreshListener.java, particularly the refresh method around line 266 where segment file upload decisions are made. Investigate how to conditionally skip metadata upload when no new segment files are present. Review the 10 comments on the issue for any proposed approaches or maintainer guidance. Consider the existing upload flow and where metadata is sent to remote store clusters.