opensearch-project/OpenSearch

[Feature Request] [Remote Store] Make metadata store independent of data store

Open

#12,919 建立於 2024年3月26日

在 GitHub 查看
 (1 留言) (1 反應) (0 負責人)Java (1,505 fork)batch import
Storage:Remoteenhancementgood first issue

倉庫指標

Star
 (8,123 star)
PR 合併指標
 (平均合併 5天 9小時) (30 天內合併 266 個 PR)

描述

Is your feature request related to a problem? Please describe

  • Currently, when we configure remote store for a cluster, both data and metadata (of segment, translog and cluster state) is stored in the same store.
  • But the access pattern and consistency requirements for data and metadata are different.
  • For example, data files are uploaded and downloaded by providing a filename whereas metadata is fetched based on certain characteristics such as recency.
  • Due to these differences, it is possible that metadata can be stored in a different data store (for example, a key-value store) as we store the data files in a blob store.

Describe the solution you'd like

The ideal solution should allow configuring a different remote store for metadata.

Related component

Storage:Remote

Describe alternatives you've considered

No response

Additional context

No response

貢獻者指南