opensearch-project/OpenSearch

[Feature Request] [Remote Store] Make metadata store independent of data store

Open

#12,919 创建于 2024年3月26日

在 GitHub 查看
 (1 评论) (1 反应) (0 负责人)Java (1,505 fork)batch import
Storage:Remoteenhancementgood first issue

仓库指标

Star
 (8,123 star)
PR 合并指标
 (平均合并 5天 9小时) (30 天内合并 266 个 PR)

描述

Is your feature request related to a problem? Please describe

  • Currently, when we configure remote store for a cluster, both data and metadata (of segment, translog and cluster state) is stored in the same store.
  • But the access pattern and consistency requirements for data and metadata are different.
  • For example, data files are uploaded and downloaded by providing a filename whereas metadata is fetched based on certain characteristics such as recency.
  • Due to these differences, it is possible that metadata can be stored in a different data store (for example, a key-value store) as we store the data files in a blob store.

Describe the solution you'd like

The ideal solution should allow configuring a different remote store for metadata.

Related component

Storage:Remote

Describe alternatives you've considered

No response

Additional context

No response

贡献者指南