opensearch-project/OpenSearch

How to decouple dedicate index storage by project?

Open

#9,602 创建于 2023年8月24日

在 GitHub 查看
 (2 评论) (0 反应) (0 负责人)Java (1,505 fork)batch import
Indexinghelp wanted

仓库指标

Star
 (8,123 star)
PR 合并指标
 (平均合并 5天 9小时) (30 天内合并 266 个 PR)

描述

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.2.0, running on VMs

Describe the issue: Hi , I push all applicative logs on 3 Opensearch nodes Everything is working perfectly fine for 1 year now.

We would need to push logs from our APIC Tools : but its 10 X more logs than the applicative logs.

We would like to be able to make these logs available in the same Opensearch Instance But without compromising our applicative indexes.

We would like to set the shards routing properly to fullfill our needs And we wonder how we can implement it properly.

In other words we wonder how we can set Opensearch Worker nodes to store exclusively

  • applicative shards ( master / replicas )
  • APIC shards ( master / replicas )

Is is possible to decouple Worker nodes

  • some nodes would be dedicated to Applicative shards storing
  • the others nodes would be dedicated to APIC shards storing

Should I set a different rack for each worker node in conf ( applicative rack or apic rack ) as follow : node.attr.rack_id: rack_applicative or node.attr.rack_id: rack_apic

And then set explicitly for any kind of indexes where he should be stored ? as follow : to force the index routing for a test index to be stored on applicative rack nodes for example : PUT test/_settings { "index.routing.allocation.require.rack": "rack_applicative" }

Is it a good solution ? is there an other way ?

Thank you for support

贡献者指南