opensearch-project/OpenSearch

How to decouple dedicate index storage by project?

Open

#9.602 aberto em 24 de ago. de 2023

Ver no GitHub
 (2 comments) (0 reactions) (0 assignees)Java (1.505 forks)batch import
Indexinghelp wanted

Métricas do repositório

Stars
 (8.123 stars)
Métricas de merge de PR
 (Mesclagem média 5d 9h) (266 fundiu PRs em 30d)

Description

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.2.0, running on VMs

Describe the issue: Hi , I push all applicative logs on 3 Opensearch nodes Everything is working perfectly fine for 1 year now.

We would need to push logs from our APIC Tools : but its 10 X more logs than the applicative logs.

We would like to be able to make these logs available in the same Opensearch Instance But without compromising our applicative indexes.

We would like to set the shards routing properly to fullfill our needs And we wonder how we can implement it properly.

In other words we wonder how we can set Opensearch Worker nodes to store exclusively

  • applicative shards ( master / replicas )
  • APIC shards ( master / replicas )

Is is possible to decouple Worker nodes

  • some nodes would be dedicated to Applicative shards storing
  • the others nodes would be dedicated to APIC shards storing

Should I set a different rack for each worker node in conf ( applicative rack or apic rack ) as follow : node.attr.rack_id: rack_applicative or node.attr.rack_id: rack_apic

And then set explicitly for any kind of indexes where he should be stored ? as follow : to force the index routing for a test index to be stored on applicative rack nodes for example : PUT test/_settings { "index.routing.allocation.require.rack": "rack_applicative" }

Is it a good solution ? is there an other way ?

Thank you for support

Guia do colaborador