opensearch-project/OpenSearch

How to decouple dedicate index storage by project?

Open

#9602 aperta il 24 ago 2023

Vedi su GitHub
 (2 commenti) (0 reazioni) (0 assegnatari)Java (1505 fork)batch import
Indexinghelp wanted

Metriche repository

Star
 (8123 star)
Metriche merge PR
 (Merge medio 5g 9h) (266 PR mergiate in 30 g)

Descrizione

Versions (relevant - OpenSearch/Dashboard/Server OS/Browser): 2.2.0, running on VMs

Describe the issue: Hi , I push all applicative logs on 3 Opensearch nodes Everything is working perfectly fine for 1 year now.

We would need to push logs from our APIC Tools : but its 10 X more logs than the applicative logs.

We would like to be able to make these logs available in the same Opensearch Instance But without compromising our applicative indexes.

We would like to set the shards routing properly to fullfill our needs And we wonder how we can implement it properly.

In other words we wonder how we can set Opensearch Worker nodes to store exclusively

  • applicative shards ( master / replicas )
  • APIC shards ( master / replicas )

Is is possible to decouple Worker nodes

  • some nodes would be dedicated to Applicative shards storing
  • the others nodes would be dedicated to APIC shards storing

Should I set a different rack for each worker node in conf ( applicative rack or apic rack ) as follow : node.attr.rack_id: rack_applicative or node.attr.rack_id: rack_apic

And then set explicitly for any kind of indexes where he should be stored ? as follow : to force the index routing for a test index to be stored on applicative rack nodes for example : PUT test/_settings { "index.routing.allocation.require.rack": "rack_applicative" }

Is it a good solution ? is there an other way ?

Thank you for support

Guida contributor