microsoft/garnet

Easy Cluster mode on kubernetes

Open

#741 opened on Oct 21, 2024

View on GitHub
 (4 comments) (12 reactions) (0 assignees)C# (660 forks)batch import
enhancementhelp wanted

Repository metrics

Stars
 (11,827 stars)
PR merge metrics
 (Avg merge 5d 16h) (55 merged PRs in 30d)

Description

Feature request type

Kubernetes Operator, or easy config file setup for garnet

Is your feature request related to a problem? Please describe

Using cluster mode on kubernetes is impossible without an operator to meet the other nodes in a statefulset. Indeed, garnet uses the same cluster model as redis where stafulset/service (with clusterIP none) dns naming pattern is unknown at the config level.

Describe the solution you'd like

provide an operator that orchestrates cluster setup, recovery, migration based on a custom CRD that describes the cluster topology :

apiVersion: garnet.microsoft/v1
kind: GarnetCluster
metadata:
  name: garnet-cluster
spec:
  masterCount: 3
  replicasPerMaster: 1
  storage:
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
  • Changing masterCount should reshard and migrate data to resized masters
  • changing replicasPerMaster should replicate data

Describe alternatives you've considered

Allow graceful recovery of the cluster when one or many nodes restart using a dns naming pattern (would be nice) using a predefined cluster DNS names lists. Example : in garnet.conf

{
  "clusterMasterNames": ["garnet-0", "garnet-1", "garnet-3"],
  "clusterReplicaNames" : ["garnet-4", "garnet-5", "garnet-6"]
}
  • clusterReplicaNames vector should have a size that is multiple of clustesMasterNames
  • each cluster know who he is by getting it's hostname as kubernetes would provision it whensetting a statefulset
  • only static setup would be supported (no dynamic resharding, or dynamic replication)

Additional context

No response

Contributor guide