First-class support (or an officially supported recipe) for multi-Kubernetes / multi-region DR
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 18/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Da chiarire
- Stato di attività
- Attiva
- Stack tecnologico
- go, kubernetes
- Ambito
- documentation, infrastructure
Direzione di ricerca
This is a design question, not a code change yet. Start with the recovery-architecture doc linked in the issue and the CHI settings it mentions, zookeeper.nodes and remote_servers, to see what the operator already supports across clusters. A first useful step is a scoping discussion with maintainers on whether a docs and test checklist for two-CHI primary/warm-standby is accepted. Done means an agreed scope, not an implementation.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
We run ClickHouse via clickhouse-operator (currently looking at 0.27.x) and need a clear story for disaster recovery across Kubernetes clusters, often in different regions: primary site + warm standby (or equivalent), with a tested failover path.
Today the operator is excellent inside a single Kubernetes cluster, but multi-site DR still looks like DIY. We’d like this to become a first-class, documented, and tested capability — or at least an explicitly supported topology with a maintained guide tied to operator releases.
What we need
Something teams can rely on in production, for example:
- Documented supported topology for primary / warm-standby (or another Altinity-recommended shape) across two Kubernetes clusters / regions, including Keeper placement (voting members vs observers), naming/
{replica}rules, and failover / failback steps. - Operator-level help where it makes sense — even if a full “Federated CHI” is a bigger project. Examples: clearer cross-site guidance in-tree, validation/warnings, helpers for
remote_servers/ macros, or a thin multi-cluster controller later. - Soak / failover testing so this isn’t only a blog/webinar recipe.
We’re fine doing networking ourselves (peering, DNS, service exposure). The gap is knowing what’s supported vs best-effort, and not inventing a one-off design that breaks on the next operator upgrade.
What we already understand (not asking for this)
- CHI is namespaced and the operator reconciles inside one Kubernetes cluster — that matches what we’ve seen in the CRD and in your materials.
- Multiple entries in
spec.configuration.clusters[]in one CHI are a same-cluster layout, not multi-region DR. - A real CHK is one Raft ensemble; we won’t treat extra CHK
clusters[]examples as multi-site Keeper HA. - Cross-region setups are possible today with two operators/CHIs, shared Keeper via
zookeeper.nodes, and a lot of network/DNS work (as in your blog and #1380).
Your DR webinar also called this out directly:
The operator operates within a single Kubernetes cluster, so multi-cluster deployments require something sitting outside the operator itself. […] eBay […] implemented a “Federated ClickHouseInstallation” […] We did not build it yet because the multi-region DR scenario is rare enough that it has not risen to the top of the priority list. But we are totally open to collaborating on it.
We’re essentially asking to raise the priority of that gap (docs + tests first is already valuable; federation later if it makes sense).
References
- https://docs.altinity.com/operationsguide/availability-and-recovery/recovery-architecture/
- https://altinity.com/blog/setting-up-cross-region-clickhouse-replication-in-kubernetes
- https://altinity.com/webinarspage/clickhouse-disaster-recovery-tips-and-tricks-to-avoid-trouble-in-paradise
- https://github.com/Altinity/clickhouse-operator/issues/1380
- https://github.com/Altinity/clickhouse-operator/issues/1258
Ask
- Is multi-Kubernetes / multi-region DR on the operator roadmap?
- Short of a federated controller, would you accept / help maintain an official in-repo guide + test checklist for the primary/warm-standby pattern with two CHIs?
- Lingua principale
- Go
- Stelle
- 2.6k
- Fork
- 579
- Merge medio
- 21h 2m
- PR unite (30g)
- 4
Preparare l'ambiente
- Include un Dockerfile o un file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di Altinity/clickhouse-operator
-
Counter metrics are missing the conventional _total suffixForse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
Altinity/clickhouse-operator#2093 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
Altinity/clickhouse-operator#2106 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
Altinity/clickhouse-operator#2098 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
Altinity/clickhouse-operator#2096 ·
I maintainer di solito rispondono entro 1 giorno
-
CHI reconcile aborts (and is never retried) when the API server briefly rejects owner-referenced objects right after the CRD is created (OpenShift OwnerReferencesPermissionEnforcement)Forse già presa Una pull request collegata a questa issue è aperta o già unita. Aperta
Difficoltà 3/5 1-2 giorni Idoneità per principianti 72/100
Altinity/clickhouse-operator#2094 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di Altinity/clickhouse-operator
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
QuantumNous/new-api#7839 ·
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
puppe1990/amarra-cais#355 ·
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
Gentleman-Programming/engram#1899 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 88/100
I maintainer di solito rispondono entro 1 giorno