Restarting a KVM VMs after its host fails when the management server is also unreachable
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Anfängerfreundlichkeit
- 25/100
- Issue-Typ
- Feature
- Klarheit
- Größtenteils klar
- Aktivitätsstatus
- Aktiv
- Bereich
- backend, cloud, distributed-systems, infrastructure
Rechercherichtung
The issue describes a distributed systems problem requiring a new cluster mechanism for VM HA. Start by reviewing the existing KVM agent code and the linked PR #13589 and issue #14206. Understand how CloudStack currently handles host failures and VM restarts via the management server. Research Corosync/Pacemaker and LINSTOR/drbd-reactor integration possibilities. The goal is to design a system where agents can autonomously restart VMs during a management server outage.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
The required feature described as a wish
Today if a KVM host fails and the management server and/or database is unreachable at the same moment (a remote site outage, a network partition, or the management server itself running as a guest on the failing host) CloudStack cannot restart the affected VMs. Every HA and DRS decision path runs inside the management server process and depends on the database.
I propose to add a small, storage-aware cluster mechanism that can restart a VM on a surviving host using its own quorum and fencing, and hands control back cleanly once the management server returns. This might require the Agents to actually keep shadow copy of each of the VMs configuration (of just enough per-VM data to rebuild the libvirt domain: disk identifiers, NIC MACs, CPU/memory, bridge/VLAN mapping). Possibly the agents can exchange information with each other and in case one Agent looses connectivity to management server act as a proxy towards the management server?
This might be an addition/similar or even fix to the https://github.com/apache/cloudstack/pull/13589 , https://github.com/apache/cloudstack/issues/14206 , or rely on some other mechanisms as surviving hosts must agree "host A (and possibly the management server) is gone" without a central arbiter and must not false-positive on an ordinary network hiccup,
Options I see as viable:
- Generic - Corosync for membership, Pacemaker for orchestration, IPMI/BMC STONITH for fencing.
- Storage specific - LINSTOR + drbd-reactor, CloudStack already ships a maintained LINSTOR primary storage plugin whose KVM-side driver talks straight to the LINSTOR controller, bypassing the management server for volume attach, and drbd quorum enforces split-brain protection, so it might be a nice addition.
- Vorherrschende Sprache
- Java
- Sterne
- 3.1k
- Forks
- 1.4k
- Ø Merge
- 6 T. 20 Std.
- Gemergte PRs (30 T.)
- 27
Beitragsleitfaden
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus apache/cloudstack
-
bug
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 90/100
apache/cloudstack#14222 ·
-
create-kubernetes-binaries-iso.sh builds the ISO without setting a volume ID on EL8 based os's Offenbug component:kubernetes
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 88/100
apache/cloudstack#14180 ·
-
bug component:projects component:UI
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 88/100
apache/cloudstack#14070 · 5 Kommentare ·
-
component:backup
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 76/100
apache/cloudstack#14013 ·
-
KVM agent fails to connect to Ceph RBD storage pool after upgrading Ceph client to Tentacle 20.2.4 Offenbug component:ceph
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
apache/cloudstack#13989 · 3 Kommentare ·
Alle Issues in apache/cloudstack
Ähnliche Issues
-
executions.Query — startDate and timeRange filters are sent with inverted comparison operators Offenarea/plugin
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
kestra-io/plugin-kestra#190 ·
-
litertlm-android AAR ships no consumer ProGuard rules → "mid == null" SIGABRT in minified apps Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 70/100
google-ai-edge/LiteRT-LM#3739 ·
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
integra-team-red/meet-map#249 ·
-
[Studio][Bug] Cancelled create-user dialog keeps the password and admin switch for the next attempt Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
apache/rocketmq-dashboard#5064 ·
-
Consent portal: creating a duplicate Purpose shows a generic error instead of "already exists" Offen
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
wso2/dpdp-accelerator#287 ·