KVM agent deletes externally-managed (netplan/networkd) bridges and VLAN interfaces it did not create
還沒有人認領這個 Issue。
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 45/100
- Issue 類型
- 缺陷
- 描述清晰度
- 描述清楚
- 活躍度
- 活躍
研究方向
The issue is in BridgeVifDriver.deleteVnetBr() and the pattern matching for bridge ownership. Start by examining the Java source files for the KVM agent, particularly the network bridge management logic. Look at how bridges are created and tracked. The fix involves adding provenance tracking or a safer ownership check. Test by setting up a similar environment with netplan and verifying bridge deletion behavior.
由索引模型根據 Issue 內容生成。
描述
problem
When the last vif is unplugged from the public bridge, the agent deletes the bridge and its parent VLAN interface — even though CloudStack never created them.
BridgeVifDriver.deleteVnetBr() decides whether a bridge is CloudStack-owned solely by matching its name against a regex:
Pattern brNameRegex = Pattern.compile("^br(\\S+)-(\\d+)$");
This is intended to match bridges the agent created via generateVnetBrName() ("br" + pifName + "-" + vnetId). But the pattern matches any similarly-named bridge. There is no provenance tracking — the naming convention is the only ownership test.
A pre-existing, persistently-declared bridge named brbond0-103 (bond bond0, VLAN 103) matches exactly, so the agent treats it as its own and runs:
modifyvlan.sh -o delete -v 103 -p bond0 -b brbond0-103 -d true
which deletes both interfaces:
ip link delete $vlanDev type vlan # → ip link delete bond0.103
ip link set $vlanBr down
ip link delete $vlanBr type bridge # → ip link delete brbond0-103
netplan/networkd do not recreate these without netplan apply or a reboot, so the loss is permanent until manual intervention.
The failure is silent for as long as the agent keeps running. The public NIC is validated only as a one-time startup precondition in LibvirtComputingResource.configure():
if (pifs.get("public") == null) {
LOGGER.error("Failed to get public nic name");
throw new ConfigurationException("Failed to get public nic name");
}
There is no retry and no degraded mode. The running agent never re-checks — it merely logs, once per minute:
WARN [kvm.resource.LibvirtComputingResource] Failed to read the rx_bytes for brbond0-103
from /sys/class/net/brbond0-103/statistics/rx_bytes
java.io.FileNotFoundException: File '/sys/class/net/brbond0-103/statistics/rx_bytes' does not exist
so the host continues to report healthy. The damage only surfaces at the next agent restart — in our case 44 hours later — when the host fails to reconnect.
versions
ACS 4.22.1.0
Ubuntu 24.04
KVM hypervisor, Ubuntu 24.04, systemd-networkd + netplan.
The Public traffic type uses a persistently declared bridge, created at boot by netplan — not by CloudStack:
# /etc/netplan/50-cloud-init.yaml (abridged)
vlans:
bond0.103:
id: 103
link: bond0
mtu: 1500
bridges:
brbond0-103:
interfaces: [bond0.103]
mtu: 1500
parameters: {stp: false, forward-delay: 0}
agent.properties
public.network.device=brbond0-103
Zone config: Public traffic type has kvmnetworklabel=brbond0-103; public IP ranges are vlan://103.
OS / ENVIRONMENT
Ubuntu 24.04, kernel 6.11, systemd-networkd/netplan, bonded 802.3ad uplink
The steps to reproduce the bug
- On a KVM host, persistently declare (netplan/networkd/ifupdown) a VLAN interface
and bridge whose name matches ^br(\S+)-(\d+)$ — e.g. bond0.103 enslaved to
brbond0-103. - Set public.network.device=brbond0-103 and the zone's Public
kvmnetworklabel=brbond0-103, with a vlan://103 public IP range. - Start the agent; create an isolated network so a VR is placed on this host with
a nic on the public bridge. Confirm brbond0-103 has the VR vif as its only VM port. - Destroy that VR (or restart the network so the VR is recreated elsewhere), so the
last vif leaves the bridge. - Observe: bond0.103 and brbond0-103 are deleted from the running kernel.
- Restart cloudstack-agent.
EXPECTED RESULTS
The agent does not delete network interfaces it did not create. Bridges and VLAN
interfaces that are persistently declared by the host's network configuration are
left in place when the last vif is unplugged.
ACTUAL RESULTS
Step 5 — kernel logs the interfaces being removed (note "unregistering", i.e.
device deletion rather than link-down):
kernel: brbond0-103: port 1(bond0.103) entered disabled state
systemd-networkd: bond0.103: Link DOWN
kernel: bond0.103 (unregistering): left promiscuous mode
systemd-networkd: brbond0-103: Link DOWN
Step 6 — the agent then refuses to start:
ERROR [kvm.resource.LibvirtComputingResource] Failed to get public nic name
ERROR [cloud.agent.AgentShell] Unable to start agent:
javax.naming.ConfigurationException: Failed to get public nic name
at LibvirtComputingResource.configure(LibvirtComputingResource.java:1495)
at com.cloud.agent.Agent.<init>(Agent.java:239)
...
The host stays Disconnected until the interfaces are recreated by hand. Guest VMs
already running are unaffected (libvirt and the data plane are independent of the
agent), which is part of why the fault goes unnoticed.
What to do about it?
Impact is wider than it first appears. The trigger is "the last vif leaves the public bridge", not "the host is idle" — in our case the host had four VMs running throughout, none of them on the public bridge. Since only virtual routers attach to the public network, and VRs are among the most frequently destroyed/recreated VMs (network restart, offering change, VR upgrade, HA event), any host that transiently has no VR on the public network is exposed. The resulting breakage is invisible until the next agent restart, which in practice means it surfaces during a maintenance window.
Why "just name it differently" is not a workaround. The bridge name used at plug time is derived, not read from config — createVnetBr() → generateVnetBrName(_pifs.get(trafficLabel), vNetId) → "br" + "bond0" + "-" + "103". Renaming the declared bridge causes the agent to create a second bridge under the constructed name rather than using the renamed one. For a VLAN-tagged public network there is no configuration that avoids the matching name.
Suggested fixes (in rough order of preference):
Track provenance — only delete bridges the agent itself created (e.g. a marker, or state recorded at creation time), rather than inferring ownership from a name.
Failing that, an opt-out agent property, e.g. network.bridge.externally.managed=brbond0-103,..., consulted by deleteVnetBr() before deletion.
At minimum, never delete the parent VLAN interface. Even where the bridge is plausibly CloudStack's, bond0.103 belongs to the host's network configuration.
Two related observations, filed separately to keep this report focused:
modifyvlan.sh sets no MTU when creating interfaces, so an agent-created VLAN inherits the parent's (9000 on a jumbo-frame bond) rather than the intended value.
The ls /sys/class/net/
/brif "is the bridge in use?" check is not atomic with the deletion, so a concurrent plug can race it.
Happy to test a patch.
- 主要語言
- Java
- 星號
- 3.1k
- 分支
- 1.4k
- 平均合併
- 6 天 20 小時
- 30 天內合併 PR
- 27
貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
apache/cloudstack 的其他 Issue
-
bug
難度 1/5 1 小時以內 新手友好度 90/100
apache/cloudstack#14222 ·
-
bug component:kubernetes
難度 1/5 1 小時以內 新手友好度 88/100
apache/cloudstack#14180 ·
-
bug component:projects component:UI
難度 1/5 1 小時以內 新手友好度 88/100
apache/cloudstack#14070 · 5 則留言 ·
-
component:backup
難度 2/5 1-3 小時 新手友好度 76/100
apache/cloudstack#14013 ·
-
KVM agent fails to connect to Ceph RBD storage pool after upgrading Ceph client to Tentacle 20.2.4 未關閉bug component:ceph
難度 2/5 1-3 小時 新手友好度 78/100
apache/cloudstack#13989 · 3 則留言 ·
查看 apache/cloudstack 的全部 Issue
相似的 Issue
-
area/plugin
難度 2/5 1-3 小時 新手友好度 75/100
kestra-io/plugin-kestra#190 ·
-
難度 2/5 1-3 小時 新手友好度 70/100
google-ai-edge/LiteRT-LM#3739 ·
-
bug
難度 2/5 1-3 小時 新手友好度 75/100
-
難度 2/5 1-3 小時 新手友好度 75/100
integra-team-red/meet-map#249 ·
-
[Studio][Bug] Cancelled create-user dialog keeps the password and admin switch for the next attempt 未關閉
難度 2/5 1-3 小時 新手友好度 75/100
apache/rocketmq-dashboard#5064 ·