KVM: VM that gets a NIC on a config drive network keeps a data volume on a reserved device id and cannot start

Open
#14,513 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
java

Research direction

Trace volume device ID assignment and config drive or CD-ROM reservations in the KVM VM start path. Reproduce the reported q35 flow: deploy without config drive, attach three data volumes, add a NIC on the config drive network, then start with vm.iso.max.count=2. Confirm completion by showing the VM starts and no data volume shares a device ID with a reserved ISO or CD-ROM.

Written by the indexing model from the issue text.

Description

bug component:kvm component:networking component:storage
Problem

The device ids reserved for the CD-ROMs and the config drive ISO depend on whether the VM uses config drive. A VM without config drive can have a data volume on a device id that becomes reserved when a NIC on a config drive network is added later. The volume keeps its device id, and on q35 its target is the same as a CD-ROM, so the VM cannot start (XML error: target '...' duplicated).

Not caused by the device id fix: the volume was given that id while the VM did not use config drive, and on 4.23.0.0 the same flow fails with the 4.23 device ids (1, 2, 4) already with vm.iso.max.count=1, and with data volumes on 1, 2, 5 it fails with vm.iso.max.count=2 (target 'sde' duplicated between the empty CD-ROM and the config drive ISO). The notes of both fixes say that volumes already on a reserved device id are not changed.

Versions

Fails before the device id fixes:

  • 4.22.1.x: not run for this flow (the 4.22 device id fix does not change the device ids of VMs without config drive, so the 3rd volume is on 4 there too).
  • 4.23.0.0: data volume on deviceid 4 (the 3rd volume on 4.23): the VM cannot start (target 'sde' duplicated with the config drive ISO), with vm.iso.max.count=1 and 2. Data volumes on 1, 2, 5: starts with vm.iso.max.count=1, cannot start with 2.

Still fails after the device id fixes (volumes already on an id that becomes reserved are not changed):

  • 4.22 build with the device id fix (4.22.2.0-SNAPSHOT): data volume on deviceid 4, the VM cannot start (target 'sde' duplicated with the config drive ISO), default vm.iso.max.count=1.
  • main (24.0.0-SNAPSHOT): the 3rd volume is now on deviceid 5. The VM starts with vm.iso.max.count=1 (better than 4.23.0.0) and cannot start with vm.iso.max.count=2 (target 'sdf' duplicated with ISO 2 or its empty CD-ROM).
Environment:

KVM on OL 9, qemu-kvm 7.2.0, libvirt 9.0.0, machine type pc-q35-rhel9.2.0 (template settings rootDiskController=scsi, kvm.guest.os.machine.type=pc-q35-rhel9.2.0, UEFI=LEGACY), host.cdrom.max.count 2. NFS primary storage.

The steps to reproduce the bug
  1. Create two L2 networks: N1 with DefaultL2NetworkOffering, N2 with DefaultL2NetworkOfferingConfigDrive.
  2. (main only) Set vm.iso.max.count to 2.
  3. Deploy a q35 scsi VM on N1 (stopped) and attach 3 data volumes. They get device ids 1, 2, 4 (4.22) or 1, 2, 5 (main).
  4. Add a NIC on N2 to the VM (addNicToVirtualMachine).
  5. Start the VM. It fails with 530 "Unable to orchestrate the start of VM instance", and the management server log has "Unable to start VM on Host ... XML error: target 'sde' (4.22) or 'sdf' (main) duplicated".
  6. On 4.22: detach the 3rd volume and attach it again. It gets deviceid 5 and the VM starts. (Not tried on main for this flow.)
What to do about it?

Expected: a VM that starts using config drive after a NIC is added starts, with its data volumes not on the device ids of the CD-ROMs and the config drive ISO.

Dominant language
Java
Stars
3.1k
Forks
1.4k
Avg merge
7d 4h
Merged PRs (30d)
13

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/cloudstack

All issues in apache/cloudstack

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.