[BUG] Changing a GPU desktop's instance type across GPU families leaves the session in ERROR

Inizia con gpu_drivers.sh, soprattutto con il controllo della presenza del driver e il passaggio nvidia-xconfig, quindi segui la modifica del tipo di istanza e il flusso di ripresa in virtual_desktop_controller_utils.py e virtual_desktop_server_utils.py. Riproduci una modifica da g4dn a g6 o da g6 a g4dn e ispeziona il file /etc/X11/xorg.conf generato. È completato quando un desktop esistente riprende sulla nuova famiglia di GPU e raggiunge READY con una configurazione X corrispondente alla GPU attiva.

Changing an existing GPU virtual desktop's instance type to a different GPU family via the RES UI, for example g4dn/T4 to g6/L4, leaves the session in ERROR after RESUMING/INITIALIZING. The instance starts and the GPU driver and DCV server come
up, but the DCV console session's X server never initializes because /etc/X11/xorg.conf is still pinned to the create-time GPU's PCI BusID. The host bootstrap does re-run on every boot, but the X-config step self-skips when a driver is already present, so the stale BusID is never refreshed. Fresh desktops created directly on the target type reach READY, and CPU-only type changes reach READY, so the issue is specific to GPU-family changes on existing desktops. Reproduced both g4dn to g6 and g6 to g4dn.

Environment

• RES 2026.06
• Base OS: Ubuntu 24.04 and Amazon Linux 2023. The Linux GPU path is OS-agnostic; see Root cause.
• GPU instances: g4dn (T4), g6 (L4); NVIDIA GRID driver 595.71.05 from s3://ec2-linux-nvidia-drivers/latest/
• Region: us-east-1

Steps to reproduce

1. Create a RES Linux GPU desktop on g4dn.2xlarge. It reaches READY.
2. Stop the desktop.
3. Change its instance type to a different GPU family, e.g. g6.2xlarge.
4. Resume the desktop.

Expected

The desktop resumes on the new GPU type and reaches READY, as a fresh create on that type does.

Actual

The session goes RESUMING to INITIALIZING to ERROR. On the host the instance is running, nvidia-smi reports the new GPU and driver, and dcvserver is active, but no X server is running and the DCV console session never initializes.

Root cause

RES bakes the GPU family into the host bootstrap's install.sh -g <family> argument at create time, computed from the create-time instance type. The Linux userdata cloud-config runs on every boot via cloud_final_modules: [scripts-user, always],
and an instance-type change (ModifyInstanceAttribute) followed by resume (StartInstances) does not rebuild the userdata, so on resume the instance re-runs the create-time bootstrap.

For a GPU-family change the re-run does not refresh the GPU X configuration. install_nvidia_grid_drivers in gpu_drivers.sh short-circuits on which nvidia-smi — the pre-existing GRID driver satisfies the check — and returns before
running nvidia-xconfig --preserve-busid, the step that writes /etc/X11/xorg.conf pinned to the GPU's PCI BusID. So the xorg.conf generated for the original GPU is never regenerated for the new GPU: X cannot start on the new
GPU's BusID, the DCV console session never initializes, and RES marks the session ERROR.

A fresh create reaches READY because no driver is present, so the guard passes and nvidia-xconfig runs for the correct GPU. A CPU-only change reaches READY because no GPU/X stack is involved.

Code (RES 2026.06):

• GPU family baked into install.sh -g at create time: virtual_desktop_controller_utils.py L93-L97, L152
• Userdata cloud-config runs on every boot: bootstrap_userdata_linux.sh.jinja2 L11-L12
• Type change performs only ModifyInstanceAttribute; resume performs only StartInstances; userdata is not rebuilt: virtual_desktop_controller_utils.py L727-L742, virtual_desktop_server_utils.py L100-L113
• X-config step self-skips on which nvidia-smi, so nvidia-xconfig --preserve-busid never re-runs after a GPU change; skip at L34-L36, nvidia-xconfig at L63: gpu_drivers.sh L33-L64

gpu_drivers.sh branches by instance family, not OS, and is shared by Ubuntu, Amazon Linux, and RHEL through the DCV bootstrap, so all Linux distros are expected to be affected.

Evidence

Same AMI, driver, and kernel across all cases.

• g4dn -> g6: ERROR. Instance running, nvidia-smi shows the L4, dcvserver active, but no "First frame captured" and dcvgltestreturns "Unable to open display". Started immediately (noInsufficientInstanceCapacity`) and DCV-ready ~2.4 min after start, still ERROR — rules out capacity and readiness-timeout.
• g6 -> g4dn: ERROR. xorg.conf = BusID "PCI:49:0:0" / BoardName "NVIDIA L4"; live GPU Tesla T4 at 00000000:00:1E.0 (bus 0x1E) — BusID mismatch. pgrep Xorg/Xdcv empty, no /var/log/Xorg.0.log; X never started.
• Fresh create on g4dn -> READY. Fresh create on g6 -> READY. Isolates the failure to the type-change path.
• CPU-only c8g -> c9g (Amazon Linux 2023) -> READY. Scopes the bug to GPU desktops.

Suggested fix

Refresh the GPU X configuration when the GPU changes. Options:

• On boot, detect a GPU change (current PCI BusID or GPU model differs from what /etc/X11/xorg.conf was generated for) and e-run nvidia-xconfig --preserve-busid --enable-all-gpus even when the driver is already installed. Do not gate the X-config step behind the which nvidia-smi driver-presence check.
• Or force host GPU/DCV/X re-provisioning on the next resume after a GPU-family change.
• Or block cross-GPU-family instance-type changes on an existing desktop and direct the user to recreate on the target type.

Workaround

Recover an already-ERROR desktop in place, validated. On the host as root:

rm -f /etc/X11/xorg.conf
nvidia-xconfig --enable-all-gpus
# confirm the BusID now matches `nvidia-smi --query-gpu=pci.bus_id`

Then use the RES Reboot action, the only action available from ERROR. It sets the session to PROVISIONING and re-creates the DCV session. The corrected xorg.conf persists on the EBS root volume and the which nvidia-smi skip guard leaves it
untouched, so X starts on the correct GPU and the session reaches READY. Verified: the desktop returned to READY after the reboot and remained READY across a subsequent stop/start.

Alternatively, recreate the desktop on the target instance type. A fresh create runs the full create-time bootstrap and configures the GPU display stack for the correct GPU. User data on shared home storage is preserved.