[Bug]: Windows NEM guest restart hangs with an accepted TRPM_NMI pending (candidate patch)

Read src/VBox/VMM/VMMR3/target-x86/NEMR3Native-win-x86.cpp at NEMR3RunGC, then trace the TRPM and IEM event handling in EMR3.cpp and the native NEM run path. Review whether an accepted TRPM event needs delivery before WHP execution, including state imports and status propagation. Done means a reviewed fix builds and passes five consecutive ordinary guest restarts at three active vCPUs with fresh boot records and guest readiness after each.

Version

7.2.12 r174389 (the captured runtime). The v7.2.22 and current main source paths were also inspected; those versions have not been tested live here.

Host OS Type

Windows

Host OS name + version

Windows 11 Pro, 10.0.26200.9550 in the captured VBox.log. Intel Core i7-13700F, 24 logical host processors, approximately 32 GiB RAM. Hyper-V is active; VirtualBox uses the NEM/WHP backend.

Host Architecture

x86-64

Guest OS Type

Windows

Guest Architecture

x86-64

Guest OS name + version

Windows 11, build 26100 (Guest Additions logs report 10.0.26100.4349); ntoskrnl file version 10.0.26100.4343.

Component

Guest SMP / VMM (Windows NEM)

What happened?

Repeated normal guest-initiated restarts can stop before the VM reset. In the captured failed run, one target vCPU was idle with a pending TRPM event of type 3 (TRPM_NMI), while the NMI force flag was already cleared.

A candidate patch is included below. It has not been built as a complete VMM or validated in a live VM. We are asking for review and, if the approach is suitable, a normal Oracle-signed Windows x64 test build so that we can perform the real acceptance test without changing host security settings.

The existing VM used 3 vCPUs, 4096 MiB RAM, EFI64, PIIX3, VirtualBox software APIC, and HPET enabled for this capture. A development storage filter and service were loaded in the guest. This is not an established clean-guest minimum reproduction, and the proposed hypervisor cause should be treated as a candidate rather than proof that guest drivers cannot contribute.

Expected: each ordinary Windows restart completes, all active processors participate, and a fresh guest boot becomes observable.

Observed: three restarts completed in approximately 102, 108 and 100 seconds; the fourth failed to reach a VM reset within 242 seconds. The VM reset-transition count remained at 3. A later forced power-off was recovery only and was not counted as a successful restart.

How can we reproduce this?

1. On a Windows host with Hyper-V active, run an existing Windows 11 guest using the Windows NEM/WHP backend, with the configuration above.
2. Issue an ordinary guest-side shutdown /r /t 10.
3. Wait for a fresh Windows boot record and guest readiness, then repeat on the same running VM. Do not replace the guest restart with a VirtualBox reset.
4. If it hangs, use the public VBoxManage debugvm <vm> info cpumguest, info trpmevent, info fflags, and APIC diagnostics to inspect the halted processors.
5. In the captured failure, vCPU2 was the restart initiator, vCPU0 had acknowledged it, and vCPU1 remained idle with the accepted NMI still in TRPM.

This is intermittent: a short passing sequence is not sufficient evidence of a fix. We do not claim that every earlier SMP hang has this same cause.

Captured diagnostic evidence

Relevant backend lines from the failed run:

OS Product: Windows 11
OS Release: 10.0.26200.9550
HM: HMR3Init: Attempting fall back to NEM: VT-x is not available
NEM: Created partition [address omitted]. APIC emulation mode: VirtualBox
NEM: NEMR3Init: Snail execution mode is active!
CPUM: Logical host processors: 24 present, 24 max, 24 online

Guest instruction pointers resolved against the matching kernel symbols:

vCPU0: nt!HalpInterruptResetThisProcessor+0x5e
vCPU1: nt!HalProcessorIdle+0xf
vCPU2: nt!HalpNmiReboot+0xde

The following TRPM state was observed in two separate reads during the failure:

CPU[0]: TRPM event (None)
CPU[1]: TRPM event
 Type       = 0x3 (Invalid!)
CPU[2]: TRPM event (None)

In this version, enum value 3 is TRPM_NMI. The diagnostic name table omits that value, so the displayed Invalid! text is not by itself evidence of corrupt event data.

Force flags from the failure:

Global FFs: 0x0
CPU 0 FFs: 0x80000
    TLB_FLUSH
CPU 1 FFs: 0x80000
    TLB_FLUSH
CPU 2 FFs: 0x80005
    INTERRUPT_APIC, TIMER, TLB_FLUSH

vCPU1 repeatedly exited through HLT. The initiating CPU's APIC ICR was consistent with targeting vCPU0/vCPU1 (NMI logical destination; low 0x0c00, high 0x03000000; LDR values 1/2/4). Please distinguish an injection-attempt counter from proof that the guest handler actually ran.

These are selected diagnostic excerpts. A full unredacted VBox.log, VM disks, credentials and memory dumps are not attached. I have not marked the template's “all pertinent files uploaded” checkbox as completed.

Suspected code path

In v7.2.12 EMR3.cpp, EM can accept an NMI into TRPM with TRPMAssertTrap(..., TRPM_NMI), clear VMCPU_FF_INTERRUPT_NMI, and request rescheduling.

Windows NEMR3RunGC then directly enters nemHCWinRunGC. The native interrupt force-flag path handles an NMI which has not already been transferred into TRPM, and asserts that there is no pending TRPM event. This appears to leave an already accepted event without a delivery step at that transition.

The v7.2.22 NEM/EM/EMR3Nem paths and current main NEM entry were inspected on 2026-10-27 and still lack the candidate delivery step below. This is source inspection, not a claim to have reproduced the failure on those binaries.

Candidate patch against v7.2.12

The intended change is to import the IEM exception state, invalidate IEM TLB state, and inject an accepted TRPM event before native execution. Non-success status is propagated. VINF_IEM_RAISED_XCPT is an alias of VINF_EM_RESCHEDULE, so successful NMI injection can return to EM for rescheduling.

--- a/src/VBox/VMM/VMMR3/target-x86/NEMR3Native-win-x86.cpp
+++ b/src/VBox/VMM/VMMR3/target-x86/NEMR3Native-win-x86.cpp
@@ -5266,6 +5266,21 @@
 VMMR3_INT_DECL(VBOXSTRICTRC) NEMR3RunGC(PVM pVM, PVMCPU pVCpu)
 {
     Assert(VM_IS_NEM_ENABLED(pVM));
+
+    /* EM may have accepted an NMI into TRPM while leaving the halted state.
+       Consume that accepted event before entering WHP: the native FF path
+       only sees interrupts which have not yet been transferred into TRPM. */
+    if (TRPMHasTrap(pVCpu))
+    {
+        VBOXSTRICTRC rcStrict = nemHCWinImportStateIfNeededStrict(pVCpu,
+                                    NEM_WIN_CPUMCTX_EXTRN_MASK_FOR_IEM_XCPT, "TRPM");
+        if (rcStrict != VINF_SUCCESS)
+            return rcStrict;
+        IEMTlbInvalidateAll(pVCpu);
+        rcStrict = IEMInjectTrpmEvent(pVCpu);
+        if (rcStrict != VINF_SUCCESS)
+            return rcStrict;
+    }
     return nemHCWinRunGC(pVM, pVCpu);
 }
 

Validation performed and request

An extracted-entry regression test failed on the old body and passed five cases on this candidate. It used substituted import/IEM/native boundaries. It therefore checks ordering and status propagation only: it does not exercise real IEM, EM, WHP, NMI blocking or a running guest. Source review did not identify a confirmed functional error, but full build and integration validation remain open.

Could the VirtualBox team review this candidate and provide an Oracle-signed Windows x64 test build containing either this change or an equivalent fix? We can validate it on the existing VM with five consecutive normal guest-initiated restarts at three active vCPUs, checking a new boot record and service/driver readiness after each. No forced reset, pause/resume, injected NMI or host security changes would count as passing.

Related report #799 concerns a different primary symptom (EFI boot hang). We have not established that it shares this exact cause.

This report and the candidate patch were prepared with AI assistance from captured diagnostics and source review. Original upstream licensing/notices remain applicable. This is not presented as a verified fix or an Oracle release.