Power off VM from Guest OS Reboot capability in vSphere 8

In 2019, a new VM advanced setting called vmx.reboot.PowerCycle was introduced and greatly simplified the remediation of CPU vulnerabilities for our customers. The operational challenge that came with applying CPU microcode updates was that all running VMs on an ESXi host would need to go through a complete power cycle (power off and power back on) before the guest operating system(s) would be fully protected.

The new VM setting would notify ESXi to convert a guest operating system reboot into a VM power cycle operation, which aligns nicely when an organization applies a guest OS update or patch and the guest OS is then rebooted afterwards. The benefit to our customers is that the remediation of CPU vulnerabilities can co-exists with an organizations existing maintenance window and the OS is remediated through a regular guest OS reboot. From the guest OS point of view, nothing has changed, it still sees a reboot but from the VM virtual hardware point of view, a complete power cycle has occurred. A very cool and innovative solution if you ask me!?

The reason for this background, there is another use case that has also been operationally challenging which is upgrading the VM Compatibility, also known as VM Virtual Hardware. A VM must be powered off before you can change the VM Compatibility and a simliar challenge arises with coordinating the downtime of a VM with application owners/teams. What if we had a simliar capability like the guest OS reboot triggering a power cycle, but instead of power cycling the VM, it would simply power it off?

Well, we now have this new capability in both vSphere 8 and VMware Cloud on AWS with the latest 1.20 release and as the title suggests, it is about powering off a VM from a guest OS reboot. Before we dive in and see how this feature works, I do want to acknowledge that this feature was a result of a feature enhancement that was submitted by Patrick Kremer, who many of you know in the VMware Community. While Patrick is no longer with VMware, it was certainly nice to see his feature request get implemented and now made available to customers! I also had the opportunity to work with the VMware Engineer who implemented the feature and through early testing and feedback, he was able to improve the user experience from the original request.

So how does it work? In vSphere 8, a new API property is introduced for the VM managed object called rebootPowerOff which is a boolean that specifies whether this capability is enabled or not. If enabled, which can only be applied to a VM when it is powered on, will notify ESXi to convert the guest operating system reboot into a VM powered off operation.

Here is the quick PowerCLI snippet to reconfigure a VM and enable the rebootPowerOff property:

$vm = Get-VM "Test"
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$spec.rebootPowerOff = $true
$vm.ExtensionData.ReconfigVM_Task($spec)

Note: A new version of PowerCLI will be required that understands the new API property, but the snippet above will work once you have an updated version of PowerCLI. In the mean time, you can use vSphere MOB and simply pass in single field called rebootPowerOff = "true" to the XML spec or you could even automate the vSphere MOB operation.

Once the rebootPowerOff flag is enabled and a guest OS reboot is performed for a VM, you will see the VM is then powered off. The rebootPowerOff property is automatically reset back to the default value of false and if you wish to enable this capability, you will need to reapply it.

You can also see the result of the guest OS reboot getting converted into a VM power off by looking at the vCenter Events for the VM as shown in the screenshot below.

In addition to powering off the VM, a new vSphere Event is also generated by vCenter for this operation called com.vmware.vim.vm.reboot.powerOff and this can then be easily consumed by the VMware Event Broker Appliance (VEBA) solution to automatically upgrade to the desired VM Compatibility when this event occurs, further reducing the operational overhead and simplifying VM Compatibility management or any other offline VM reconfiguration that requires a VM to be powered off completely.