How to determine when a Virtual Machine is ready for additional operations?

This might sound like a pretty simple and basic question, right? However, the answer will actually depend on what you are trying to do. The motivation for this blog post actually stemmed from a conversation I had with one of our Engineers who works over in our core Platform Management Infrastructure which includes VMware Tools, Guest OS Customization, Guest Operations, etc. Although I was aware of some of these methods in determining when a VM was ready, I came to learn about a few new other methods that I was not aware of before. This is really one of the things I love about my job, constantly learning new things and diving deeper into our technologies and sharing that back with our customers to help enable them and their business.

So, what does it even mean for a Virtual Machine to be "ready"? Is it when the VM is powered on? Is it when the VM obtains an IP Address? Is it when the GuestOS is fully customized or is it when I can SSH or RDP to the system? Depending on what you are trying to accomplish, the answer will vary. In addition to that, there are two distinct VM states to consider. The first being the actual Virtual Machine state and the second being the GuestOS state.

Virtual Machine State

The VM state is pretty straight forward, it describes the "Hard" power state of the VM which can either be powered on, powered off or suspended. As you can probably guess, you will need to make sure the VM is powered on before you attempt to check whether the GuestOS is ready 🙂

Using the vSphere API, you can find the VM powerState under:

VirtualMachine->Runtime->powerState

Here is a PowerCLI snippet using the out of box cmdlet to get the power state for a VM named "air":

Get-VM -Name air | Select PowerState

Here is a PowerCLI snippet that uses the vSphere API via Get-View to retrieve the exact same information:

$vm = Get-View -ViewType Virtualmachine -Property Name,Runtime -Filter @{"name"="air"}
$vm.Runtime.PowerState

GuestOS State

There are four specific GuestOS state properties to be aware of which provides different pieces of information about the state of the GuestOS:

1. guestState
2. guestOperationsReady
3. interactiveGuestOperationsReady
4. guestStateChangeSupported

Lets take a closer look at each them and the use cases they support.

guestState

This property (available in all vSphere versions) describes the "Soft" power state of the VM. It can have one of the following values (this is taken directly from the vSphere API Reference Guide):

• "running" - Guest is running normally.
• "shuttingdown" - Guest has a pending shutdown command.
• "resetting" - Guest has a pending reset command.
• "standby" - Guest has a pending standby command.
• "notrunning" - Guest is not running.
• "unknown" - Guest information is not available.

This particular state tells us a couple of things. The GuestOS has booted up for the most part (it does not necessary mean all applications are already) and that it also has started the VMware Tools service which is how we are retrieving this information in the first place. In general, this state is usually sufficient in indicating that remote connectivity to the VM like RDP or SSH is now available and you can perform other guest related actions using other remote tooling.

We can also take advantage of this property when we do not see a "running" state which can then be used to debug further by taking a VM console screenshot using the vSphere API to determine where an OS boot may have failed.

Using the vSphere API, you can find the VM guestState under:

VirtualMachine->Guest->guestState

Here is a PowerCLI snippet using the Get-VM cmdlet to retrieve the power state for a VM named "air":

(Get-VM -Name air).ExtensionData.guest.guestState

Here is a PowerCLI snippet that uses the vSphere API via Get-View to retrieve the exact same information:

$vm = Get-View -ViewType VirtualMachine -Property Name,Guest -Filter @{"name"="Air"}
$vm.Guest.guestState

guestOperationsReady

If true, this property (introduced in vSphere 5.0) means that you can use any of the vSphere GuestOperations managers like aliasManager, authManager, fileManager, guestWindowsRegistryManager and processManager to manipulate filesystem and/or running processes within the OS that does NOT require desktop interaction like an interactive GUI application for example.

Using the vSphere API, you can find the VM guestOperationsReady under:

VirtualMachine->Guest->guestOperationsReady

Here is a PowerCLI snippetusing the Get-VM cmdlet to retrieve the state for a VM named "air":

(Get-VM -Name air).ExtensionData.guest.guestOperationsReady

Here is a PowerCLI snippet that uses the vSphere API via Get-View to retrieve the exact same information:

$vm = Get-View -ViewType VirtualMachine -Property Name,Guest -Filter @{"name"="Air"}
$vm.Guest.guestOperationsReady

interactiveGuestOperationsReady

If true, this property (introduced in vSphere 5.0) means that you can now interact with the desktop using vSphere's GuestOperations. This can be useful for any type of interactive sessions such as working with a Windows network filesystem. For this scenario to work, you must first ensure that the same user is logged into the desktop as the user performing the GuestOperations. Also make sure to set the interactiveSession flag as part of the GuestAuthentication structure.

Using the vSphere API, you can find the VM interactiveGuestOperationsReady under:

VirtualMachine->Guest->interactiveGuestOperationsReady

Here is a PowerCLI snippet using the Get-VM cmdlet to retrieve the state for a VM named "air":

(Get-VM -Name air).ExtensionData.guest.interactiveGuestOperationsReady

Here is a PowerCLI snippet that uses the vSphere API via Get-View to retrieve the exact same information:

$vm = Get-View -ViewType VirtualMachine -Property Name,Guest -Filter @{"name"="Air"}
$vm.Guest.interactiveGuestOperationsReady

guestStateChangeSupported

If true, this property (introduced in vSphere 6.0) means that you can perform "soft" power operations on the VM such as a GuestOS shutdown, reboot or standby.

Using the vSphere API, you can find the VM guestStateChangeSupported under:

VirtualMachine->Guest->guestStateChangeSupported

Here is a PowerCLI snippet using the Get-VM cmdlet to retrieve the state for a VM named "air":

(Get-VM -Name air).ExtensionData.guest.guestStateChangeSupported

Here is a PowerCLI snippet that uses the vSphere API via Get-View to retrieve the exact same information:

$vm = Get-View -ViewType Virtualmachine -Property Name,Guest -Filter @{"name"="Air"}
$vm.Guest.guestStateChangeSupported

GuestOS Customization

Lastly, I wanted to quickly cover GuestOS customization as this topic has come up from customers in the past. Whether a given GuestOS customization is successful or not, it is really out of vCenter Servers control. To give you an example, if you customizing a Windows OS and your Active Directory server was unreachable or if you tried to use a duplicate IP Address or FQDN, then the customization will fail and you will end up with a VM that is not configured as we would expect.

Having said that, you can rely on vCenter Server to provide you with information about the GuestOS customization task and specifically when it started, if it succeeded or failed. With this information, you can then re-attempt the provisioning operation or take additional actions to check your underlying infrastructure. To do so, we can monitor for the following events that would be generated from a GuestOS customization task:

• CustomizationStartedEvent
• CustomizationFailed
• CustomizationSucceeded

For failed GuestOS customization, the error message also includes a reference to where you can find the GuestOS customization logs within the Guest for additional troubleshooting.