Automation – Page 164

How to audit vSphere Standalone VMRC or HTML5 VMRC connections?

07.11.2016 by William Lam // Leave a Comment

An interesting question that came in last week from one of our TAMs was how to identify and audit Virtual Machine Remote Console (VMRC) logins from vSphere? The TAM was specifically interested in being able to correlate that a particular user had logged into the VMRC of a VM during a specific period of time. Luckily, this is easily retrievable through vCenter Servers's Event sub-system that stores information about everything that happens in your vSphere environment. The Events can be accessed using either the vSphere Web Client shown below or programmatically using the vSphere API which the UI is built on top of.

You can obviously filter your search in the UI and focus on a particular VM, but often times there can be dozens if not hundreds of "Events" generated for a given VM. I personally prefer to leverage Automation when needing to look for a specific type of Event and more importantly, you can further process the results to either send out reports or hook into other third party systems. Now that we know, "where" to find our data, the next thing is identifying the type of Event that is generated for a VMRC connection.

As of vSphere 5.5 Update 2b, the VMRC in the vSphere Web Client can be accessed in one of two ways: The new HTML5 VMRC by clicking onto the VM screenshot thumbnail or the Standalone VMRC by clicking on the link directly beneath the VM screenshot.

Each VMRC connection method will generate a unique vCenter Server Event. For HTML5 VMRC connections, the Event is called VmAcquiredMksTicketEvent and for Standalone VMRC connections, the Event is called VmAcquiredTicketEvent. As I mentioned earlier, the vCenter Server Event sub-system can be accessed using the vSphere API and you can find the complete list of Events documented here. To demonstrate the use of this particular vSphere API, below is a PowerCLI example using the Get-VIEvent cmdlet. My fellow colleague Alan Renouf has actually blogged about working with Events using PowerCLI which I will be adapting one of his examples for our use case.

We first retrieve the VM that we are interested in by running the following command (specify the name of your VM):

$vm = Get-VM -Name "VCSA-60u2"

To retrieve HTML5 VMRC connections, run the following PowerCLI command:

Get-VIEvent -Entity $vm | Where { $_.Gettype().Name -eq "VmAcquiredMksTicketEvent"} | Select CreatedTime, UserName, FullFormattedMessage | ft -wrap -AutoSize

Here is an example of what the output would look like

To retrieve Standalone VMRC connections, run the following PowerCLI command:

Get-VIEvent -Entity $vm | Where { $_.Gettype().Name -eq "VmAcquiredTicketEvent"} | Select CreatedTime, UserName, UserAgent, FullFormattedMessage | ft -wrap -AutoSize

Here is an example of what the output would look like:

Quick Tip - How to quickly find the VM Display Name of your vCenter Server?

07.07.2016 by William Lam // 1 Comment

It is a pretty common practice for customers to use their vCenter Server to manage the underlying ESXi host which is also running the vCenter Server VM, sometimes referred to as a self managed vSphere Cluster. For organizations that have a strict naming convention which includes the VM display name as well as the OS hostname (FQDN), it is generally not too difficult to identify the actual VM that is running your vCenter Server.

However, for customers who may not have a VM display name that can easily be associated to their vCenter Server, it can be pretty difficult and time consuming to locate the vCenter Server VM for troubleshooting and/or updating purposes. Historically, this mapping and association has been left to our customers to document whether it is using a sticky, CMDB, vSphere Tags/Folders, etc. to be able to identify some of these more important VMs.

Luckily, there are some built-in mechanisms within vCenter Server itself that we can leverage to help us quickly identify the VM display name of our vCenter Server. Just this week while working on something, I actually came across another method which I think is even easier than the one I was familiar with.

Option #1 (harder) - When vCenter Server detects that it is in a self-managed configuration, it will automatically add a "System" metadata tag (not to be confused with vSphere Tags) to the actual vCenter Server VM. For vCenter Server, the "Tag" will have the value of SYSTEM/COM.VMWARE.VIM.VC which you can query on a VM using the vSphere API. Below is a PowerCLI example exercising the vSphere API:

$vm = Get-VM -name VCSA-60u2
(Get-View $vm).ExtensionData.Tag

The main issue here with this option is that because this information is at the VM level, you must first find the VM. This means, you would have to iterate through all of your VMs and seeing which has this particular property set with this value.

Option #2 (easier) - As mentioned earlier, I came across this new option by accident while browsing through the vCenter Server Advanced Settings. I noticed an interesting value for the following key: config.registry.key_VCVmId After a quick investigation, I found that vCenter Server also sets this Advanced Setting to the Managed Object Reference (MoRef) ID of the actual vCenter Server VM. This means, in one step, I know exactly which VM is my vCenter Server and I just need to convert the MoRef ID to a VM Object to then query the VM display name. How cool!?

The following PowerCLI example demonstrates how to extract the vCenter Server Advanced Setting, convert the MoRef ID to a VM Object which we can then query for its VM Display Name.

$vc_server = "192.168.1.51"
$vc_username = "*protected email*"
$vc_password = "VMware1!"

$server = Connect-VIServer -Server $vc_server -User $vc_username -Password $vc_password

# Retrieve the MoRef ID from following VC Advanced Setting
$vcMoRef = (Get-AdvancedSetting -Entity $server -Name config.registry.key_VCVmId).Value

# Construct VM from MoRef
$vm = New-Object VMware.Vim.ManagedObjectReference
$vm.Type = "VirtualMachine"
$vm.Value = $vcMoRef

Write-Host "The VM display name of this vCenter Server is"(Get-View $vm).name

Disconnect-viserver $server -confirm:$false

As you can see from the screenshot below, with just a few lines of code, we can quickly figure out our vCenter Server VM regardless if our inventory is 10 or 10,000 VMs. In fact, you can even output the exact ESXi host that is currently running the VM as well, but I will leave that as an exercise for the reader 🙂

Using the vSphere API to remotely collect ESXi configuration file (esx.conf)

06.23.2016 by William Lam // 9 Comments

Last week we took a look at two new Automated solutions here and here that allows us to leverage vCenter Server and the vSphere APIs to remotely extract information that historically required logging in directly into an ESXi host. While working on the two scripts, I was reminded of another use that could also be really useful which builds on top of some information that I had shared back in 2012. ESXi provides a very basic file manipulation capability that is exposed as a simple HTTPS-based interface.

Here is a quick recap of the three URLs which can be accessed by opening a browser and logging into the ESXi host:

https://esxi-1.primp-industries.com/host
https://esxi-1.primp-industries.com/folder
https://esxi-1.primp-industries.com/tmp

For the purpose of this article, we will be focusing on the first url endpoint /host and below is an example screenshot on some of the configuration files (46 in total) that you would be able to access using this interface.

One of the available ESXi configuration files that you access is the esx.conf file directly where it might be useful to periodically capture the state of this file for either auditing or troubleshooting purposes.

Note: Although esx.conf does contain some amount of the ESXi configurations, it does not represent the full state of the ESXi host. If you wish to perform periodic full backups of your ESXi host (which includes esx.conf by default among other files), there is a vSphere API for this by using the HostFirmwareSystem and the BackupFirmwareConfiguration() method.

Applying the same technique as I have described here, we can easily retrieve the esx.conf for a specific ESXi host being managed by vCenter Server without needing directly login to the ESXi host or worse connecting via SSH. I have created a PowerCLI script called Get-Esxconf.ps1 which just accepts a VMHost object.

Here is an example of how you would use the function and screenshot below of the output:

$esxConf = Get-VMHost -Name "esxi-1" | Get-Esxconf

If you are interested in a specific key within the esx.conf configuration file, we further process the output. The following snippet below searches for the following key /system/uuid and will return the value as it iterates through the esx.conf output.

$esxConf = Get-VMHost -Name "esxi-1" | Get-Esxconf

$keyToSearchFor = "/system/uuid"

foreach ($line in $esxConf.Split("`n")) {
    $data = $line.split("=").trim().replace('"',"")
    if($data[0] -eq $keyToSearchFor) {
        Write-Host "Key:" $keyToSearchFor 
        write-Host "Value:" $data[1]
    }
}

Hopefully this gave you an idea of just one of the many use cases that can now be enabled through the use of the vSphere API and this ESXi interface. Here are just a few other use cases that I can think of on the top of my mind that could come in handy:

Managing ESXi SSH public/private keys, we have mostly been using httpGet, but you can also use an httpPut to upload these files without needing to go to each and every ESXi host
Replacing Custom SSL Certificates if you are not using VMCA, you can also use an httpPut request to upload these files (you will need to restart hostd or reboot the host for the new SSL Certificates to go into effect)
Quickly access the vpxa.cfg (vCenter Server agent) configuration file for troubleshooting purposes