WilliamLam.com

SPBM (Storage Policy Based Management) MOB in vSphere 5.5

by // 8 Comments

A handy little tool that was useful for me which might come in handy for others too while working on my VSAN VM Storage Policy recovery article is the SPBM (Storage Policy Based Management) MOB which works similar to the vSphere MOB and FDM MOB for those of you who have used this interface before. The MOB stands for Managed Object Browser and simply put is an interface that allows you to browse the objects and properties of your vSphere environment by just using a web browser. You can also interact with the MOB by executing the same set of vSphere API methods as you would if you were to write a script or application which is useful for quickly getting a sense of what a certain property might look like or the output of an API method without writing a single line of code.

In vSphere 5.5, there is now an SPBM API which is available as a separate API endpoint on the vCenter Server. This new API allows you to manage the full lifecycle of a VM Storage Policy formally known as VM Storage Profiles from a programmatic standpoint which is very important when working with VSAN as everything is policy driven. For more more details about the new SPBM API, please take a look here. As mentioned earlier, one great way to learn about the API is by exploring the MOB and luckily the SPBM API includes one!

To access the SPBM MOB in vSphere 5.5, you will need to point your browser to your vCenter Server as that is where the endpoint is exposed using the following URL format:

http://[VC-IP-OR-HOSTNAME]:8190/mob

To access the SPBM MOB in vSphere 6.0, you will need to point your browser to your vCenter Server as that is where the endpoint is exposed using the following URL format:

http://[VC-IP-OR-HOSTNAME]/pbm/mob

You will be prompted for your vCenter Server credentials before the MOB will allow you to login. Once you are logged in you will be brought to the main service content of the SPBM endpoint similar to the vCenter Server service content and you can then click on content link to explore the various sub-managers that are available.
Note: You will be able to get more details on each of these sub-managers by taking a look at the VM Storage Policy Programming Guide and VM Storage Policy API reference guide.
To quickly show you around, I will provide a couple of examples using the ProfileManager and I am sure you can probably guess what type of functionality it provides :). The first method that we will take a look at is the PbmQueryProfile which will return the list of available VM Storage Policies that have been defined. You will need to set the resourceType property to "STORAGE" and remove the profileCategory and then click on "Invoke Method".
If you are using VSAN and you do not have any VM Storage Policies defined, there will still be two default VM Storage Policies that is automatically created when VSAN is enabled. What you will see are the internal identifiers for each of the VM Storage Policy and as you can see from the output I have 5 VM Storage Policies.
You will notice that the output does not contain the human readable display name for each VM Storage Policy, to retrieve that information we will need to use the PbmRetrieveContent which accepts a list of VM Storage Profile ID's and in return provide the human readable name as well as other properties such as the initial creation date and last modified date. Using the pre-canned input form, you can specify one or more VM Storage Profile IDs from the previous step and then click on "Invoke Method".
In my example, I specified two of my VM Storage Policies and I can see they map to the names  "Aluminum" and "Copper" which is what I named them when I first created the policies.
From here on out, we will be using the VM Storage Policy ID as that is what is used to uniquely identify a VM Storage Policy and input for majority of the SPBM API methods. Now if we want to see what objects (VM Home directory or VMDKs) are associated with a particular VM Storage Policy we can use the PbmQueryAssociatedEntity method. You will need to provide the VM Storage Policy ID and remove the entityType and then click on the "Invoke Method".
As you can see from the output this a virtualMachine object type which tells us this VM Storage Policy is used for the VM Home. Lets go ahead and specify a VM Storage Policy that is used for a Virtual Machine's VMDK and see what that looks like.
We now see the object type is virtualDiskId and you can see the particular VMDK and the associated Virtual Machine by looking at the key which has the format of vm-mo-ref:vmdk-key. Now what if we wanted to perform the reverse look up, by providing only a Virtual Machine or VMDK as input? Well, we can easily do this lookup by using the PbmQueryAssociatedProfiles method. This API method requires you to specify three parameters: objectType, key and serverUuid (technically speaking the serverUuid can be left out).
From the above examples you will get an idea of what the expected input format is for either a Virtual Machine or VMDK query.
Here is an example of a Virtual Machine query:
Here is an example of a VMDK query:
Hopefully this quick introduction of the SPBM MOB will give you a good idea on how you can leverage this interface, especially if you plan on using the new SPBM API to automate and manage your VM Storage Policies.

Automate the reverse, migrating from vSphere Distributed Switch to Virtual Standard Switch using PowerCLI 5.5

by // 14 Comments

Last week I demonstrated how you can easily leverage the new PowerCLI 5.5 VDS cmdlets to migrate from a VSS (Virtual Standard Switch) to a VDS (vSphere Distributed Switch). During the development of the script, I needed a way to easily jump back and fourth between VSS->VDS and VDS->VSS and I wanted to automate this so I did not have to use the UI to reset my environment.

I initially thought this was not possible after playing around with a couple of the cmdlets but thanks to Kamen, a PowerCLI Engineer who was able to provide me with the necessary information to create a reverse migration script going from VDS to VSS.

Here is what my lab environment looks like which includes three ESXi hosts connected to a VDS called "VDS-01" which is backed by 4 pNICSs. The VDS contains 3 VMkernel interfaces and here are their respective DVPortgroup names: Management Network, Storage Network and vMotion Network.

On each ESXi host, there is an already created VSS called "vSwitch0". If one is not created or if you decide to name it something differently, then you will need to modify the script. Here is a quick screenshot of what that looks like

The PowerCLI example script below uses the Add-VirtualSwitchPhysicalNetworkAdapter cmdlet which accepts a list of pNICs, VMkernel interfaces and the portgroups to migrate from VDS to VSS. The order in which the VMkernel and portgroups are specified is critically important as they will be assigned based on the provided ordering. The script also create the necessary portgroups on the VSS which of course can be modified based on your environment. Once the migration has been completed, it will then use the Remove-VDSwitchVMHost cmdlet to remove the ESXi hosts from the VDS.

Disclaimer: Please ensure you test this script in a development/test lab before using it in a production environment.

Connect-VIServer -Server vcenter55-1.primp-industries.com -User *protected email* -Pass vmware</b></i>

# ESXi hosts to migrate from VSS-&gt;VDS
$vmhost_array = @("vesxi55-1.primp-industries.com","vesxi55-2.primp-industries.com","vesxi55-3.primp-industries.com")

# VDS to migrate from
$vds_name = "VDS-01"
$vds = Get-VDSwitch -Name $vds_name

# VSS to migrate to
$vss_name = "vSwitch0"

# Name of portgroups to create on VSS
$mgmt_name = "Management Network"
$storage_name = "Storage Network"
$vmotion_name = "vMotion Network"

foreach ($vmhost in $vmhost_array) {
Write-Host "`nProcessing" $vmhost

# pNICs to migrate to VSS
Write-Host "Retrieving pNIC info for vmnic0,vmnic1,vmnic2,vmnic3"
$vmnic0 = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmnic0"
$vmnic1 = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmnic1"
$vmnic2 = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmnic2"
$vmnic3 = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmnic3"

# Array of pNICs to migrate to VSS
Write-Host "Creating pNIC array"
$pnic_array = @($vmnic0,$vmnic1,$vmnic2,$vmnic3)

# vSwitch to migrate to
$vss = Get-VMHost -Name $vmhost | Get-VirtualSwitch -Name $vss_name

# Create destination portgroups
Write-Host "`Creating" $mgmt_name "portrgroup on" $vss_name
$mgmt_pg = New-VirtualPortGroup -VirtualSwitch $vss -Name $mgmt_name

Write-Host "`Creating" $storage_name "Network portrgroup on" $vss_name
$storage_pg = New-VirtualPortGroup -VirtualSwitch $vss -Name $storage_name

Write-Host "`Creating" $vmotion_name "portrgroup on" $vss_name
$vmotion_pg = New-VirtualPortGroup -VirtualSwitch $vss -Name $vmotion_name

# Array of portgroups to map VMkernel interfaces (order matters!)
Write-Host "Creating portgroup array"
$pg_array = @($mgmt_pg,$storage_pg,$vmotion_pg)

# VMkernel interfaces to migrate to VSS
Write-Host "`Retrieving VMkernel interface details for vmk0,vmk1,vmk2"
$mgmt_vmk = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmk0"
$storage_vmk = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmk1"
$vmotion_vmk = Get-VMHostNetworkAdapter -VMHost $vmhost -Name "vmk2"

# Array of VMkernel interfaces to migrate to VSS (order matters!)
Write-Host "Creating VMkernel interface array"
$vmk_array = @($mgmt_vmk,$storage_vmk,$vmotion_vmk)

# Perform the migration
Write-Host "Migrating from" $vds_name "to" $vss_name"`n"
Add-VirtualSwitchPhysicalNetworkAdapter -VirtualSwitch $vss -VMHostPhysicalNic $pnic_array -VMHostVirtualNic $vmk_array -VirtualNicPortgroup $pg_array  -Confirm:$false
}

Write-Host "`nRemoving" $vmhost_array "from" $vds_name
$vds | Remove-VDSwitchVMHost -VMHost $vmhost_array -Confirm:$false

<i><b>Disconnect-VIServer -Server $global:DefaultVIServers -Force -Confirm:$false

Here is a screenshot of the script executing:

How to generate a pre-authenticated HTML5 VM console link in vSphere 5.5 that works on all OSes!

by // 79 Comments

One of the coolest feature that I have been personally looking forward to is the ability to access a virtual machine's remote console on a Mac OS X system which is now available as part of the vSphere 5.5 release. When you launch the VM's remote console using the vSphere Web Client on a Mac OS X system, instead of using the regular VMRC, it launches an HTML5 based console for your VM.

One thing that you might have noticed when performing this same action on a Windows desktop using the vSphere Web Client is that VMRC is used instead of the HTML5 console (notice the generated URLs are different). My understanding is that the VMRC is currently more performant than the HTML5 console and hence it is preferred when possible. I have been using the HTML5 based console for quite sometime now and I have not had any issues with it. I really hope to see us use the HTML5 console as the default console in the future!

Having said all this, there is a way for all users can benefit from this new HTML5 based VM console which is to automatically generate the URL which can then be loaded into any HTML5 supported web browser on either a Mac OS X, Windows or even Linux system. I used a similar method in generating the VM Remote Console for the vSphere Web Client which is VMRC specific.

UPDATE (07/26/17): I have just published a PowerCLI script called GenerateVMConsoleURL.ps1 which runs against a vSphere 6.5 environment and supports generating the HTML5 VM Console, Standalone VMRC and WebMKS URLs. As noted by several folks in the comments below, the pre-auth mechanism no longer works post-6.0, so you will need to have logged already for the console to automatically load OR you will be prompted to login before being re-directed. For those that wish to generate VM Console URLs for older vSphere versions, you can modify the script to handle those other scenarios.

The really cool part about this solution is that you can provide a one time pre-authenticated HTML5 based VM console URL that can then be given to your end users to access their VM. This of course can be automatically generated through a custom portal without needing to provide direct access to the vSphere Web Client.

Here is an example of what the HTML5 VM console URL looks like in vSphere 5.5:

http://reflex.primp-industries.com:7331/console/?vmId=vm-23&vmName=VCSA&host=reflex.primp-industries.com&sessionTicket=cst-VCT-5254c455-4340-2185-e149-01ce44b146e1--tp-4A-88-17-7C-F5-D0-79-E6-9D-A1-E3-83-97-52-97-EA-E5-D3-D8-07&thumbprint=4A:88:17:7C:F5:D0:79:E6:9D:A1:E3:83:97:52:97:EA:E5:D3:D8:07

In vSphere 5.5 Update 2, the HTML5 VM Console now defaults to a secure connection and the two components of the URL that needs to be modified is from HTTP to HTTPS and from port 7331 to port 7343. The script has been updated to support a new command-line option called isvSphere55u2 which by default is set to false but can be set to true to generate an updated URL if you are running vSphere 5.5 Update 2

There are basically seven important components to the URL:

  • Hostname of the vCenter Server - reflex.primp-industries.com
  • The HTML5 console port, default is 7331 but can automatically change depending on available ports on the system (for vSphere 5.5 Update 2 the port has changed to 7343 for a secure connection)
  • The MoRef ID of the virtual machine - vm-23
  • The virtual machine name - VCSA
  • The vCenter Server advanced setting "VirtualCenter.FQDN" - reflex.primp-industries.com
  • The session ticket generated from vCenter Server - cst-VCT-5254c455-4340-2185-e149-01ce44b146e1--tp-4A-88-17-7C-F5-D0-79-E6-9D-A1-E3-83-97-52-97-EA-E5-D3-D8-07
  • The vCenter Server SHA1 SSL Thumbprint - 4A:88:17:7C:F5:D0:79:E6:9D:A1:E3:83:97:52:97:EA:E5:D3:D8:07

Disclaimer: The HTML5 VM console URL format/behavior is not guaranteed and may change in the future. The only officially supported method of accessing the console is by launching it through the vSphere Web Client.

Here is a vSphere SDK for Perl script called generateHTML5VMConsole.pl that given a VM name as input, will automatically generate a one-time pre-authetnicated HTML5 VM console URL that can be loaded into any supported web browser.

Note: To ensure the URL is valid, you will need to make sure your application is setup to run like a daemon or agent. To simulate this in my sample script, I just sleep for 60 seconds before disconnecting the session. I also used the openssl utility to extract the SHA1 thumbprint, so you will need a system that has that installed along with the vSphere SDK for Perl if you wish to use the script. The quickest way is to leverage vMA.

To load the HTML5 VM console, take the URL that is generated and point it to a browser:

Since this is an HTML5 based VM console, I can even load this into my iPad!

Hopefully this article will give you new ideas on how you can leverage and integrate the new HTML5 VM Console within your environment and hopefully this will be the defacto console in the future!

Here is an example of what the URL looks like for vSphere 5.1 & 5.5:

https://reflex.primp-industries.com:9443/vsphere-clien/vmrc/vmrc.jsp?vm=urn:vmomi:VirtualMachine:vm-1506:EE26E7F6-591B-4256-BD7A-402E5AC9E0A8

There are basicallythree important components to the URL:

  • Hostname of the vCenter Server - reflex.primp-industries.com
  • The vCenter instanceUUID which used to uniquely identify a vCenter Server - EE26E7F6-591B-4256-BD7A-402E5AC9E0A8
  • The MoRef ID of the virtual machine - vm-1506