WilliamLam.com

SPBM (Storage Policy Based Management) MOB in vSphere 5.5

by // 8 Comments

A handy little tool that was useful for me which might come in handy for others too while working on my VSAN VM Storage Policy recovery article is the SPBM (Storage Policy Based Management) MOB which works similar to the vSphere MOB and FDM MOB for those of you who have used this interface before. The MOB stands for Managed Object Browser and simply put is an interface that allows you to browse the objects and properties of your vSphere environment by just using a web browser. You can also interact with the MOB by executing the same set of vSphere API methods as you would if you were to write a script or application which is useful for quickly getting a sense of what a certain property might look like or the output of an API method without writing a single line of code.

In vSphere 5.5, there is now an SPBM API which is available as a separate API endpoint on the vCenter Server. This new API allows you to manage the full lifecycle of a VM Storage Policy formally known as VM Storage Profiles from a programmatic standpoint which is very important when working with VSAN as everything is policy driven. For more more details about the new SPBM API, please take a look here. As mentioned earlier, one great way to learn about the API is by exploring the MOB and luckily the SPBM API includes one!

To access the SPBM MOB in vSphere 5.5, you will need to point your browser to your vCenter Server as that is where the endpoint is exposed using the following URL format:

http://[VC-IP-OR-HOSTNAME]:8190/mob

To access the SPBM MOB in vSphere 6.0, you will need to point your browser to your vCenter Server as that is where the endpoint is exposed using the following URL format:

http://[VC-IP-OR-HOSTNAME]/pbm/mob

You will be prompted for your vCenter Server credentials before the MOB will allow you to login. Once you are logged in you will be brought to the main service content of the SPBM endpoint similar to the vCenter Server service content and you can then click on content link to explore the various sub-managers that are available.
Note: You will be able to get more details on each of these sub-managers by taking a look at the VM Storage Policy Programming Guide and VM Storage Policy API reference guide.
To quickly show you around, I will provide a couple of examples using the ProfileManager and I am sure you can probably guess what type of functionality it provides :). The first method that we will take a look at is the PbmQueryProfile which will return the list of available VM Storage Policies that have been defined. You will need to set the resourceType property to "STORAGE" and remove the profileCategory and then click on "Invoke Method".
If you are using VSAN and you do not have any VM Storage Policies defined, there will still be two default VM Storage Policies that is automatically created when VSAN is enabled. What you will see are the internal identifiers for each of the VM Storage Policy and as you can see from the output I have 5 VM Storage Policies.
You will notice that the output does not contain the human readable display name for each VM Storage Policy, to retrieve that information we will need to use the PbmRetrieveContent which accepts a list of VM Storage Profile ID's and in return provide the human readable name as well as other properties such as the initial creation date and last modified date. Using the pre-canned input form, you can specify one or more VM Storage Profile IDs from the previous step and then click on "Invoke Method".
In my example, I specified two of my VM Storage Policies and I can see they map to the names  "Aluminum" and "Copper" which is what I named them when I first created the policies.
From here on out, we will be using the VM Storage Policy ID as that is what is used to uniquely identify a VM Storage Policy and input for majority of the SPBM API methods. Now if we want to see what objects (VM Home directory or VMDKs) are associated with a particular VM Storage Policy we can use the PbmQueryAssociatedEntity method. You will need to provide the VM Storage Policy ID and remove the entityType and then click on the "Invoke Method".
As you can see from the output this a virtualMachine object type which tells us this VM Storage Policy is used for the VM Home. Lets go ahead and specify a VM Storage Policy that is used for a Virtual Machine's VMDK and see what that looks like.
We now see the object type is virtualDiskId and you can see the particular VMDK and the associated Virtual Machine by looking at the key which has the format of vm-mo-ref:vmdk-key. Now what if we wanted to perform the reverse look up, by providing only a Virtual Machine or VMDK as input? Well, we can easily do this lookup by using the PbmQueryAssociatedProfiles method. This API method requires you to specify three parameters: objectType, key and serverUuid (technically speaking the serverUuid can be left out).
From the above examples you will get an idea of what the expected input format is for either a Virtual Machine or VMDK query.
Here is an example of a Virtual Machine query:
Here is an example of a VMDK query:
Hopefully this quick introduction of the SPBM MOB will give you a good idea on how you can leverage this interface, especially if you plan on using the new SPBM API to automate and manage your VM Storage Policies.

Restoring VSAN VM Storage Policy without vCenter Part 2: Using vSphere API

by // 3 Comments

Last week I demonstrated a manual method of recovering a VSAN VM Storage Policy when vCenter Server is no longer available by using a VSAN command-line utility found in the ESXi Shell called cmmds-tool. Though this approach works, it can also be quite tedious and error prone as you have to manually go through various configuration files and extract out the individual VSAN Object UUIDs. Luckily, one can automate the process outlined in the previous article by leveraging the vSphere API to connect directly to an ESXi host and access the VSAN Internal CMMDS system (Clustering Monitoring, Membership and Directory Services).

Disclaimer: This script is provided as a sample, please ensure it is properly tested before using it in a production environment.

I have created a sample vSphere SDK for Perl script called queryVSANVMStoragePolicyMapping.pl to demonstrate the vSphere API functionality. To use this script, you will need to ensure you have either the vCLI or vSphere SDK for Perl 5.5 installed on a system or you can use the vMA 5.5 appliance. You will also need to install an additional Perl module called JSON which is used in the script.

In my environment, I have a Virtual Machine called VSAN-VM-1 which has the following VM Storage Policies assigned to it.

Lets say our vCenter Server is now gone, how do we go about recovering the VM Storage Policy configurations to rebuild it on our new vCenter Server? With this script it is quite simple to recover the information by simply connecting to the ESXi host and specifying the name of the Virtual Machine.

Here is a sample execution of the script for my VSAN-VM-1:

From the output we can see that it automatically identifies the VSAN Object UUID for the VM Home directory as well as all VMDKs that are associated with that Virtual Machine. The script then uses that information to pass to the QueryCmmds API method which is part of the vsanInternalSystem manager to perform the query. The output is then returned as a JSON string which is parsed by the script to display the VM Storage Policy MoRef ID for each corresponding Virtual Machine component along with their configured VSAN policies.

The VM Home directory maps to our "Copper" policy which looks like the following in the vSphere Web Client:

The first VMDK maps to our "Platinum" policy which looks like the following in the vSphere Web Client:

The final VMDK maps to our "Aluminum" policy which looks like the following in the vSphere Web Client:

How to evaluate vSphere Data Protection without DNS?

by // 15 Comments

A fellow colleague of mine was looking to deploy the latest vSphere Data Protection Advanced (VDP[A]) 5.5 in his lab this past week to check out some of the new enhancements made to VDP and interoperability between other VMware solutions. However, he was unable to move forward with his evaluation due to the mandatory DNS requirement for VDP. I can understand the need for having proper DNS in a production or even a POC environment but for a quick evaluation or home lab it is a bit much. In my opinion I think it is a nice to have but it should not be a mandatory requirement.

As part of the networking settings, VDP assumes that configured IP Address is fully resolvable both forward/reverse lookups and manually adding an entry into /etc/hosts will not work. If you try to move forward in the wizard, you will receive this nice error message.

I decided to look into this as I faced a similar challenge in my personal home lab which does not have a DNS Server due to limited resources unlike my remote home lab which has a proper DNS server running. Though we can not get rid of the DNS requirement since it is part of the VDP code, we can however leverage an application found within VDP to help us. This nifty little application is called dnsmasq which is a lightweight server that provides basic DNS, DHCP & TFTP services for a small network which works great for an evaluation or home lab.

Disclaimer: I have been able to verify basic VDP functionality including a complete VM image backup/restore using the method below. However, it is still recommended that you still have proper DNS configured in your environment for in-depth testing of VDP, especially when evaluating for a production environment.

Step 1 - Deploy the VDP OVA as you normally would and when you specify the network configurations, ensure you set the DNS entry to 127.0.0.1 (loop back address) and then you can power on the VM after deployment.

Step 2 - Once VDP is up and running, SSH into the system using root as username and changeme as the password. We will then need to update /etc/hosts to reflect the IP Address/Hostname of our VDP appliance as well as any others you may want VDP to be able to resolve. In my environment I have the following:

192.168.1.104 vdp.primp-industries.com vdp
192.168.1.103 vcenter.primp-industries.com vcenter
192.168.1.200 mini.primp-industries.com mini

Note: I would also recommend adding your "search" domain into /etc/resolv.conf (e.g. search primp-industries.com)

Step 3 - Next we need to edit the dnsmasq configuration file located in /etc/dnsmasq.conf and add the following entry (replace it with the IP Address you have assigned your VDP appliance):

server=192.168.1.104

Step 4 - Finally we just need to start the dnsmasq service as well as enable it to automatically startup on boot by running the following two commands:

/etc/init.d/dnsmasq start
chkconfig dnsmasq on

Step 5 - To confirm that everything has been configured correctly, we will perform both a forward and reverse lookup of our VDP hostname and IP Address. You can do so by using "nslookup" command or any others such as dig or host. In this example, I will be using nslookup and you will need to run the following two commands (ensure to replace it with your hostname/IP Address):

nslookup vdp.primp-industries.com
nslookup 192.168.1.104

Once you have confirmed both forward and reverse lookups are successful, you can then proceed to configuring your VDP appliance by opening a browser to https://[VDP-ADDRESS]:8543/vdp-configure and proceed with moving forward with the VDP configuration wizard.

Note: Ensure that 127.0.0.1 is shown in the VDP configuration UI for the DNS entry. For hostname, you will need to use the short hostname even though a hostname is officially defined as specifying the FQDN

One of the cool new features that I personally wanted to check out is the new emergency Direct-to-Host Emergency Restore functionality. This feature allows you to restore VM backups in the scenario that vCenter Server is not available and one such use case is when you are using VDP to backup vCenter Server itself which I have been able to successfully test in my home lab.