WilliamLam.com

How to evaluate vSphere Data Protection without DNS?

by // 15 Comments

A fellow colleague of mine was looking to deploy the latest vSphere Data Protection Advanced (VDP[A]) 5.5 in his lab this past week to check out some of the new enhancements made to VDP and interoperability between other VMware solutions. However, he was unable to move forward with his evaluation due to the mandatory DNS requirement for VDP. I can understand the need for having proper DNS in a production or even a POC environment but for a quick evaluation or home lab it is a bit much. In my opinion I think it is a nice to have but it should not be a mandatory requirement.

As part of the networking settings, VDP assumes that configured IP Address is fully resolvable both forward/reverse lookups and manually adding an entry into /etc/hosts will not work. If you try to move forward in the wizard, you will receive this nice error message.

I decided to look into this as I faced a similar challenge in my personal home lab which does not have a DNS Server due to limited resources unlike my remote home lab which has a proper DNS server running. Though we can not get rid of the DNS requirement since it is part of the VDP code, we can however leverage an application found within VDP to help us. This nifty little application is called dnsmasq which is a lightweight server that provides basic DNS, DHCP & TFTP services for a small network which works great for an evaluation or home lab.

Disclaimer: I have been able to verify basic VDP functionality including a complete VM image backup/restore using the method below. However, it is still recommended that you still have proper DNS configured in your environment for in-depth testing of VDP, especially when evaluating for a production environment.

Step 1 - Deploy the VDP OVA as you normally would and when you specify the network configurations, ensure you set the DNS entry to 127.0.0.1 (loop back address) and then you can power on the VM after deployment.

Step 2 - Once VDP is up and running, SSH into the system using root as username and changeme as the password. We will then need to update /etc/hosts to reflect the IP Address/Hostname of our VDP appliance as well as any others you may want VDP to be able to resolve. In my environment I have the following:

192.168.1.104 vdp.primp-industries.com vdp
192.168.1.103 vcenter.primp-industries.com vcenter
192.168.1.200 mini.primp-industries.com mini

Note: I would also recommend adding your "search" domain into /etc/resolv.conf (e.g. search primp-industries.com)

Step 3 - Next we need to edit the dnsmasq configuration file located in /etc/dnsmasq.conf and add the following entry (replace it with the IP Address you have assigned your VDP appliance):

server=192.168.1.104

Step 4 - Finally we just need to start the dnsmasq service as well as enable it to automatically startup on boot by running the following two commands:

/etc/init.d/dnsmasq start
chkconfig dnsmasq on

Step 5 - To confirm that everything has been configured correctly, we will perform both a forward and reverse lookup of our VDP hostname and IP Address. You can do so by using "nslookup" command or any others such as dig or host. In this example, I will be using nslookup and you will need to run the following two commands (ensure to replace it with your hostname/IP Address):

nslookup vdp.primp-industries.com
nslookup 192.168.1.104

Once you have confirmed both forward and reverse lookups are successful, you can then proceed to configuring your VDP appliance by opening a browser to https://[VDP-ADDRESS]:8543/vdp-configure and proceed with moving forward with the VDP configuration wizard.

Note: Ensure that 127.0.0.1 is shown in the VDP configuration UI for the DNS entry. For hostname, you will need to use the short hostname even though a hostname is officially defined as specifying the FQDN

One of the cool new features that I personally wanted to check out is the new emergency Direct-to-Host Emergency Restore functionality. This feature allows you to restore VM backups in the scenario that vCenter Server is not available and one such use case is when you are using VDP to backup vCenter Server itself which I have been able to successfully test in my home lab.

Restoring VSAN VM Storage Policies without vCenter Part 1: Using cmmds-tool

by // 5 Comments

A scenario that I have been been looking into recently while testing VSAN in my lab is what happens when vCenter Server is no longer available and the impact that might have on your environment.  We know that VSAN from a configuration perspective works very similiar to vSphere HA where vCenter Server is only required for the initial VSAN Cluster configuration. Once the ESXi hosts has been added to the VSAN Cluster, vCenter Server is no longer part of the picture from a functional perspective and the ESXi hosts will know how to communicate with each other within the VSAN Cluster. We can even build a single VSAN node to help bootstrap vCenter Server itself for greenfield deployments.

So what does that leave us with? Well, the Virtual Machines of course. The Virtual Machines will continue to run without any impact whether or not vCenter Server is available. VSAN will continue to govern and maintain compliance for the VM Storage Policies that have been assigned to each and every Virtual Machine. However, in the scenario where you can not restore vCenter Server which is primarily where the VM Storage Policies are stored and you need to build out a new environment, how do you go about restoring the VM Storage Policies?

Well it turns out that vCenter Server is not the only place where the VM Storage Policies are stored at. To ensure that VSAN can continue enforcing the policies that have been assigned to each Virtual Machine and their associated VMDKs, there is a copy of the VM Storage Policies that is distributed amongst all the ESXi hosts within the VSAN Cluster. In this first first article I will demonstrate how to recover the VM Storage Policies for a particular Virtual Machine running on an ESXi host where vCenter Server is no longer available using a utility located in the ESXi Shell called cmmds-tool. In part two of the article I will demonstrate the same recovery process but leveraging the vSphere API which will be more user friendly.

Disclaimer: The cmmds-tool is not meant for troubleshooting, you should only use under VMware GSS/Engineering supervision. If you choose to use it, do so at your own risk.

In the ESXi Shell, there is a nifty little VSAN utility called cmmds-tool which stands for Clustering Monitoring, Membership and Directory Services. This tool allows you to perform a variety of operations and queries against the VSAN nodes and their associated objects. One interesting command is the "find" operation which will allow us to lookup a specific VM Storage Policy, a bit more on this later.

Lets say we have a Virtual Machine called VSAN-VM-1 and it is associated with three VM Storage Policies called Copper, Aluminum and Platinum. We have one for the VM Home and one for each of the two VMDKs. Here is a screenshot of what that looks like in the vSphere Web Client:

Now lets say vCenter Server is some how lost or unrecoverable for whatever reason, but we still have access to the ESXi host and the running Virtual Machine. Lets go ahead and recover the VM Storage Policies so we can then rebuild a new vCenter Server and re-create the policies.

Step 1 - We need to first identify a couple of pieces of information. The first is going to be the UUID of the VM Home directory (VSAN uses with UUIDs for all its objects). Login to ESXi Shell of the ESXi host that is currently hosting the Virtual Machine and run the following command:

vim-cmd vmsvc/getallvms | grep [DISPPLAY-NAME-OF-YOUR-VM]

The VM Home directory UUID will be part of the Virtual Machine directory name which can be seen in the screenshot above highlighted in green. Make a note of that UUID as you will need it in a later step. You should also make a note of the Virtual Machine MoRef ID which is the first numeric value on the left hand side of the output. In this example, I have 1 as the MoRef ID

Step 2 - Next we need to identify the UUID for each of the VMDKs for that given Virtual Machine. To do so, we need to take a look at the descriptor file for each of the VMDKs in the Virtual Machine home directory. You can use vim-cmd vmsvc/get.filelayout [VM-MOREF-ID] to get the VMDK paths or you can change into the Virtual Machine directory and cat out the files. In my example I have the following two VMDK descriptor files:

/vmfs/volumes/vsanDatastore/51108952-6e91-b30b-a5ab-005056ad9acf/VSAN-VM-1.vmdk
/vmfs/volumes/vsanDatastore/51108952-6e91-b30b-a5ab-005056ad9acf/VSAN-VM-1_1.vmdk

You can just grep for the keyword "vsan" by using the following command (replacing the path of your VMDKs):

grep "vsan" /vmfs/volumes/vsanDatastore/51108952-6e91-b30b-a5ab-005056ad9acf/VSAN-VM-1.vmdk

From the output you will see vsan:// and UUID associated with each VMDK, please make a note of the UUID for each VMDK. We are now ready to query the VM Storage Policy configuration which will help us rebuild the policy in our new vCenter Server.

Step 3 - To look up the VM Home VM Storage Policy, run the following command and specify the UUID of the VM Home in Step 1:

cmmds-tool find -t POLICY -u 51108952-6e91-b30b-a5ab-005056ad9acf -f json

The VM Storage Policy configurations is stored in the "content" field and you will need to translate the properties back to the VSAN policy you have defined. As part of the output you will also see a property called spbmProfileId which is the unique identifier for VM Storage Policy which you can query if you are using the VM Storage Policy APIs that were introduced in vSphere 5.5.

Here is a table that will help you translate the keys to the apporopirate VSAN Policies:

VSAN Capability Description VSAN Capability Key
Number of failures to tolerate hostFailuresToTolerate
Number of disk stripes per object stripeWidth
Force provisioning forceProvisioning
Object space reservation proportionalCapacity
Flash read cache reservation cacheReservation

Step 4 - To lookup the VMDK VM Storage Policies, we will perform the same command and just replace the UUID with our VMDK UUIDs.

Once you have recorded the configurations for each of the VM Storage Policy, you can then head over to your new vCenter Server and re-create the VM Storage Policies and then re-associate the policy with the Virtual Machines.

As you can see the steps to recover a VSAN VM Storage Policy is not too difficult but can be a bit tedious. In the next article, we will simplify this by leveraging the vSphere API which has access to the same CMMDS system but make querying the VM Storage Policy super easy by only requiring the user to provide the name of the Virtual Machine.

How cool is that!? Using VMware Workstation to manage your ESXi hosts (including Free ESXi) & VMs

by // 9 Comments

To be completely honest, I have not played with VMware Workstation in quite awhile as my day-to-day job primarily revolves around our Enterprise suite of products. In a recent meeting that I was in, I picked up on some interesting tidbits about the latest version of VMware Workstation 10 and after giving it a try in my lab, I thought I would share one very cool feature that you may be aware of (there is actually a lot of cool features in latest release, check what's new here).

The very first thing I noticed is that unlike other downloads from VMware in which you need to register the product and get an evaluation key. VMware Workstation can be downloaded without any registration and you can start the 30-day free trial immediately after installation! I think that is a really slick and can also come in handy if you need to install Workstation right away for something. Make sure you download from this page here by clicking on "Try for Free" instead of going to www.vmware.com/downloads

One of the capabilities that Workstation introduced probably a couple of releases ago was the ability to connect to a remote system whether that is another Workstation instance, vCenter Server and even an ESXi host. At the time I assumed this was to enable users to easily cold migrate a Virtual Machine that was created locally onto one of these remote targets.

What I did not realize was that you could do a lot more with this capability than to just copy offline Virtual Machines. To my surprise I found that you could fully manage the Virtual Machines on these remote targets including changing the virtual hardware configurations such as adding memory, cpu, disk, etc. guestOS as well as provision new Virtual Machines. The VM Console is fully functional leveraging VMRC and you can even connect to Free ESXi instances and get same capabilities you had with the legacy vSphere C# Client.  The other neat thing about this is you can also manage your Virtual Hardware 10 VMs even though the latest vSphere C# Client does not allow this because VMware Workstation 10 is vHW10 aware.

Here is a screenshot of managing my Free ESXi host which is running on my Apple Mac Mini as well as my vCenter Server. As you can see you can have multiple connections open up which is quite useful, especially if you have a couple of Free ESXi hosts in which you would like a single pane of glass to manage.

Another nice feature is the amount of backwards capability it provides for vSphere. You can go as far back as vSphere 4.1 (vCenter Server & ESXi). To prove this in my environment, I provisioned a Nested ESXi running on vSphere 4.1, 5.0, 5.1 and 5.5 and connected them all to Workstation. This is another great way to manage standalone ESXi hosts if you still need to run older versions.

Lastly, you do not need to be running the Windows version of VMware Workstation to get these benefits. You can also do the same using Workstation for Linux and here is a screenshot of running Workstation on an Ubuntu desktop.

As you can see this is just one of many new and cool capabilities of VMware Workstation 10 and I have to say for $250, this is a steal to be able to easily manage not only your VMs running locally but also remote systems like vCenter Server, ESXi hosts including Free ESXi which is a huge deal IMHO. The Workstation team really knocked it out of the park and I am glad I had the opportunity to check out their latest release. I also hope VMware Fusion will be getting these capabilities in the near future! Simon, I hope you see this 😉