WilliamLam.com

VM Provisioning on Datastore Clusters in vSphere 5

by // 24 Comments

Last year I wrote an article called Automating Storage DRS & Datastore Cluster Management in vSphere 5 and I provided a pretty comprehensive vSphere SDK for Perl script to help administrators automate Storage DRS configurations. These past few months I have noticed an increase in interest on the VMTN developer forums relating to Storage DRS. Majority of the questions has been related to which vSphere API methods to use and how to use these methods for cloning VMs to datastore clusters.

If you have cloned a VM before, the underlying vSphere API method being used is the CloneVM_Task(), but when cloning a VM to a datastore cluster, a different API must be used.

In vSphere 5, VMware introduced several new API methods in the StorageResourceManager and the two specific ones relating to provisioning VMs are RecommendDatastores() and ApplyStorageDrsRecommendation_Task(). The process to clone a VM to a datastore cluster is a two step process:

  1. Call RecommendDatastores() which accepts very similar input as the CloneVM_Task(). In addition, you need to specify the datastore cluster also known as a Storage Pod in the vSphere API as well as the "type" (create, clone, relocate, reconfigure), in this example we will be performing a clone operation. This method will then generate a recommendation on where to place the VM which is based on the SDRS placement engine. No provisioning will occur at this point, just a placement recommendation.
  2. To perform the actual provision of the VM, you will need to call ApplyStorageDrsRecommendation_Task() which accepts a recommendation ID that was generated from the first step. Once the recommendation is applied, the provisioning of the VM will start just like it did when you called the CloneVM_Task().

Note: The RecommendDatastores() will return multiple recommendations, the best one will be first entry in the array. This is the same algorithm used when performing this same operation in the vSphere Client, it also selects the first recommendation.

Now that we understand how the APIs work, let's take a look at how we can leverage this in a script for some automation! Here is a simple vSphere SDK for Perl script called datastoreClusterVMProvisioning.pl which allows you to clone an existing VM onto a datastore cluster. You will need a system that has vCLI 5.0 installed or you can use VMware vMA 5 to run the script. You will also need to connect to a vCenter Server 5 for all SDRS operations.

The script requires 4 input parameters:

  • vmname - Name of the VM you wish to clone from
  • clonename - Name of the cloned VM
  • vmfolder - Name of a vCenter folder
  • datastorecluster - Name of a datastore cluster

Here is a screenshot of cloning an existing VM onto a datastore cluster:

The script is pretty straight forward and it can easily be adapted to include other configurations as required in your own environment.

Hopefully this gives you a better idea on how to leverage the new provisioning APIs for Storage DRS and start automating your VM deployments onto datastore clusters and get the benefits Storage DRS in your vSphere environment.

Automating Dead Space Reclamation in ESXi 5.0u1

by // 4 Comments

VMware released vSphere 5.0 Update 1 last week, which mainly included bug fixes but it also brought back one very cool feature that was initially introduced with the release of vSphere 5.0 called Thin Provisioning UNMAP primitive for an ESXi host. You can read more about the details in this article by my colleague Cormac Hogan.

As you can see from From Cormac's article, the process of reclaiming of dead space on a thin provisioned LUN is currently a manual process, but does it have to be? The answer is, No of course, we can can definitely automate this!

Disclaimer: This script is not officially supported by VMware, please test this in a development environment before using on production system. This script is provided as an example on you can automate this manual process.

Before you proceed, please understand that the UNMAP operation can potentially take a few minutes up to a few hours depending on the size of your datastore and how your array handles this operation. You should consider performing this operation during a maintenance window or during off peak hours else you could impact VMs residing on the datastore. You should also ensure you have a VAAI-capable storage array before performing running this script.

I wrote a simple shell script called reclaimMyDeadsSace.sh which needs to be executed on the ESXi Shell via SSH. The script will also perform some validation such as ensuring you are running ESXi 5.0 Update 1 and that your host is in maintenance mode as a per-caution to ensure no running VMs are on the host during this process.

You will only need to run the script on one of the hosts connected to all the datastores you wish to reclaim dead space on. You may use scp or WinSCP to transfer the script to your ESXi host and ensure you set the execute permission on the script (chmod +x reclaimMyDeadSpace.sh)

The script can be executed in two ways:

  1. Identify ALL VMFS3 and VMFS5 volumes and perform the reclaim based on the percentage entered by the user
  2. Reclaim on specific datastores specified by the user as well as the percentage to be reclaimed (this is recommended, that way script does not choose all datastores including local ones)

Here is an example of selecting ALL VMFS3 and VMFS5 datastores to reclaim 60% of free space:

Here is an example of selecting just 4 datastores specified in a file and we will be reclaiming 60% of free space:

In this example, we created a file called "datastore_list.txt" (you may name the file anything you want) which contains the following:
iSCSI-1
iSCSI-2
iSCSI-3
iSCSI-4

So if you are using thin provisioned LUNs and would like to reclaim some of that dead space back and have a VAAI-capable storage array, be sure to check out the UNMAP functionality in ESXi 5.0u1.

How to Run WSX as a Standalone

by // 3 Comments

This weekend I got chance to deploy the new Workstation Technology Preview 2012 in my lab and specifically play with the new WSX feature, which allows you to access your virtual machines from anywhere with just a browser. Currently WSX is only available for the Linux version of Workstation and is bundled together as part of the installer. I wanted to run WSX in one of my management VMs, and did not want the large disk footprint that came with Workstation. I did some digging and found it was quite easy to extract the WSX bits and run it on another Linux system, and in my case I tried it with vMA.

Disclaimer: This is mainly for educational and testing purposes as this is not officially supported by VMware.

The main prerequisite to install WSX is a Linux system that has Python 2.6 installed. You will still need to perform a full installation of Workstation to extract the WSX components, as recommended you can use the latest Ubuntu image.

Note: If you want to install Workstation Tech Preview in a VM, you may get an error for the version of VMware Tools not being up to date. You can by-pass that by running the following command:

VMWARE_FORCE_INSTALL_IN_VM=yes ./VMware-Workstation-Full-e.x.p-646643.i386.bundle

Step 1 - You will need to create a few directories on the destination system in which you will be copying the WSX files to:

mkdir -p /etc/vmware/wsx
mkdir -p /usr/lib/vmware/{setup,scripts,lib,bin}
mkdir -p /var/lib/vmware/wsx/

Step 2 - You will now copy the following directory/files to destination system using scp:

scp /usr/lib/vmware/bin/vmware-wsx-server *protected email*:/usr/lib/vmware/bin
scp /etc/init.d/vmware-wsx-server *protected email*:/etc/init.d
scp /etc/vmware/bootstrap  *protected email*:/etc/vmware
scp -r /usr/lib/vmware/setup *protected email*:/usr/lib/vmware/
scp -r /usr/lib/vmware/scripts *protected email*:/usr/lib/vmware/
scp -r /usr/lib/vmware/lib/python2.6 *protected email*:/usr/lib/vmware/lib

Step 3 - Next you need to re-create the WSX config file which will be stored in /etc/vmware/wsx/config using the following command:

/usr/lib/vmware/lib/python2.6/site-packages/wsx/vmware-wsx-server --generate_config

If you wish to change the default port of 8888, you may edit the file before starting the WSX service.

Step 4 - Finally, you are now ready to start the WSX service by running the following command:

/etc/init.d/vmware-wsx-server start

Note: I ran into an odd issue with the initial login to WSX from the browser, in which I needed to create a secondary account other than the default "vi-admin". You need to login with "vi-admin" first, clear the cookie, so you can login with another user account before you add new servers. This was mainly looking at some of the errors from the logs and performing sqlite dump of WSX db.

Here are a few screenshot of accessing WSX from browser, iPad and iPhone:

The interface was pretty easy to use and it's pretty damn cool to be able to access your desktop from any platform that has a browser! Really looking forward to see where WSX is headed and hopefully it will be available in the future as a standalone installer and also with a logout button 🙂