WilliamLam.com

How to Create an SE Sparse (Space-Efficient) Disk in vSphere 5.1

by // 8 Comments

You probably may have heard, that with the upcoming release of vSphere 5.1, a new virtual machine disk format will be introduced called called SE Sparse (Space-Efficient). One of it's features is to provide the ability to reclaim unused blocks from within the guestOS. I would highly recommend you check out a recent blog post vSphere 5.1 Storage Enhancements – Part 2: SE Sparse Disks by Cormac Hogan for more details about the new SE Sparse disk format as well as other storage improvements in vSphere 5.1.

As Cormac points out, this new disk format will initially be leveraged by VMware View (in a future release from my understanding), as there are additional integrations required to use this feature than just using the new SE Sparse disk format. Having said that, the SE Sparse disk format is a feature of the vSphere 5.1 platform and with that, you do have the ability to create an SE Sparse disk.

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Please test this in a development environment before using it on actual systems.

There are two methods in which you can create an SE Sparse disk, directly on the ESXi Shell of an ESXi 5.1 host or remotely connecting to an ESXi 5.1 host.

Option 1 - Using vmkfstools on ESXi Shell 

Though it may not be documented, you can easily create a new VMDK with the new SE Sparse disk format by running the following command (10GB disk in this example):

vmkfstools -c 10g -d sesparse WindowsXP.vmdk

Here is a screenshot of new SE Sparse disk descriptor file to prove we have successfully created a new VMDK using the new format:

Option 2 - Using vSphere 5.1 API w/modified remote version of vmkfstools

As mentioned, the SE Sparse disk format is a feature of the vSphere 5.1 platform and as so, you can also leverage the vSphere 5.1 API to create a new VMDK using the virtualDiskManager and specifying the new SeSparseVirtualDiskSpec.

Note: Even though the vSphere API reference mentions the ability to set grain size via grainSizeKb property, I have found that it is not possible and just leaving it blank will automatically default to 1024K (1MB) which might be a system default for now.

You can download the modified version of the remote vmkfstools called vmkfstools-lamw which requires the the installation of vCLI 5.1 or vMA 5.1.

Here is an example of creating the same 10GB VMDK using the new SE Sparse disk format:

./vmkfstools-lamw --server 172.30.0.187 --username root -c 10G -d sesparse "[datastore1] WindowsXP.vmdk"

After you have created your new SE Sparse disk, the next logical step is assign it to a virtual machine. Since this is a new feature in vSphere 5.1, you will need to use the new vSphere Web Client to perform the operation as the legacy vSphere C# Client is not aware of this new disk type. You will also need to ensure that the virtual machine is running the latest ESXi 5.1 compatibility and later (virtual hardware version 9).

Once you have added our newly created disk from the datastore, it should now show up in the vSphere Web Client as Flex-SE for the disk type.

Additional Resources:

 

Enabling/Disabling EVC using the vSphere MOB

by // 2 Comments

There were some discussions this morning on twitter regarding the configuration of EVC for a vSphere Cluster using one of the vSphere CLI's such as PowerCLI or directly leveraging the vSphere API. Unfortunately, this is not possible today as the operations pertaining to EVC are not currently exposed in the vSphere API. This means you will not be able to use the vCLI, PowerCLI, vCO or the vSphere API to configure and manage EVC configurations, you will need to use the vSphere Client to do so.

Having said that, one could still "potentially" automate EVC configurations using the vSphere MOB interface using the private vSphere API, but it may not be ideal and will require some "creativity" and custom coding to integrate with your existing automation solution. This particular limitation of the vSphere API is one that I have personally faced and have filed a bug with VMware awhile back. I am hoping this will eventually be added to the public vSphere API, so that users can fully automate all aspects and configurations of a vSphere Cluster.

Disclaimer: This is not officially supported by VMware, use at your own risk and discretion.

Step 1 - Connect to your vCenter MOB and traverse to the vSphere Cluster of interest (note the MOID will be different in your specific cluster).

Step 2 -  Now replace the URL with the following while substituting the cluster MOID that you see in your browser:

https://reflex.primp-industries.com/mob/?moid=domain-c1550&method=transitionalEVCManager

and hit enter and you'll be brought to TransitionalEVCManager() method, you'll then want to click on the "Invoke Method". Once you do so, you should be returned with a task object and you'll have a link to something like evcdomain-cXXXX. Click on this and you'll be brought to ClusterTransitionalEVCManager.

Step 3 - From here you'll have have some basic evcState information which you can click on to see what the current EVC configuration is set to, guaranteedCPUFeatures and valid EVC Modes (the last part will be important for reconfiguring EVC)

Step 4 - Now let's say the cluster currently has EVC Mode set to intel-merom and you would like to change it to Nehalem, you would need to retrieve the key from the previous page, in our example it's intel-nehalem. Now, you need to click on the method link called ConfigureEVC_Task which is pretty straight forward, it just accepts the EVC Mode Key, enter the string and click on "Invoke Method" and now your cluster will be reconfigured if you go back to the evcState or look at your vCenter task. You can also disable EVC by using DisableEVC_Task

 
Note: If EVC is already configured in your vSphere Cluster, you can use the vSphere API to view it's current configuration by looking at the ClusterComputeResource's summary property. You just will not be able to make any changes or disabling EVC using the vSphere API.

VM Provisioning on Datastore Clusters in vSphere 5

by // 24 Comments

Last year I wrote an article called Automating Storage DRS & Datastore Cluster Management in vSphere 5 and I provided a pretty comprehensive vSphere SDK for Perl script to help administrators automate Storage DRS configurations. These past few months I have noticed an increase in interest on the VMTN developer forums relating to Storage DRS. Majority of the questions has been related to which vSphere API methods to use and how to use these methods for cloning VMs to datastore clusters.

If you have cloned a VM before, the underlying vSphere API method being used is the CloneVM_Task(), but when cloning a VM to a datastore cluster, a different API must be used.

In vSphere 5, VMware introduced several new API methods in the StorageResourceManager and the two specific ones relating to provisioning VMs are RecommendDatastores() and ApplyStorageDrsRecommendation_Task(). The process to clone a VM to a datastore cluster is a two step process:

  1. Call RecommendDatastores() which accepts very similar input as the CloneVM_Task(). In addition, you need to specify the datastore cluster also known as a Storage Pod in the vSphere API as well as the "type" (create, clone, relocate, reconfigure), in this example we will be performing a clone operation. This method will then generate a recommendation on where to place the VM which is based on the SDRS placement engine. No provisioning will occur at this point, just a placement recommendation.
  2. To perform the actual provision of the VM, you will need to call ApplyStorageDrsRecommendation_Task() which accepts a recommendation ID that was generated from the first step. Once the recommendation is applied, the provisioning of the VM will start just like it did when you called the CloneVM_Task().

Note: The RecommendDatastores() will return multiple recommendations, the best one will be first entry in the array. This is the same algorithm used when performing this same operation in the vSphere Client, it also selects the first recommendation.

Now that we understand how the APIs work, let's take a look at how we can leverage this in a script for some automation! Here is a simple vSphere SDK for Perl script called datastoreClusterVMProvisioning.pl which allows you to clone an existing VM onto a datastore cluster. You will need a system that has vCLI 5.0 installed or you can use VMware vMA 5 to run the script. You will also need to connect to a vCenter Server 5 for all SDRS operations.

The script requires 4 input parameters:

  • vmname - Name of the VM you wish to clone from
  • clonename - Name of the cloned VM
  • vmfolder - Name of a vCenter folder
  • datastorecluster - Name of a datastore cluster

Here is a screenshot of cloning an existing VM onto a datastore cluster:

The script is pretty straight forward and it can easily be adapted to include other configurations as required in your own environment.

Hopefully this gives you a better idea on how to leverage the new provisioning APIs for Storage DRS and start automating your VM deployments onto datastore clusters and get the benefits Storage DRS in your vSphere environment.