WilliamLam.com

A Pretty Cool Method of Upgrading to ESXi 5.1

by // 44 Comments

I recently came across an interesting article by Andreas Peetz which shows you how to patch an ESXi host using an image profile that is directly available on VMware's online depot within the ESXi shell. I knew that VMware had online depots for use with VUM and Auto Deploy but I was not aware of this particular method, especially directly from the host.

Disclaimer: This method assumes you can install the default ESXi Image Profile with no additional drivers or packages, else you may have connectivity issue after the upgrade. If you still need to customize the ESXi Image Profile before installation, you will still need to use something like Image Builder and then upload that to your online depot.

Note: There are many ways that you can patch/upgrade your ESXi hosts, here is another article that provides more details for command-line only methods.

Before you get started, you will need to make sure that your ESXi host has the httpClient firewall rule enabled, else you will not be able to connect to VMware's online depot. To enable this, run the following ESXCLI command:

esxcli network firewall ruleset set -e true -r httpClient

Also make sure that your ESXi host can reach the following URL (you can specify a proxy if needed):

https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

To view the available ESXi Image Profiles, run the following ESXCLI command (use the --proxy if you need to specify a proxy to reach VMware's online depot):

esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

If you are able to successfully connect to the online depot, you see a list of all the ESXi Image Profiles that are available to you. You will see two ESXi 5.1 Image Profiles (these were recently published), one with VMware Tools and one without VMware Tools.

Note: Before you begin, make sure you do not have any running VMs and put your host into maintenance mode.

Let's go ahead and upgrade our ESXi 5.0 Update 1 host to latest ESXi 5.1. To install the new Image Profile, run the following command:

esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.1.0-799733-standard

This can take a few minutes to complete depending on how fast you can pull down the Image Profile. Once it is done, you will see all the new VIBs that have been updated and you will be asked to reboot for the changes to go into effect and then you will be running ESXi 5.1! Pretty cool IMO!

How To Initiate a Wipe & Shrink Operation On an SE Sparse Based Disk

by // 6 Comments

In my previous two articles, I showed you how to create your own SE Sparse disks as well as creating new virtual machine Linked Clones leveraging the new SE Sparse disk format. If you recall earlier, one of the features of the SE Sparse disk format is to provide the ability to reclaim unused blocks within the guestOS which is a two step process: wipe and shrink.

Here is a screenshot that describes the process which was taken from the What's New In vSphere 5.1 Storage Whitepaper by my colleague Cormac Hogan. I highly recommend you check out the whitepaper which includes more details about this feature and other storage improvements in vSphere 5.1

The process of kicking off this wipe and shrink operation will be done through an integration with VMware View (a future release from my understanding). Now, it's important to understand that it's not just simply calling these two operations but it is also when they are called. The wipe operation is more CPU intensive as it scans for unused space within the guestOS filesystem and the shrink operation is more I/O intensive as it issues the SCSI unmaps commands. I can only assume that these operations will be scheduled based on the utilization of the guestOS to help reduce the impact to the VM workload.

Now having said that, since the SE Sparse disk format is a feature of the vSphere 5.1 platform, so are both the wipe and shrink operations. Though they are not exposed in the public vSphere API like the SE Sparse disk format, you can still access the private APIs if you know where to look 😉

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Use at your own risk.

With some help from my good friend the vSphere MOB and some digging, I have located the two vSphere API methods for wipe and shrink operation. Before getting started, ensure you have a VM with at least one SE Sparse disk, else these commands will not be very useful.

Note: In this experiment, I tested the wipe and shrink operation with Windows XP image, this may or may not work on other OSes.

First you will need to search for the VM in question and retrieve it's vSphere MOB URL which is in the format of https://[vcenter-server]/mob/?moid=vm-X where X is the MoRef ID for your VM. You can either navigate through the vSphere MOB or use my MoRef finder script.

Wipe Operation

To issue the wipe API, enter the following URL into your web browser (remember to replace the MoRef ID with the one of your VM)

https://[vcenter-server]/mob/?moid=vm-X&method=wipeDisk

Here is a screenshot of what that looks like if you are able to successfully access the private API:

Go ahead and click on "Invoke Method" which will then execute the wipe operation. If you take a look at the vSphere Web Client, you should now see a new task for the wipe operation.

This can take a bit of time as it scans through the guestOS filesystem for unused space.

Shrink Operation

Once the wipe operation as completed, we then need to issue the shrink API. Enter the following URL into your web browser (remember to replace the MoRef ID with tone of your VM)

https://[vcenter-server]/mob/?moid=vm-X&method=shrinkDisk

Here is a screenshot of what that looks like if you are able to successfully access the private API:

Here you can specify particular disks (requires diskId) that you wish to perform the shrink operation on. If you leave it blank, it will try to shrink all disks associated with your VM. In our example, I will shrink all disks. Go ahead and click on the "Invoke Method" which will kick off the shrink operation. If you go back to the vSphere Web Client, you should now see a shrink task in progress.

Again, this operation can also take some time, but once it has finished, then you have successfully reclaimed any unused blocks within your guestOS.

Creating SE Sparse Linked Clones Using the vSphere 5.1 API

by // 9 Comments

In my previous article, I showed you how you can easily create your own SE Sparse (Space-Efficient) disks for you virtual machines running on vSphere 5.1. In addition to just creating new virtual disks using the new disk format, you can also create SE Sparse Linked Clones by leveraging the vSphere 5.1 API. I had previously shared a vSphere SDK for Perl sample script called vGhettoLinkedClone.pl that allows you to create your own Linked Clones and I have updated the script to now support the new SE Sparse disk format.

The script now includes new parameter called --convert which allows you to specify whether or not the destination Linked Clone should follow the same source virtual disk format or the new SE Sparse disk format. Remember to upgrade your VMs to the latest ESXi 5.1 compatibility and later (virtual hardware version 9) if you wish to leverage the new SE Sparse disk format. The way this is accomplished is by specifying a new disk backing  VirtualDiskFlatVer2BackingInfo in the VirtualMachineRelocateSpec and passing in the deltaDiskFormat to be of type seSparseFormat.

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Please test this in a development environment before using it on actual systems.

Note: Linked Clones is a feature of vCenter Server, make sure you are connecting to a vCenter Server 5.1

If you have a VM that has a VMDK that is NOT an SE Sparse disk format but wish to create Linked Clones with the new SE Sparse disk format, then you can use the --convert sesparse option, here is an example command:

./vGhettoLinkedClone.pl --server vcenter51-1.primp-industries.com --username root --vmname Regular-VM --vmname_destination Linkedclone-SeSparse-Destination --snapname pristine --vmhost vesxi51-1.primp-industries.com --datastore datastore1 --convert sesparse

If you have a VM that has a VMDK that IS an SE Sparse disk format and you wish to create Linked Clones with the same disk format, then you can use the --convert source option, here is an example command:

./vGhettoLinkedClone.pl --server vcenter51-1.primp-industries.com --username root --vmname SeSparse-VM --vmname_destination Linkedclone-SeSparse-Destination --snapname pristine --vmhost vesxi51-1.primp-industries.com --datastore datastore1 --convert source

If we head over to our vSphere Web Client and take a look at our new Linked Clones, we can confirm their virtual disks is using the new SE Sparse disk format.