WilliamLam.com

Scripts to Extract vCloud Director Chain Length and Linked Clone Mappings

by // 18 Comments

There were some questions this week about extracting the chain length for a virtual machine as well as figuring out which Linked Clones were mapped to a particular vAppTemplate and/or shadowVM in vCloud Director. If you are not familiar with the concept of a Linked Clone (Fast Provisioning in vCloud Director 1.5 leverages Linked Clones) and their relationship to chain length, there is a very good white paper called VMware vCloud Director 1.5 Performance and Best Practices that goes into detail about Linked Clones and some of the things to be aware of from a performance standpoint.

Depending on how you deploy a new vApp whether that is from a vAppTemplate in your catalog or copying from an existing vApp, the chain length of a Linked Clone tree will grow either breadth wise or length wise.
The chain length can grow much faster when copying from an existing vApp and that can impact the performance of your virtual machines. Below is a diagram between deploying from a vAppTemplate in a catalog (breadth wise) and consecutive copies from an existing vApp (length wise).

vCloud Director limits the chain length to 30 and automatically spins up a full clone (called a shadow VM) when the maximum is hit and all sub-sequent Linked Clones will be based off of this new shadow VM. A shadow VM can also spin up automatically if the current datastore is getting to full capacity or if the VM needs to exists on a different vCenter Server as Linked Clones do not span datastores or different vCenter Servers.

To view the current chain length of a given virtual machine, you will need to login with a System Administrator account and this can be seen using the vCloud UI at the VM level. You can also get the chain length using the vCloud API which is part of the VCloudExtension property called VirtualDisksMaxChainLength and can be viewed by performing a GET operation on a VM.

Note: Though the property has "MaxChainLength" in the name, this property represents the current chain length and it is not configurable, as the system max is 30.

To be able to quickly get a summary of the chain length of all VMs that include vAppTemplates and Shadow VMs, I wrote a vCloud SDK for PHP script called vcloudVMChainLength.php. The script provides a few pieces of information: VM name as shown in vCloud Director, name of the vApp the VM is part of in vCloud Director, whether it is a vAppTemplate, the vCenter Server hosting the VM, the MoRef of the VM and chain length.

Before I provide the sample output, let me give you some background about my lab environment. I have two vAppTemplates called "Application Server" and "Database Server". The "Application Server" is currently residing on a datastore with close to full capacity and I will be deploying 3 new vApps from this vAppTemplate called "App-Test-1", "App-Test-2" and "App-Test-3". I will also be deploying 3 new vApps from "Database Server" but only the first clone will be from the vAppTemplate, the remainder will be a clone of the previous vApp in the workspace.

Here is a diagram of the 9 VMs (2 vAppTemplates, 1 shadow VM and 6 vApps):

Here is the sample output of the environment above:

We can see that during the deployment of the "App-Test-*" vApps, the datastore capacity was low and vCloud Director automatically spun up a shadow VM which is a full copy and created the 3 new Linked Clones off of that image and the chain length is max of 2. With the "DB-Test-*", we started out with deploying from the vAppTemplate, but all subsequent copies was from the previous vApp which grew the chain length to 4. If we continue to copy from the vApp length wise, we will quickly reach the maximum chain length and possibly degrade the performance of the VMs as the deepest node will need to traverse back to the parent to perform it's disk read operations. 

The other question I had was about mapping the Linked Clones back to their parent VMs. This is not something you can really do in vCloud Director nor using the vCloud API. However, with a little bit of creativity and information from vCloud Director, you can leverage the vSphere API to help you get a list of VMs that are linked off of a given VM. I wrote a vSphere SDK for Perl script called vCloudVMLinkedClonesMapping.pl that can help you get this information. 

Note: This was tested in my lab which had a minimal configuration, ensure you do additional validation before making any decisions based on the output.

To use the script, you will need to provide the MoRef ID for a VM that is either a vAppTemplate, shadow VM or just a regular VM which may have Linked Clones. You can get this information by using the script that was shown earlier.

Let's take a look at our "Application-Server" VM which has MoRef ID 677 and see if there are any Linked Clones:

As we expected, there are no Linked Clones as the datastore it is currently residing on is getting to full capacity and a shadow VM was spun up for the new 3 deployments.

Let's take a look at the shadow VM "shadow-App-VM" with MoRef ID 721 which is automatically prefix with "shadow-" of the source VM name:

Just as we expected, we can see that our 3 vApps are linked off of the Shadow VM that we just checked.

Note: The output includes the display name and URN ID of the vApp in vCloud Director so you can easily identify it.

Finally, let's take a look at our "Database-Server" VM which has MoRef ID 675 and we should also see the 3 Linked Clones:

You might ask if it is necessary to keep track of all this information and the answer is yes/no. Though it is good to understand how vApps are being deployed by your consumers and ensure they are efficiently doing so by deploying from a catalog. vCloud Director has built in mechanisms to automatically handle deep chain lengths or low datastore space by deploying additional shadow VMs to ensure users are still able to request new vApps without any impact.

Org vDC to vCenter Resource Pool Workflow Using vCenter Orchestrator

by // Leave a Comment

I was helping a colleague of mind this evening with a question about retrieving a vCenter Resource Pool given a vCloud Director Organization vDC using vCenter Orchestrator. However, this particular workflow does not exists out of the box with vCO, but with a little help from the vCloud API, we can easily create our own workflow to accomplish this request. We will be leveraging the Query Service introduced in vCloud Director 1.5 and the "orgVdcResourcePoolRelation" query type which provides a mapping between an Org vDC to vCenter Resource Pool.

You can download the vCO workflow that I created called Get Org vDC to Resource Pool Mapping and import it into your vCO environment. You will need to make sure you have the vCloud Director vCO plugin installed.

Here is a example of running the workflow which accepts a vCloud Director Org vDC:

Here is the results of the workflow:

You will notice that it produces the MoRef (Managed Object Reference) to a vCenter Resource Pool instead of the actual Resource Pool object. The reason for this is the query only returns the href of the Org vDC, href of vCenter Server and the MoRef of the Resource Pool. Using the MoRef, you can connect to your vSphere environment and retrieve the Resource Pool, but I will leave that as an exercise for my colleague 🙂

Note: If you go through the query types, you may have noticed the resourcePool query type, the reason this will not work is that it only provides a list of root Resource Pools (basically vSphere Clusters) and it does not return the sub-resource pools that are created for Organization vDCs.

How to Create Bootable ESXi 5 ISO & Specifying Kernel Boot Options

by // 21 Comments

This week I helped to answer a few questions about creating your own ESXi 5 bootable ISO along with automatically using a static IP Address when the custom ISO first boots up. Although all this information is available via the vSphere documentation, it may not always be easy to put all the pieces together and thought I share the steps for others to also benefit.

You will need access to a UNIX/Linux system and a copy of the base ESXi 5 ISO image. In this example I will be using VMware vMA and VMware-VMvisor-Installer-5.0.0.update01-623860.x86_64.iso and walk you through two different configurations. We will also be referencing the vSphere documentation Create an Installer ISO Image with a Custom Installation or Upgrade Script and Kernel Boot Options.

Create ESXi 5 Bootable ISO w/Remote ks.cfg:

In this configuration, we will create a custom ESXi ISO that will boot with a static IP Address and use a remote ks.cfg (kickstart) configuration file.

Step 1 - Mount base ESXi ISO using the "mount" utility:

$ mkdir esxi_cdrom_mount
$ sudo mount -o loop VMware-VMvisor-Installer-5.0.0.update01-623860.x86_64.iso esxi_cdrom_mount

Step 2 - Copy the contents of the mounted image to a local directory called "esxi_cdrom":

$  cp -r esxi_cdrom_mount esxi_cdrom

Step 3 - Unmount the ISO after you have successfully copied it and change into the esxi_cdrom directory

$ sudo umount esxi_cdrom_mount
$ cd esxi_cdrom

Step 4 - Edit the boot.cfg and specifically the "kernelopt" line to not use the weasel installer but kickstart and also specifying the remote location of your ks.cfg. To get more details on the various kernel boot options, please take a look at the vSphere Boot Options documentation above.

You will also need to specify the static IP Address you wish to have the host automatically use when the ISO first boots up on the same line.

Step 5 - Once you have finished your edits and saved the boot.cfg, you will now change back to the parent directory and use the "mkisofs" to create your new bootable ISO. In this example, we will name the new ISO "custom_esxi.iso":

$ sudo mkisofs -relaxed-filenames -J -R -o custom_esxi.iso -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table esxi_cdrom/

You now have a new bootable ESXi 5 ISO called "custom_esxi.iso" which will now automatically boot up with the specified static IP Address and install based on the ks.cfg that was specified.

Create ESXi 5 Bootable ISO w/Local ks.cfg:

Similar to the above configuration, we will create a custom ESXi ISO that will boot with a static IP Address but use a local ks.cfg (kickstart) configuration file that will be included within the custom ISO.

Step 1 through 3 is exactly the same as above

Step 4 - By default, a basic ks.cfg is included in the ESXi 5 ISO in /etc/vmware/weasel/ks.cfg and we will create a custom *.tgz file that will include our ks.cfg within the ISO. First off by creating a temporary directory which will be used to store our ks.cfg:

$ mkdir -p temp/etc/vmware/weasel

Step 5 - Copy your ks.cfg file into the temp/etc/vmware/weasel:

$ cp ks_custom.cfg temp/etc/vmware/weasel

Step 6 - Create a *.tgz file containing the path to our ks.cfg using the "tar" utility. In this example, we will called it customks.tgz:

$ cd temp
$ tar czvf customks.tgz *

Step 7 -  Copy the customks.tgz from temp directory to your esxi_cdrom directory:

$ cp temp/customks.tgz esxi_cdrom

Step 8 -  Change into the "esxi_cdrom" directory and edit the boot.cfg just like the above, but we will be using the "file://" stanza to specify the path to our ks.cfg, static IP Address as well as adding our customks.tgz to the module list to ensure that it loads up which contains the actual ks.cfg file that is called in the boot.cfg.

Step 9 - Same as Step 5 above, you now just need to run the "mkisofs" utility to create your bootable ISO.

You now have a new bootable ESXi 5 ISO called "custom_esxi.iso" which will now automatically boot up with the specified static IP Address and install based on the ks.cfg that is included within the ISO.