WilliamLam.com

Which VM was this vSphere VM cloned from?

by // 2 Comments

This was a question that I saw back in December on the VMware {code} Slack which was quickly answered by the always awesome Luc Dekens. The solution is to look at vCenter Server Events, which are super rich in information and can be used for a number of things including identifying the source VM that it was cloned from. When I was a customer, this was something I did all the time, using events for auditing purposes but also identifying who, what and when a certain operation was performed including source VMs for cloning operations.

Although this information maybe known to some, there is still not an elegant solution that can help someone quickly identify the source VM for a specific vSphere VM that was cloned. This topic also intrigued me as I have seen this question come up in the past. I figure I might as well add this to my random scripting backlog and take a look when I had some time.

Before taking a look at the solution, it is important to understand the different types of clones that exists in vSphere today and also the respective vCenter Server events that can help us correlate to both the source VM but also the specific clone type.

Cloning Types

  • Full Clone - An independent copy of a virtual machine that shares nothing with the parent virtual machine after the cloning operation. Ongoing operation of a full clone is entirely separate from the parent virtual machine
  • Linked Clone - A copy of a virtual machine that shares virtual disks with the parent virtual machine in an ongoing manner. This conserves disk space, and allows multiple virtual machines to use the same software installation
  • Instant Clone - An independent copy of a virtual machine that starts executing from the exact running state of the source powered on virtual machine. Instant Clone uses rapid in-memory cloning of a running parent virtual machine and copy-on-write, simliar to that of Linked Cloning to rapidly deploy virtual machines

[Read more...]

Creating SE Sparse Linked Clones Using the vSphere 5.1 API

by // 9 Comments

In my previous article, I showed you how you can easily create your own SE Sparse (Space-Efficient) disks for you virtual machines running on vSphere 5.1. In addition to just creating new virtual disks using the new disk format, you can also create SE Sparse Linked Clones by leveraging the vSphere 5.1 API. I had previously shared a vSphere SDK for Perl sample script called vGhettoLinkedClone.pl that allows you to create your own Linked Clones and I have updated the script to now support the new SE Sparse disk format.

The script now includes new parameter called --convert which allows you to specify whether or not the destination Linked Clone should follow the same source virtual disk format or the new SE Sparse disk format. Remember to upgrade your VMs to the latest ESXi 5.1 compatibility and later (virtual hardware version 9) if you wish to leverage the new SE Sparse disk format. The way this is accomplished is by specifying a new disk backing  VirtualDiskFlatVer2BackingInfo in the VirtualMachineRelocateSpec and passing in the deltaDiskFormat to be of type seSparseFormat.

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Please test this in a development environment before using it on actual systems.

Note: Linked Clones is a feature of vCenter Server, make sure you are connecting to a vCenter Server 5.1

If you have a VM that has a VMDK that is NOT an SE Sparse disk format but wish to create Linked Clones with the new SE Sparse disk format, then you can use the --convert sesparse option, here is an example command:

./vGhettoLinkedClone.pl --server vcenter51-1.primp-industries.com --username root --vmname Regular-VM --vmname_destination Linkedclone-SeSparse-Destination --snapname pristine --vmhost vesxi51-1.primp-industries.com --datastore datastore1 --convert sesparse

If you have a VM that has a VMDK that IS an SE Sparse disk format and you wish to create Linked Clones with the same disk format, then you can use the --convert source option, here is an example command:

./vGhettoLinkedClone.pl --server vcenter51-1.primp-industries.com --username root --vmname SeSparse-VM --vmname_destination Linkedclone-SeSparse-Destination --snapname pristine --vmhost vesxi51-1.primp-industries.com --datastore datastore1 --convert source

If we head over to our vSphere Web Client and take a look at our new Linked Clones, we can confirm their virtual disks is using the new SE Sparse disk format.

Scripts to Extract vCloud Director Chain Length and Linked Clone Mappings

by // 18 Comments

There were some questions this week about extracting the chain length for a virtual machine as well as figuring out which Linked Clones were mapped to a particular vAppTemplate and/or shadowVM in vCloud Director. If you are not familiar with the concept of a Linked Clone (Fast Provisioning in vCloud Director 1.5 leverages Linked Clones) and their relationship to chain length, there is a very good white paper called VMware vCloud Director 1.5 Performance and Best Practices that goes into detail about Linked Clones and some of the things to be aware of from a performance standpoint.

Depending on how you deploy a new vApp whether that is from a vAppTemplate in your catalog or copying from an existing vApp, the chain length of a Linked Clone tree will grow either breadth wise or length wise.
The chain length can grow much faster when copying from an existing vApp and that can impact the performance of your virtual machines. Below is a diagram between deploying from a vAppTemplate in a catalog (breadth wise) and consecutive copies from an existing vApp (length wise).

vCloud Director limits the chain length to 30 and automatically spins up a full clone (called a shadow VM) when the maximum is hit and all sub-sequent Linked Clones will be based off of this new shadow VM. A shadow VM can also spin up automatically if the current datastore is getting to full capacity or if the VM needs to exists on a different vCenter Server as Linked Clones do not span datastores or different vCenter Servers.

To view the current chain length of a given virtual machine, you will need to login with a System Administrator account and this can be seen using the vCloud UI at the VM level. You can also get the chain length using the vCloud API which is part of the VCloudExtension property called VirtualDisksMaxChainLength and can be viewed by performing a GET operation on a VM.

Note: Though the property has "MaxChainLength" in the name, this property represents the current chain length and it is not configurable, as the system max is 30.

To be able to quickly get a summary of the chain length of all VMs that include vAppTemplates and Shadow VMs, I wrote a vCloud SDK for PHP script called vcloudVMChainLength.php. The script provides a few pieces of information: VM name as shown in vCloud Director, name of the vApp the VM is part of in vCloud Director, whether it is a vAppTemplate, the vCenter Server hosting the VM, the MoRef of the VM and chain length.

Before I provide the sample output, let me give you some background about my lab environment. I have two vAppTemplates called "Application Server" and "Database Server". The "Application Server" is currently residing on a datastore with close to full capacity and I will be deploying 3 new vApps from this vAppTemplate called "App-Test-1", "App-Test-2" and "App-Test-3". I will also be deploying 3 new vApps from "Database Server" but only the first clone will be from the vAppTemplate, the remainder will be a clone of the previous vApp in the workspace.

Here is a diagram of the 9 VMs (2 vAppTemplates, 1 shadow VM and 6 vApps):

Here is the sample output of the environment above:

We can see that during the deployment of the "App-Test-*" vApps, the datastore capacity was low and vCloud Director automatically spun up a shadow VM which is a full copy and created the 3 new Linked Clones off of that image and the chain length is max of 2. With the "DB-Test-*", we started out with deploying from the vAppTemplate, but all subsequent copies was from the previous vApp which grew the chain length to 4. If we continue to copy from the vApp length wise, we will quickly reach the maximum chain length and possibly degrade the performance of the VMs as the deepest node will need to traverse back to the parent to perform it's disk read operations. 

The other question I had was about mapping the Linked Clones back to their parent VMs. This is not something you can really do in vCloud Director nor using the vCloud API. However, with a little bit of creativity and information from vCloud Director, you can leverage the vSphere API to help you get a list of VMs that are linked off of a given VM. I wrote a vSphere SDK for Perl script called vCloudVMLinkedClonesMapping.pl that can help you get this information. 

Note: This was tested in my lab which had a minimal configuration, ensure you do additional validation before making any decisions based on the output.

To use the script, you will need to provide the MoRef ID for a VM that is either a vAppTemplate, shadow VM or just a regular VM which may have Linked Clones. You can get this information by using the script that was shown earlier.

Let's take a look at our "Application-Server" VM which has MoRef ID 677 and see if there are any Linked Clones:

As we expected, there are no Linked Clones as the datastore it is currently residing on is getting to full capacity and a shadow VM was spun up for the new 3 deployments.

Let's take a look at the shadow VM "shadow-App-VM" with MoRef ID 721 which is automatically prefix with "shadow-" of the source VM name:

Just as we expected, we can see that our 3 vApps are linked off of the Shadow VM that we just checked.

Note: The output includes the display name and URN ID of the vApp in vCloud Director so you can easily identify it.

Finally, let's take a look at our "Database-Server" VM which has MoRef ID 675 and we should also see the 3 Linked Clones:

You might ask if it is necessary to keep track of all this information and the answer is yes/no. Though it is good to understand how vApps are being deployed by your consumers and ensure they are efficiently doing so by deploying from a catalog. vCloud Director has built in mechanisms to automatically handle deep chain lengths or low datastore space by deploying additional shadow VMs to ensure users are still able to request new vApps without any impact.