WilliamLam.com

Scripts to Extract vCloud Director Chain Length and Linked Clone Mappings

by // 18 Comments

There were some questions this week about extracting the chain length for a virtual machine as well as figuring out which Linked Clones were mapped to a particular vAppTemplate and/or shadowVM in vCloud Director. If you are not familiar with the concept of a Linked Clone (Fast Provisioning in vCloud Director 1.5 leverages Linked Clones) and their relationship to chain length, there is a very good white paper called VMware vCloud Director 1.5 Performance and Best Practices that goes into detail about Linked Clones and some of the things to be aware of from a performance standpoint.

Depending on how you deploy a new vApp whether that is from a vAppTemplate in your catalog or copying from an existing vApp, the chain length of a Linked Clone tree will grow either breadth wise or length wise.
The chain length can grow much faster when copying from an existing vApp and that can impact the performance of your virtual machines. Below is a diagram between deploying from a vAppTemplate in a catalog (breadth wise) and consecutive copies from an existing vApp (length wise).

vCloud Director limits the chain length to 30 and automatically spins up a full clone (called a shadow VM) when the maximum is hit and all sub-sequent Linked Clones will be based off of this new shadow VM. A shadow VM can also spin up automatically if the current datastore is getting to full capacity or if the VM needs to exists on a different vCenter Server as Linked Clones do not span datastores or different vCenter Servers.

To view the current chain length of a given virtual machine, you will need to login with a System Administrator account and this can be seen using the vCloud UI at the VM level. You can also get the chain length using the vCloud API which is part of the VCloudExtension property called VirtualDisksMaxChainLength and can be viewed by performing a GET operation on a VM.

Note: Though the property has "MaxChainLength" in the name, this property represents the current chain length and it is not configurable, as the system max is 30.

To be able to quickly get a summary of the chain length of all VMs that include vAppTemplates and Shadow VMs, I wrote a vCloud SDK for PHP script called vcloudVMChainLength.php. The script provides a few pieces of information: VM name as shown in vCloud Director, name of the vApp the VM is part of in vCloud Director, whether it is a vAppTemplate, the vCenter Server hosting the VM, the MoRef of the VM and chain length.

Before I provide the sample output, let me give you some background about my lab environment. I have two vAppTemplates called "Application Server" and "Database Server". The "Application Server" is currently residing on a datastore with close to full capacity and I will be deploying 3 new vApps from this vAppTemplate called "App-Test-1", "App-Test-2" and "App-Test-3". I will also be deploying 3 new vApps from "Database Server" but only the first clone will be from the vAppTemplate, the remainder will be a clone of the previous vApp in the workspace.

Here is a diagram of the 9 VMs (2 vAppTemplates, 1 shadow VM and 6 vApps):

Here is the sample output of the environment above:

We can see that during the deployment of the "App-Test-*" vApps, the datastore capacity was low and vCloud Director automatically spun up a shadow VM which is a full copy and created the 3 new Linked Clones off of that image and the chain length is max of 2. With the "DB-Test-*", we started out with deploying from the vAppTemplate, but all subsequent copies was from the previous vApp which grew the chain length to 4. If we continue to copy from the vApp length wise, we will quickly reach the maximum chain length and possibly degrade the performance of the VMs as the deepest node will need to traverse back to the parent to perform it's disk read operations. 

The other question I had was about mapping the Linked Clones back to their parent VMs. This is not something you can really do in vCloud Director nor using the vCloud API. However, with a little bit of creativity and information from vCloud Director, you can leverage the vSphere API to help you get a list of VMs that are linked off of a given VM. I wrote a vSphere SDK for Perl script called vCloudVMLinkedClonesMapping.pl that can help you get this information. 

Note: This was tested in my lab which had a minimal configuration, ensure you do additional validation before making any decisions based on the output.

To use the script, you will need to provide the MoRef ID for a VM that is either a vAppTemplate, shadow VM or just a regular VM which may have Linked Clones. You can get this information by using the script that was shown earlier.

Let's take a look at our "Application-Server" VM which has MoRef ID 677 and see if there are any Linked Clones:

As we expected, there are no Linked Clones as the datastore it is currently residing on is getting to full capacity and a shadow VM was spun up for the new 3 deployments.

Let's take a look at the shadow VM "shadow-App-VM" with MoRef ID 721 which is automatically prefix with "shadow-" of the source VM name:

Just as we expected, we can see that our 3 vApps are linked off of the Shadow VM that we just checked.

Note: The output includes the display name and URN ID of the vApp in vCloud Director so you can easily identify it.

Finally, let's take a look at our "Database-Server" VM which has MoRef ID 675 and we should also see the 3 Linked Clones:

You might ask if it is necessary to keep track of all this information and the answer is yes/no. Though it is good to understand how vApps are being deployed by your consumers and ensure they are efficiently doing so by deploying from a catalog. vCloud Director has built in mechanisms to automatically handle deep chain lengths or low datastore space by deploying additional shadow VMs to ensure users are still able to request new vApps without any impact.

Org vDC to vCenter Resource Pool Workflow Using vCenter Orchestrator

by // Leave a Comment

I was helping a colleague of mind this evening with a question about retrieving a vCenter Resource Pool given a vCloud Director Organization vDC using vCenter Orchestrator. However, this particular workflow does not exists out of the box with vCO, but with a little help from the vCloud API, we can easily create our own workflow to accomplish this request. We will be leveraging the Query Service introduced in vCloud Director 1.5 and the "orgVdcResourcePoolRelation" query type which provides a mapping between an Org vDC to vCenter Resource Pool.

You can download the vCO workflow that I created called Get Org vDC to Resource Pool Mapping and import it into your vCO environment. You will need to make sure you have the vCloud Director vCO plugin installed.

Here is a example of running the workflow which accepts a vCloud Director Org vDC:

Here is the results of the workflow:

You will notice that it produces the MoRef (Managed Object Reference) to a vCenter Resource Pool instead of the actual Resource Pool object. The reason for this is the query only returns the href of the Org vDC, href of vCenter Server and the MoRef of the Resource Pool. Using the MoRef, you can connect to your vSphere environment and retrieve the Resource Pool, but I will leave that as an exercise for my colleague 🙂

Note: If you go through the query types, you may have noticed the resourcePool query type, the reason this will not work is that it only provides a list of root Resource Pools (basically vSphere Clusters) and it does not return the sub-resource pools that are created for Organization vDCs.

How to Access vCloud Director Remote Console using vCloud & VMRC API

by // 17 Comments

If you are familiar with the vCloud Director UI and have used the virtual console for a virtual machine, you may have noticed it is serviced by the VMRC (Virtual Machine Remote Console) vCD browser plugin which is only accessible in the vCD UI. If you are building a custom provisioning portal, you may want to provide similar functionality in your own portal for your users to access their VMs.

You can do so by leveraging the vCloud API and the new VMRC API that was made available with the release of vCloud Director 1.5.

In addition to the VMRC API documentation, there is an sample VMRC API Reference Implementation that you can download and experiment with. When you extract the contents of the zip file, you will find an index.html and console.html web page and you can use this locally on your system or host it on a web server.

The index.html is the landing page that you will use to provide the vCloud VM's screen ticket using the vCloud API's AcquireTicket REST API method. The console.html will then load the VMRC for the requested VM assuming you provided a valid screen ticket.

Here are the four simple steps to access a vCloud VM's remote console using the vCloud REST API & VMRC API using curl:

Step 1 - Login

You will need a system that has curl installed and you will need an account in vCD that has access to some powered on VMs. In the example below, I will be logging into a regular organization and not the System organization. There are a few parameters you need to specify to login to vCD and obtain an authorization token. You will need to specify the following parameters and the URL to your vCloud Director instance which should be in the form of https://vcd-fqdn/api/sessions:

  • -i = Include headers
  • -k = Performs an "insecure" SSL connection
  • -H = Setting the header for the version of vCloud Director (1.5 in this example)
  • -u = User credentials in the format of [username@org:password]
  • -X = Request type

Note: For more details on the cURL flags, please refer to the cURL documentation.

curl -i -k -H "Accept:application/*+xml;version=1.5" -u coke-admin@Coke:vmware -X POST https://vcd.primp-industries.com/api/sessions

HTTP/1.1 200 OK
Date: Tue, 28 Feb 2012 22:20:54 GMT
x-vcloud-authorization: n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=
Set-Cookie: vcloud-token=n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=; Secure; Path=/
Content-Type: application/vnd.vmware.vcloud.session+xml;version=1.5
Date: Tue, 28 Feb 2012 22:20:55 GMT
Content-Length: 894


<?xml version="1.0" encoding="UTF-8"?>
<Session xmlns="http://www.vmware.com/vcloud/v1.5" user="coke-admin" org="Coke" type="application/vnd.vmware.vcloud.session+xml" href="https://vcd.primp-industries.com/api/session/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
<Link rel="down" type="application/vnd.vmware.vcloud.orgList+xml" href="https://vcd.primp-industries.com/api/org/"/>
<Link rel="down" type="application/vnd.vmware.admin.vcloud+xml" href="https://vcd.primp-industries.com/api/admin/"/>
<Link rel="down" type="application/vnd.vmware.vcloud.query.queryList+xml" href="https://vcd.primp-industries.com/api/query"/>
<Link rel="entityResolver" type="application/vnd.vmware.vcloud.entity+xml" href="https://vcd.primp-industries.com/api/entity/"/>
</Session>

If you have successfully logged in, you should get an HTTP 200 response and get similar output as above. You will need to make note of your authorization token which is located on the third line that starts with "x-vcloud-authorization". This will be needed throughout the remainder of the session

Step 2 - Query All Powered On VMs

Using the new Query Service API in vCloud 1.5, we will locate all VMs within the vCD instance with the vm type. If you are logged in as a System Administrator and using the System Org, use the adminVM type instead. You can see the command is very similar to the one above but instead of specifying the credentials, we are using the authorization token from step 1. We are also including specific fields in the XML output for readability (VM name + vApp name), if you remove '&fields=name,containerName', you will see the other fields in the XML response. We also want to filter for only powered-on VMs, we can do this by adding '&filter=status==POWERED_ON'.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=" -X GET "https://vcd.primp-industries.com/api/query?type=vm&filter=status==POWERED_ON&fields=name,containerName"

HTTP/1.1 200 OK
Date: Tue, 28 Feb 2012 22:22:09 GMT
Content-Type: application/*+xml;version=1.5
Date: Tue, 28 Feb 2012 22:22:09 GMT
Content-Length: 1356

<?xml version="1.0" encoding="UTF-8"?>
<QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" total="2" pageSize="25" page="1" name="vm" type="application/vnd.vmware.vcloud.query.records+xml" href="https://vcd.primp-industries.com/api/query?type=vm&page=1&pageSize=25&format=records&filter=status==POWERED_ON&fields=name,containerName" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
<Link rel="alternate" type="application/vnd.vmware.vcloud.query.references+xml" href="https://vcd.primp-industries.com/api/query?type=vm&page=1&pageSize=25&format=references&filter=status==POWERED_ON&fields=name,containerName"/>
<Link rel="alternate" type="application/vnd.vmware.vcloud.query.idrecords+xml" href="https://vcd.primp-industries.com/api/query?type=vm&page=1&pageSize=25&format=idrecords&filter=status==POWERED_ON&fields=name,containerName"/>
<VMRecord name="WebServerVM" containerName="WebServer-Test-1" href="https://vcd.primp-industries.com/api/vApp/vm-25a4228e-1202-45a7-8f85-cafaeb1c3bc8"/>
<VMRecord name="ApplicationVM" containerName="AppTesting" href="https://vcd.primp-industries.com/api/vApp/vm-fadffc47-fa24-4ca0-9a90-f1048bc9f731"/>
</QueryResultRecords>

If the operation was successful, you should see a HTTP 200 response and list of VMs in your vCD instance. You will need to make a note of the VM's href property that you wish to obtain the remote console screen ticket for which is needed in the next step.

Step 3 - Obtain Screen Ticket for VM

Next we need to obtain the screen ticket using the AcquireTicket vCloud REST API method. You will need to perform a POST operation and using the href property from the previous step, append "/screen/action/acquireTicket". This link is only available IF the VM is powered-on and must be used within 30seconds or it will be considered invalid.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=" -X POST https://vcd.primp-industries.com/api/vApp/vm-25a4228e-1202-45a7-8f85-cafaeb1c3bc8/screen/action/acquireTicket

HTTP/1.1 200 OK Date: Tue, 28 Feb 2012 22:23:21 GMT Content-Type: application/vnd.vmware.vcloud.screenticket+xml;version=1.5 Date: Tue, 28 Feb 2012 22:23:21 GMT Content-Length: 895 <?xml version="1.0" encoding="UTF-8"?> <ScreenTicket xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">mks://172.30.0.140/vm-180?ticket=cst-HtefHNvLNiLmfK1na3mx74rqmLmM%2FjhWu3UHHTFrRlnGZg%2FtgjfqwanE4HUvWgvaclaR5vRI3iJrOvJS4P%2Fq6vH%2FfU06v0kMJ4U1ngSCNnt2SmHpaazWp%2B7JPLMCPQz0f88bxnU3t2NIX3bn%2BrsIznRNaDkQeIEldti%2F7pWsH%2FWm1PGIAk%2Fh54IpBuaGmlwPtNjs40zGTLjSYzRtO9oII75jgWA%2F%2FVcRBXtIPgoQzL4GHpyS6PZ7vaTklxdjCUJklJnlAbpVjo7P5uUqzfhhFQa52GNxZK%2FCLS6nNJpVkmvV12J5aUwsWe3OXdecsckW%2By3VLdcKe9Fp8N44TEgGfqrjzvCsXOv5DbiChjrfhIBKlUhN2%2BHL2fHqbtrBbKvtteNERMl08cqExR7qyf4n1w%3D%3D-X6PBbcMUeRCaF5G6lkqOrrdPbjm%2FfLOj1Exk8w%3D%3D--tp-9B%3A2B%3A82%3A8D%3AA2%3AFD%3AF1%3A8F%3A42%3A20%3ABF%3A75%3AA9%3A4F%3AFA%3AA6%3AC4%3A09%3ACA%3A4F--</ScreenTicket>

Now we just need to take the screen ticket which starts with mks://..... and copy that over to our VMRC sample web page and hit the submit button. If everything was successful, you should see the a remote console popup up for the VM you had just requested the screen ticket from. You will also notice the IP Address in the screen ticket is not the same address as your vCloud Director Cell, it is actually the console proxy address if you were wondering why the IP is different.

So this is great, but you still need some interaction with the browser to actually submit the screen ticket request. What if we just wanted to provide a single URL that would automatically launch the remote console for a requested VM? You can definitely do so, but you will need to perform a URL encoding on the mks://... ticket and construct the proper URL.

Here is a simple shell script called vcloud-vAppRemoteConsole.sh that uses the same vCloud REST API calls as above and allows a user to select the vCloud VM to to retrieve a screen ticket and returns back a single URL to access the remote console of the VM.

The script requires three parameters: (username@organization, password and vcloud-host)

Here is an sample execution:

Once you have successfully logged in, you should see a list of VMs that are currently powered on (highlighted in blue). You will need to select a VM and provide the "href" property as input (highlighted in green). Then the AcquireTicket method will be called and the screen ticket is then encoded using a quick Perl one-liner. The encoded entry is then appended to the hosted URL of the VMRC sample https://air.primp-industries.com/vmrc/console.html? (you can replace the URL to fit your environment). The output of the script (highlighted in orange) is the URL you would then paste into a supported web browser and the remote console will automatically launch without any additional user input.

With the above knowledge, you can easily translate this using any of the vCloud SDKs (Java, .NET, PHP), vCO or the new vCloud Director cmdlets and provide console access to your vCloud VMs.

Note: For vSphere virtual machine remote console access instructions, please take a look at this article here.