WilliamLam.com

Automating VMware Tools Upgrade Policy

by // 16 Comments

I received a question this week from a reader who was looking to change the VMware Tools upgrade policy for a few hundred virtual machines and wanted to know if it was possible to automate this. The answer is absolutely yes!

By default the VMware Tools upgrade policy is disabled and set to "manual" mode. If you want the vSphere platform to automatically check and upgrade VMware Tools upon a system power cycle, then you can enable it by going to Edit VM->Options->Tools->Check and upgrade Tools during power cycling.

To update this configuration, there is a property in the vSphere API called toolsUpgradePolicy which can accept two values: manual or upgradeAtPowerCycle.

Here is a vSphere SDK for Perl script updateVMToolsPolicy.pl that supports two types of operations: "list" and "update". The list operation will show you all VMs and their currently configured upgrade policy, by default they should be all manual unless you have changed it manually. The update operation will allow you to configure a list of VMs and policy you designate. This change can be done while the VM is running, you do not need to make any changes to the guestOS that is running.

Here is an example of the "list" operation:

If you want to take all the VMs that have "manual" policy and change them over to "upgradeAtPowerCycle", you can copy the output to a file and then use a find or UNIX/Linux grep command to search for entries that have the word "manual".

Here is the command you can use if you are on a UNIX/Linux system:

cat output | grep "manual" | awk -F '["|"]' '{print $2}'

Here is the command to get the first column which contains the VM display name:

cat output | grep "manual" | awk -F '["|"]' '{print $2}' > VMLIST

Lastly, you just need to take the previous command and redirect that to a file which will then be used in the "update" operation. You can also take the output and using an editor to get to the final output, use whatever you are comfortable with.

Here is an example of the commands listed above:

Now that we have the list of VMs we are interested in updating, we just need to select the policy and perform the "update" command. Here is an example:

So there you have it, you can now easily automate the the VMware Tools upgrade policy for any or all your VMs without having to edit each one manually.

Performing A Storage vMotion in vCloud Director Using vCloud REST API

by // 2 Comments

Currently there is not a way to perform vSphere Storage vMotion using the vCloud Director UI, but it is possible using the vCloud REST API. Since this feature is exposed in the vCloud API, it is also available using one of the three vCloud SDKs (Java, .NET, PHP), vCO or the new vCloud Director cmdlets but it does require some lines of code.

Since the vCloud API is exposed as a REST API, you can also interact with directly using a command line tool such as cURL or a REST client like the RESTClient Firefox plugin. In the example below, I will show you how to perform a Storage vMotion using cURL with the vCloud Director API 1.5.

Here we have a vApp in vCloud Director and it has two VMs, and as you can see both are located on the same datastore called "iSCSI-3".

Let's say we wanted to migrate the VM called "vESXi-01" to some other available datastore.

Step 1 - Login

You will need a system that has curl installed and you will need an account that has the "System Administrator" role to perform this operation. There are a few parameters you need to specify to login to vCD and obtain an authorization token. You will need to specify the following parameters and the URL to your vCloud Director instance which should be in the form of https://vcd-fqdn/api/sessions:

  • -i = Include headers
  • -k = Performs an "insecure" SSL connection
  • -H = Setting the header for the version of vCloud Director (1.5 in this example)
  • -u = User credentials in the format of [username@org:password]
  • -X = Request type

Note: For more details on the cURL flags, please refer to the cURL documentation.

curl -i -k -H "Accept:application/*+xml;version=1.5" -u administrator@system:vmware -X POST https://vcd.primp-industries.com/api/sessions

HTTP/1.1 200 OK Date: Sun, 19 Feb 2012 01:09:53 GMT Content-Type: application/*+xml;version=1.5 Date: Sun, 19 Feb 2012 01:09:53 GMT Content-Length: 1275 <?xml version="1.0" encoding="UTF-8"?> <QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" total="2" pageSize="25" page="1" name="adminVM" type="application/vnd.vmware.vcloud.query.records+xml" href="https://vcd.primp-industries.com/api/query?type=adminVM&page=1&pageSize=25&format=records&fields=name,datastoreName" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd"> <Link rel="alternate" type="application/vnd.vmware.vcloud.query.references+xml" href="https://vcd.primp-industries.com/api/query?type=adminVM&page=1&pageSize=25&format=references&fields=name,datastoreName"/> <Link rel="alternate" type="application/vnd.vmware.vcloud.query.idrecords+xml" href="https://vcd.primp-industries.com/api/query?type=adminVM&page=1&pageSize=25&format=idrecords&fields=name,datastoreName"/> <AdminVMRecord name="vESXi-01" datastoreName="iSCSI-3" href="https://vcd.primp-industries.com/api/vApp/vm-0332975f-394f-49d2-baf9-b222d126b942"/> <AdminVMRecord name="vCenter" datastoreName="iSCSI-3" href="https://vcd.primp-industries.com/api/vApp/vm-c0e4ead6-6e7d-4914-979b-0da49a8103a8"/> </QueryResultRecords>

If you have successfully logged in, you should get an HTTP 200 response and get similar output as above. You will need to make note of your authorization token which is located on the third line that starts with "x-vcloud-authorization". This will be needed throughout the remainder of the session

Step 2 - Locate VM

Using the new Query Service API in vCloud 1.5, we will locate all VMs within the vCD instance with the adminVM type since we are in the System organization. As you can see the command is very similar to the first one but instead of specifying the credentials, we using the authorization token from step 1. We are also including specific fields in the XML output for readability, if you remove '&fields=name,datastoreName', you will see the other fields in the XML response.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: aMcbkioTMvgVDDDJtdlwjrh6iutLPZ7fjL09hPa89mU=" -X GET 'https://vcd.primp-industries.com/api/query?type=adminVM&fields=name,datastoreName'

HTTP/1.1 200 OK
Date: Sun, 19 Feb 2012 01:10:15 GMT
Content-Type: application/*+xml;version=1.5
Date: Sun, 19 Feb 2012 01:10:15 GMT
Content-Length: 1232

<?xml version="1.0" encoding="UTF-8"?>
<QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" total="2" pageSize="25" page="1" name="datastore" type="application/vnd.vmware.vcloud.query.records+xml" href="https://vcd.primp-industries.com/api/query?type=datastore&page=1&pageSize=25&format=records&fields=name" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
    <Link rel="alternate" type="application/vnd.vmware.vcloud.query.references+xml" href="https://vcd.primp-industries.com/api/query?type=datastore&page=1&pageSize=25&format=references&fields=name"/>
    <Link rel="alternate" type="application/vnd.vmware.vcloud.query.idrecords+xml" href="https://vcd.primp-industries.com/api/query?type=datastore&page=1&pageSize=25&format=idrecords&fields=name"/>
    <DatastoreRecord name="iSCSI-4" href="https://vcd.primp-industries.com/api/admin/extension/datastore/a5d6bf25-7916-45cb-a65a-5dfa14d2ba38"/>
    <DatastoreRecord name="iSCSI-3" href="https://vcd.primp-industries.com/api/admin/extension/datastore/c9bff014-deaf-4ad1-8871-b2f2479f454f"/>

If the operation was succesful, you should see a HTTP 200 response and list of VMs in your vCD instance. You will need to make a note of the VM's href property that you wish to perform the Storage vMotion on.

Step 3 - Locate Datastore

Next we will use the Query Service API again to locate all available datastores within the vCD instance. We are also limiting it to the name field (href property is included by default). If you want to further filter the output to a particular vCenter Server, you can specify a filter for the particular vCenter href. For more details, you can take a look at the Query Service API documentation.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: aMcbkioTMvgVDDDJtdlwjrh6iutLPZ7fjL09hPa89mU=" -X GET 'https://vcd.primp-industries.com/api/query?type=datastore&fields=name'

HTTP/1.1 202 Accepted
Date: Sun, 19 Feb 2012 01:11:46 GMT
Location: https://vcd.primp-industries.com/api/task/2078bfe6-0a47-4615-a377-01ed8067c637
Content-Type: application/vnd.vmware.vcloud.task+xml;version=1.5
Date: Sun, 19 Feb 2012 01:11:46 GMT
Content-Length: 1283

<?xml version="1.0" encoding="UTF-8"?>
<Task xmlns="http://www.vmware.com/vcloud/v1.5" status="running" startTime="2012-02-18T17:11:46.883-08:00" operationName="vappRelocateVm" operation="Relocating Virtual Machine (0332975f-394f-49d2-baf9-b222d126b942)" expiryTime="2012-05-18T17:11:46.883-07:00" name="task" id="urn:vcloud:task:2078bfe6-0a47-4615-a377-01ed8067c637" type="application/vnd.vmware.vcloud.task+xml" href="https://vcd.primp-industries.com/api/task/2078bfe6-0a47-4615-a377-01ed8067c637" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
    <Link rel="task:cancel" href="https://vcd.primp-industries.com/api/task/2078bfe6-0a47-4615-a377-01ed8067c637/action/cancel"/>
    <Owner type="application/vnd.vmware.vcloud.vm+xml" name="" href="https://vcd.primp-industries.com/api/vApp/vm-0332975f-394f-49d2-baf9-b222d126b942"/>
    <User type="application/vnd.vmware.admin.user+xml" name="administrator" href="https://vcd.primp-industries.com/api/admin/user/b99d6670-5884-4345-a70d-f417d2e7556b"/>
    <Organization type="application/vnd.vmware.vcloud.org+xml" name="System" href="https://vcd.primp-industries.com/api/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9"/>
</Task>

Again, if the operation was successful, you should see an HTTP 200 response and a list of datastores in your vCD instance. From the output, we can see we have two available datastores "iSCSI-3" and "iSCSI-4". Let's go ahead and select "iSCSI-4" to migrate "vESXi-01" to. Also make a note of the datastore href property you wish to migrate the VM to.

Step 4 - Perform Storage vMotion

Lastly, we just now just need to invoke the "relocate" API operation for a VM. We need to first create the XML response for relocate operation as specified by the API. You will need to create a file that contains href of the datastore you wish to Storage vMotion the VM to:

<RelocateParams xmlns="http://www.vmware.com/vcloud/v1.5">
    <Datastore href="https://vcd.primp-industries.com/api/admin/extension/datastore/a5d6bf25-7916-45cb-a65a-5dfa14d2ba38"/>
</RelocateParams>

Now we are ready to craft our final command to perform the Storage vMotion operation. Taking the information we recorded earlier for the VM, we need to append to the URL "/action/relocate" which will perform the relocate operation on this particular VM. You will also notice in this curl request, it is a POST request and we including an extra header specifying the "Content-Type" for the operation as defined in the vCloud API for "relocate" operation. The last parameter is specifying some data which is our relocate-response file that is needed as part of the request body.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: aMcbkioTMvgVDDDJtdlwjrh6iutLPZ7fjL09hPa89mU=" -H "Content-Type:application/vnd.vmware.vcloud.relocateVmParams+xml" -X POST https://vcd.primp-industries.com/api/vApp/vm-0332975f-394f-49d2-baf9-b222d126b942/action/relocate -d @relocate-response

If everything went well, you should see an HTTP response code of 202 which means the operation was accepted by the server. Within the response of the "relocate" operation, you will get back a task handle for the Storage vMotion operation. If you watch your vCenter Server after you performed the command, you should see a Storage vMotion operation kick off for the VM that you have selected.

Once the operation has been completed, we can open up our vCloud Director UI and we should see that our VM has now been Storage vMotion to the datastore we specified.

Though you can interact with vCD directly with the REST API, you still may want to use one of the higher level abstraction languages to perform this type of operation in bulk.

Ruby vSphere Console (RVC) 1.6 Released

by // 10 Comments

Rich Lane the creator of Ruby vSphere Console just recently released RVC 1.6 which includes some new features and bug fixes. If you are a current RVC user, you can update to the latest version by using the "gem" utility. For those of you who are not familiar with RVC, it is console UI for vCenter Server and ESX(i) host and it provides a way to interact with your vSphere infrastructure like the vSphere Client but using a console shell. RVC can be installed on a Windows, Linux or Mac OSX system and it's built on RbVmomi which is an open source Ruby library/bindings to the vSphere API.

I recently spent sometime revising RVC as it has gotten a significant amount of new features from the last time I looked at it. RVC currently has over 120+ commands that simplify some of the most common tasks such as cloning a VM (Full & Linked Clone), vDS Management, Cluster/Host/VM Management and many many others.

There are so many cool and unique features that RVC offers that I could not name them all, but I thought I share with you my top 5 favorite features of RVC.

1. vSphere Virtual Filesystem
RVC presents the vSphere infrastructure as a virtual filesystem, this allows you to quickly navigate your Hosts, Cluster, Datacenter, VMs, etc. in a very user friendly and intuitive manner. It also allows for tab completion of commands and the use of wild cards when referring to objects.

Here is an example of connecting to a vCenter Server and quickly building out a new infrastructure by creating a new datacenter, cluster, adding an ESXi host and then creating a Linked Clone VM in just seconds:

Here is another example of what can be done with the virtualFS that Rich has created, you can easily view or edit files within the datastores (e.g. .vmx,.vmdk,vmware.log).

2. Multi-vCenter Support

You can connect to multiple vCenter Server(s) at once and quickly navigate to any of your vCenter Servers to perform various operations. Each vCenter will have it's own constructed virtual filesystem that will show up when you login.

Here is an example of connecting to 4 different vCenter Servers:

3. Performance Graphs
You can view performance charts using RVC similar to the vSphere Client. RVC quickly allows you to see what performance metrics are relevant for an object and displaying the graph using gnuplot:

Here is an example of displaying the live "cpu.usagemhz.average" metric for an ESXi host:

4. Marks
You can think of marks like an alias, RVC allows you to quickly save your favorite paths for later use and also include them within commands. This can be really helpful if you do not want to keep typing out a super long path and just give it a short and easy name to remember.

Here is an example of creating a mark called "favVM" to a particular VM in my vSphere infrastructure and perform a linked clone operation using the mark I created:

5. Extensibility

RVC as Rich stated does not (yet) have every single capability that the vSphere Client has, but is extremely extensible. Anyone with a bit of Ruby and vSphere API knowledge can extend or create new commands for RVC. It it modular in nature and when you create/extend a new command, you do not necessary need to add it to the original RVC source code. You can actually add your *.rb files into the .rvc directory in your home directory and RVC will automatically pickup the commands you added. If you create some cool and useful new commands, be sure to send them to Rich via the github project so he can include it in the main branch.

Here is an example of extending two new commands for "cluster" object. There are currently only three cluster commands by default: add, configure_ha and create. You may have noticed that configuring DRS is not one of the available commands, but wouldn't it be nice to have it? I created two new commands called "configure_drs" and "configure_automationlevel" which allows you to enable/disable DRS for a cluster and also specifying the DRS automation level.

Here is the source code for two commands that were added in .rvc/cluster.rb

# William Lam
# http://www.virtuallyghetto.com/

## Command to Enable/Disable DRS ## 

opts :configure_drs do
  summary "Configure DRS on a cluster"
  arg :cluster, nil, :lookup => VIM::ClusterComputeResource
  opt :disabled, "Disable DRS", :default => false
end

def configure_drs cluster, opts
  spec = VIM::ClusterConfigSpecEx(
    :drsConfig => {
       :enabled => !opts[:disabled],
    }
  )
  one_progress(cluster.ReconfigureComputeResource_Task :spec => spec, :modify => true)
end

## Command to Configure DRS Automation Level ###

opts :configure_automationlevel do
  summary "Configure DRS Automation level on a cluster"
  arg :cluster, nil, :lookup => VIM::ClusterComputeResource
  arg :level, "DRS Automation Level [fullyAutomated|manual|partiallyAutomated]"
end

def configure_automationlevel cluster, level
  spec = VIM::ClusterConfigSpecEx(
    :drsConfig => {
       :defaultVmBehavior => level,
    }
  )
  one_progress(cluster.ReconfigureComputeResource_Task :spec => spec, :modify => true)
end

 
Note: This was my first time writing any Ruby code, I'm sure I could have combined the two operations with some conditionals and then create the property drsConfig object.

If these five features of RVC seems really cool, I would highly recommend you go download and install RVC and give the rest a spin. If you have any feedback, questions or want to submit a patch for the RVC project, be sure to check out the RVC github project page for all the details.