WilliamLam.com

Performing A Storage vMotion in vCloud Director Using vCloud REST API

by // 2 Comments

Currently there is not a way to perform vSphere Storage vMotion using the vCloud Director UI, but it is possible using the vCloud REST API. Since this feature is exposed in the vCloud API, it is also available using one of the three vCloud SDKs (Java, .NET, PHP), vCO or the new vCloud Director cmdlets but it does require some lines of code.

Since the vCloud API is exposed as a REST API, you can also interact with directly using a command line tool such as cURL or a REST client like the RESTClient Firefox plugin. In the example below, I will show you how to perform a Storage vMotion using cURL with the vCloud Director API 1.5.

Here we have a vApp in vCloud Director and it has two VMs, and as you can see both are located on the same datastore called "iSCSI-3".

Let's say we wanted to migrate the VM called "vESXi-01" to some other available datastore.

Step 1 - Login

You will need a system that has curl installed and you will need an account that has the "System Administrator" role to perform this operation. There are a few parameters you need to specify to login to vCD and obtain an authorization token. You will need to specify the following parameters and the URL to your vCloud Director instance which should be in the form of https://vcd-fqdn/api/sessions:

  • -i = Include headers
  • -k = Performs an "insecure" SSL connection
  • -H = Setting the header for the version of vCloud Director (1.5 in this example)
  • -u = User credentials in the format of [username@org:password]
  • -X = Request type

Note: For more details on the cURL flags, please refer to the cURL documentation.

curl -i -k -H "Accept:application/*+xml;version=1.5" -u administrator@system:vmware -X POST https://vcd.primp-industries.com/api/sessions

HTTP/1.1 200 OK Date: Sun, 19 Feb 2012 01:09:53 GMT Content-Type: application/*+xml;version=1.5 Date: Sun, 19 Feb 2012 01:09:53 GMT Content-Length: 1275 <?xml version="1.0" encoding="UTF-8"?> <QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" total="2" pageSize="25" page="1" name="adminVM" type="application/vnd.vmware.vcloud.query.records+xml" href="https://vcd.primp-industries.com/api/query?type=adminVM&page=1&pageSize=25&format=records&fields=name,datastoreName" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd"> <Link rel="alternate" type="application/vnd.vmware.vcloud.query.references+xml" href="https://vcd.primp-industries.com/api/query?type=adminVM&page=1&pageSize=25&format=references&fields=name,datastoreName"/> <Link rel="alternate" type="application/vnd.vmware.vcloud.query.idrecords+xml" href="https://vcd.primp-industries.com/api/query?type=adminVM&page=1&pageSize=25&format=idrecords&fields=name,datastoreName"/> <AdminVMRecord name="vESXi-01" datastoreName="iSCSI-3" href="https://vcd.primp-industries.com/api/vApp/vm-0332975f-394f-49d2-baf9-b222d126b942"/> <AdminVMRecord name="vCenter" datastoreName="iSCSI-3" href="https://vcd.primp-industries.com/api/vApp/vm-c0e4ead6-6e7d-4914-979b-0da49a8103a8"/> </QueryResultRecords>

If you have successfully logged in, you should get an HTTP 200 response and get similar output as above. You will need to make note of your authorization token which is located on the third line that starts with "x-vcloud-authorization". This will be needed throughout the remainder of the session

Step 2 - Locate VM

Using the new Query Service API in vCloud 1.5, we will locate all VMs within the vCD instance with the adminVM type since we are in the System organization. As you can see the command is very similar to the first one but instead of specifying the credentials, we using the authorization token from step 1. We are also including specific fields in the XML output for readability, if you remove '&fields=name,datastoreName', you will see the other fields in the XML response.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: aMcbkioTMvgVDDDJtdlwjrh6iutLPZ7fjL09hPa89mU=" -X GET 'https://vcd.primp-industries.com/api/query?type=adminVM&fields=name,datastoreName'

HTTP/1.1 200 OK
Date: Sun, 19 Feb 2012 01:10:15 GMT
Content-Type: application/*+xml;version=1.5
Date: Sun, 19 Feb 2012 01:10:15 GMT
Content-Length: 1232

<?xml version="1.0" encoding="UTF-8"?>
<QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" total="2" pageSize="25" page="1" name="datastore" type="application/vnd.vmware.vcloud.query.records+xml" href="https://vcd.primp-industries.com/api/query?type=datastore&page=1&pageSize=25&format=records&fields=name" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
    <Link rel="alternate" type="application/vnd.vmware.vcloud.query.references+xml" href="https://vcd.primp-industries.com/api/query?type=datastore&page=1&pageSize=25&format=references&fields=name"/>
    <Link rel="alternate" type="application/vnd.vmware.vcloud.query.idrecords+xml" href="https://vcd.primp-industries.com/api/query?type=datastore&page=1&pageSize=25&format=idrecords&fields=name"/>
    <DatastoreRecord name="iSCSI-4" href="https://vcd.primp-industries.com/api/admin/extension/datastore/a5d6bf25-7916-45cb-a65a-5dfa14d2ba38"/>
    <DatastoreRecord name="iSCSI-3" href="https://vcd.primp-industries.com/api/admin/extension/datastore/c9bff014-deaf-4ad1-8871-b2f2479f454f"/>

If the operation was succesful, you should see a HTTP 200 response and list of VMs in your vCD instance. You will need to make a note of the VM's href property that you wish to perform the Storage vMotion on.

Step 3 - Locate Datastore

Next we will use the Query Service API again to locate all available datastores within the vCD instance. We are also limiting it to the name field (href property is included by default). If you want to further filter the output to a particular vCenter Server, you can specify a filter for the particular vCenter href. For more details, you can take a look at the Query Service API documentation.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: aMcbkioTMvgVDDDJtdlwjrh6iutLPZ7fjL09hPa89mU=" -X GET 'https://vcd.primp-industries.com/api/query?type=datastore&fields=name'

HTTP/1.1 202 Accepted
Date: Sun, 19 Feb 2012 01:11:46 GMT
Location: https://vcd.primp-industries.com/api/task/2078bfe6-0a47-4615-a377-01ed8067c637
Content-Type: application/vnd.vmware.vcloud.task+xml;version=1.5
Date: Sun, 19 Feb 2012 01:11:46 GMT
Content-Length: 1283

<?xml version="1.0" encoding="UTF-8"?>
<Task xmlns="http://www.vmware.com/vcloud/v1.5" status="running" startTime="2012-02-18T17:11:46.883-08:00" operationName="vappRelocateVm" operation="Relocating Virtual Machine (0332975f-394f-49d2-baf9-b222d126b942)" expiryTime="2012-05-18T17:11:46.883-07:00" name="task" id="urn:vcloud:task:2078bfe6-0a47-4615-a377-01ed8067c637" type="application/vnd.vmware.vcloud.task+xml" href="https://vcd.primp-industries.com/api/task/2078bfe6-0a47-4615-a377-01ed8067c637" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
    <Link rel="task:cancel" href="https://vcd.primp-industries.com/api/task/2078bfe6-0a47-4615-a377-01ed8067c637/action/cancel"/>
    <Owner type="application/vnd.vmware.vcloud.vm+xml" name="" href="https://vcd.primp-industries.com/api/vApp/vm-0332975f-394f-49d2-baf9-b222d126b942"/>
    <User type="application/vnd.vmware.admin.user+xml" name="administrator" href="https://vcd.primp-industries.com/api/admin/user/b99d6670-5884-4345-a70d-f417d2e7556b"/>
    <Organization type="application/vnd.vmware.vcloud.org+xml" name="System" href="https://vcd.primp-industries.com/api/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9"/>
</Task>

Again, if the operation was successful, you should see an HTTP 200 response and a list of datastores in your vCD instance. From the output, we can see we have two available datastores "iSCSI-3" and "iSCSI-4". Let's go ahead and select "iSCSI-4" to migrate "vESXi-01" to. Also make a note of the datastore href property you wish to migrate the VM to.

Step 4 - Perform Storage vMotion

Lastly, we just now just need to invoke the "relocate" API operation for a VM. We need to first create the XML response for relocate operation as specified by the API. You will need to create a file that contains href of the datastore you wish to Storage vMotion the VM to:

<RelocateParams xmlns="http://www.vmware.com/vcloud/v1.5">
    <Datastore href="https://vcd.primp-industries.com/api/admin/extension/datastore/a5d6bf25-7916-45cb-a65a-5dfa14d2ba38"/>
</RelocateParams>

Now we are ready to craft our final command to perform the Storage vMotion operation. Taking the information we recorded earlier for the VM, we need to append to the URL "/action/relocate" which will perform the relocate operation on this particular VM. You will also notice in this curl request, it is a POST request and we including an extra header specifying the "Content-Type" for the operation as defined in the vCloud API for "relocate" operation. The last parameter is specifying some data which is our relocate-response file that is needed as part of the request body.

curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: aMcbkioTMvgVDDDJtdlwjrh6iutLPZ7fjL09hPa89mU=" -H "Content-Type:application/vnd.vmware.vcloud.relocateVmParams+xml" -X POST https://vcd.primp-industries.com/api/vApp/vm-0332975f-394f-49d2-baf9-b222d126b942/action/relocate -d @relocate-response

If everything went well, you should see an HTTP response code of 202 which means the operation was accepted by the server. Within the response of the "relocate" operation, you will get back a task handle for the Storage vMotion operation. If you watch your vCenter Server after you performed the command, you should see a Storage vMotion operation kick off for the VM that you have selected.

Once the operation has been completed, we can open up our vCloud Director UI and we should see that our VM has now been Storage vMotion to the datastore we specified.

Though you can interact with vCD directly with the REST API, you still may want to use one of the higher level abstraction languages to perform this type of operation in bulk.

Ruby vSphere Console (RVC) 1.6 Released

by // 10 Comments

Rich Lane the creator of Ruby vSphere Console just recently released RVC 1.6 which includes some new features and bug fixes. If you are a current RVC user, you can update to the latest version by using the "gem" utility. For those of you who are not familiar with RVC, it is console UI for vCenter Server and ESX(i) host and it provides a way to interact with your vSphere infrastructure like the vSphere Client but using a console shell. RVC can be installed on a Windows, Linux or Mac OSX system and it's built on RbVmomi which is an open source Ruby library/bindings to the vSphere API.

I recently spent sometime revising RVC as it has gotten a significant amount of new features from the last time I looked at it. RVC currently has over 120+ commands that simplify some of the most common tasks such as cloning a VM (Full & Linked Clone), vDS Management, Cluster/Host/VM Management and many many others.

There are so many cool and unique features that RVC offers that I could not name them all, but I thought I share with you my top 5 favorite features of RVC.

1. vSphere Virtual Filesystem
RVC presents the vSphere infrastructure as a virtual filesystem, this allows you to quickly navigate your Hosts, Cluster, Datacenter, VMs, etc. in a very user friendly and intuitive manner. It also allows for tab completion of commands and the use of wild cards when referring to objects.

Here is an example of connecting to a vCenter Server and quickly building out a new infrastructure by creating a new datacenter, cluster, adding an ESXi host and then creating a Linked Clone VM in just seconds:

Here is another example of what can be done with the virtualFS that Rich has created, you can easily view or edit files within the datastores (e.g. .vmx,.vmdk,vmware.log).

2. Multi-vCenter Support

You can connect to multiple vCenter Server(s) at once and quickly navigate to any of your vCenter Servers to perform various operations. Each vCenter will have it's own constructed virtual filesystem that will show up when you login.

Here is an example of connecting to 4 different vCenter Servers:

3. Performance Graphs
You can view performance charts using RVC similar to the vSphere Client. RVC quickly allows you to see what performance metrics are relevant for an object and displaying the graph using gnuplot:

Here is an example of displaying the live "cpu.usagemhz.average" metric for an ESXi host:

4. Marks
You can think of marks like an alias, RVC allows you to quickly save your favorite paths for later use and also include them within commands. This can be really helpful if you do not want to keep typing out a super long path and just give it a short and easy name to remember.

Here is an example of creating a mark called "favVM" to a particular VM in my vSphere infrastructure and perform a linked clone operation using the mark I created:

5. Extensibility

RVC as Rich stated does not (yet) have every single capability that the vSphere Client has, but is extremely extensible. Anyone with a bit of Ruby and vSphere API knowledge can extend or create new commands for RVC. It it modular in nature and when you create/extend a new command, you do not necessary need to add it to the original RVC source code. You can actually add your *.rb files into the .rvc directory in your home directory and RVC will automatically pickup the commands you added. If you create some cool and useful new commands, be sure to send them to Rich via the github project so he can include it in the main branch.

Here is an example of extending two new commands for "cluster" object. There are currently only three cluster commands by default: add, configure_ha and create. You may have noticed that configuring DRS is not one of the available commands, but wouldn't it be nice to have it? I created two new commands called "configure_drs" and "configure_automationlevel" which allows you to enable/disable DRS for a cluster and also specifying the DRS automation level.

Here is the source code for two commands that were added in .rvc/cluster.rb

# William Lam
# http://www.virtuallyghetto.com/

## Command to Enable/Disable DRS ## 

opts :configure_drs do
  summary "Configure DRS on a cluster"
  arg :cluster, nil, :lookup => VIM::ClusterComputeResource
  opt :disabled, "Disable DRS", :default => false
end

def configure_drs cluster, opts
  spec = VIM::ClusterConfigSpecEx(
    :drsConfig => {
       :enabled => !opts[:disabled],
    }
  )
  one_progress(cluster.ReconfigureComputeResource_Task :spec => spec, :modify => true)
end

## Command to Configure DRS Automation Level ###

opts :configure_automationlevel do
  summary "Configure DRS Automation level on a cluster"
  arg :cluster, nil, :lookup => VIM::ClusterComputeResource
  arg :level, "DRS Automation Level [fullyAutomated|manual|partiallyAutomated]"
end

def configure_automationlevel cluster, level
  spec = VIM::ClusterConfigSpecEx(
    :drsConfig => {
       :defaultVmBehavior => level,
    }
  )
  one_progress(cluster.ReconfigureComputeResource_Task :spec => spec, :modify => true)
end

 
Note: This was my first time writing any Ruby code, I'm sure I could have combined the two operations with some conditionals and then create the property drsConfig object.

If these five features of RVC seems really cool, I would highly recommend you go download and install RVC and give the rest a spin. If you have any feedback, questions or want to submit a patch for the RVC project, be sure to check out the RVC github project page for all the details.

Automating vCenter Server Appliance 5.0 (VCSA) Configurations

by // 2 Comments

I recently had a need to deploy half a dozen or so vCenter Server Appliance (VCSA) in my home lab and even though it is one of the easiest appliances to setup, going through the web management interface can still be time consuming. I was looking to see if there was a way to automate some of the configurations and while digging in some of the vCenter logs I came across /var/log/vmware/vpx/vpxd_cfg.log which provided the answer I was looking for.

If you watch the log as you configure the VCSA through the web management interface, you will see a series of commands calling /usr/sbin/vpxd_servicecfg. This utility is actually just a shell script wrapper for the configuration of vpxd (vCenter Server Daemon) and it's respective services.

As you can see there is a warning message about the use of the tool, so I will also re-iterate with my own disclaimer:

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Please test this in a development environment before using it on actual systems.

For my lab, this was exactly what I was looking for to quickly spin up VCSA and configure all the necessary services to start the vCenter Service. In my lab, I was able to get the VCSA booted up via DHCP and execute a simple shell script remotely via SSH to VCSA and then connect to vCenter Server after the process was completed.

Here is script with the minimal commands needed for running an embedded configuration:

#!/bin/bash

echo "Accepting EULA ..."
/usr/sbin/vpxd_servicecfg eula accept

echo "Configuring Embedded DB ..."
/usr/sbin/vpxd_servicecfg db write embedded

echo "Configuring SSO..."
/usr/sbin/vpxd_servicecfg sso write embedded

echo "Starting VCSA ..."
/usr/sbin/vpxd_servicecfg service start

To run the script remotely (you do not need to copy it to VCSA), use the following command:

# ssh root@[vcsa-ip] < configureVCSA.sh

The script accepts the EULA, configures the database as embedded configuration and then starts up the vCenter service. A return code is provided upon each command execution which has VC_CFG_RESULT=N where N can be 0 which is a successful execution and any other value means there is an error. To figure out what the code means, there is a section in the shell script that lists all the codes and their meaning. In general, you should be getting 0 for all commands

Here are some other useful commands that I tested with:

Configure Active Directory:

/usr/sbin/vpxd_servicecfg ad write [AD_USERNAME] [AD_PASSWORD] [AD_DOMAIN]

Re-initialize vCenter DB *** PLEASE BE VERY CAREFUL WHEN USING THIS COMMAND, AS DB IS WIPED***

ensure vpxd has stopped

vpxd_servicecfg stop

re-init DB

vpxd_servicecfg db initialize

You can also configure an external database for vCenter, NIS directory service and even replace SSL certificates but I will leave that to the reader for some exploration.