William Lam

What is the VMware VIX API and it's future?

11.21.2010 by William Lam // 4 Comments

Some recent twitter conversations about vSphere SDK improvements, specifically around VMware VIX API and few emails around the use cases was the motivation for this post.

I will provide a very high level of what the VMware VIX API is, since there is a awesome presentation by Matt LaMantia at TechExchange in VMworld 2010 SF that goes into more details about VIX (video below). The VIX API (Virtual Infrastructure eXtension) is an API that provides guest management operations inside of a virtual machine that maybe running on VMware vSphere, Fusion, Workstation or Player. These operations are executed on behalf of the vmware-tools service that must be running within the virtual machine and guest credentials are required prior to execution.

The current incarnation of the VIX API is a little odd because it provides guest management operations which is unique to the VIX API but it also provides virtual machine operations. These virtual machine operation overlaps with some of the functionality provided by the vSphere API which is used for managing your vSphere infrastructure. Part of this overlap was due to origins of VIX which started from Workstation on the desktop. I almost consider this VMware API sprawl 🙂

Currently if you want to automate your vSphere infrastructure and want hooks into your virtual machine and guestOSes you need to leverage both the vSphere API and VIX API. I have seen this cause some confusion in the communities. Users wanting to standardize on VIX API and realize that 80% of what they want to do is available in VIX but the portion requires the use of the vSphere APIs. Don't get me wrong, the functionality of VIX is very powerful. When I first learn in version 1.6.2 of VIX where it officially supported vSphere, a whole new set of possibilities just opened up for administrators and developers.

There are several ways of using the VIX API today. If you are using PowerCLI, there are actually a few cmdlets that directly integrate with VIX, which requires you to install the VIX client libraries local to your PowerCLI installation. One such cmdlet is the Invoke-VMScript and here are a few example use cases:

If you don't prefer the dark side 😉 and want to run something like the vSphere SDK for Perl, C or COM, the VIX client libraries also includes client side bindings to these languages. If you are a Java shop and want to leverage Steve Jin's VI Java there is also an open source project called VIX Java Toolkit that can be used. Here is an example use case with vSphere SDK for Perl:

http://www.virtuallyghetto.com/2010/07/automate-update-manager-operations.html

You can also just install the VIX client libraries which also includes a pre-compiled binary called vmrun that provides majority of the VIX operations all bundled into one utility. It also supports VMware vSphere, Workstation, Fusion and Player. Here is an example use case:

http://professionalvmware.com/2008/12/vmware-vix-changing-ips-of-a-guest-vm/

There is also a very interesting application called VGC (Virtual Guest Console) created by the VMware Lab guys also known as flings. This is a graphical interface similar to VMware Remote Console, but it is 1000x better and provides integration to all the VIX operations. The application allows you to view running processes in a guest, kill a particular process, deploy application to a guest, download/upload files and browse guest filesystem and much much more. I would highly recommend taking a look at this tool!

As you can see, the possibilities are endless! but you still have to use two separate APIs. I along with others in the community have asked for a consolidation of the VIX API and merging with the vSphere API, makes the most sense. I think our feedback has finally been heard and if you watch the presentation given by Matt, you will see that future of VIX API is exactly that, consolidatation with the vSphere API. Though we will not see it anytime soon until the next major release of vSphere dubbed vSphere.next (MN) and vSphere.next^2 (OP), it is a change I am looking forward to.

One other interesting thing to point out, during the first release of the VIX API that supported vSphere which was 1.6.2, there was a tiny bug that was identified regarding licensing. As you may or may not know, VMware requires that you have a "paid" license to be able to leverage the various APIs and CLIs for automation, configuration and management of your vSphere infrastructure. This is generally not an issue when dealing with vCenter or ESX, but with ESXi, you have option of a free license.

In the VIX 1.6.2 release, you actually have full VIX read and write operations against an ESXi host using the free license. This was of course fixed in subsequent releases and the trick only works with ESXi 4.0 or older. You will notice on the VIX landing page, there is no mention of the release notes for 1.6.2 release other than 1.7.1 superseding it. You however still can download 1.6.2 release under VIX 1.6 and you can still see the release notes if you search on VMware's site or on Google.

For more details about VIX and downloads, take a look at the following:

PPC15 Guest Operations Using VMware VIX APIs and Beyond:

Guest Operations using VMware VIX APIs and Beyond from heyitspablo on Vimeo.

How to obtain GID and LWID from esxtop?

11.20.2010 by William Lam // 11 Comments

Last week I remember seeing a tweet from Duncan Epping regarding the use of VMware vsish to obtain the VMX Cartel ID, also known as the LWID (Leader World Id) for a virtual machine world. This VMX Cartel ID is then used to obtain the GID (Group Id) which was documented by Duncan on his esxtop page to limit the view in esxtop to a particular VM of interest:

VMWID=`vm-support -x | grep |awk '{gsub("wid=", "");print $1}'`
VMXCARTEL=`vsish -e cat /vm/$VMWID/vmxCartelID`
vsish -e cat /sched/memClients/$VMXCARTEL/SchedGroupID

As you can see from the above, Duncan utilizes the vm-support to obtain the WID (World ID) and then using vsish to query for the VMX Cartel ID (LWID). Finally, using LWID, he was able to obtain the GID (Group ID). This example only applies to ESXi, since classic ESX does not include vsish tool.

Here is an example of where to locate the values in esxtop under CPU section:

I remember during some of my skunk work adventures, there were other methods of obtaining these IDs. Due to my VCAP-DCD cramming last week, I did not get a change to further investigate. Now that I have some time, I thought share some of these other methods for obtaining the GID and LWID in both ESX and ESXi.

ESXi

Obtaining VMX Cartel ID (LWID)

vmdumper - This will list all running VMs including the path to the .vmx configuration file, the display name of your VM and the VMX Cartel ID

~ # /sbin/vmdumper -l
wid=16881 pid=-1 cfgFile="/vmfs/volumes/4cdeeb09-1ad4c18a-5ff9-003048d9586a/scofield/scofield.vmx" uuid="42 34 e6 bc f1 83 c2 db-fb fb 08 73 b7 0c 26 40" displayName="scofield" vmxCartelID=16880

ps - The VMX Cartel ID is actually both the PID and CID within process status in ESXi

~ # ps -Cc | grep "scofield" | grep -v grep | awk '{print $2}'
16880

vscsiStats - The VMX Cartel ID is also used in identifying the VMs when displaying vscsi information

~ # /sbin/vscsiStats -l | grep "scofield" | awk '{print $4}' | sed 's/,//g'
16880

esxcli - You can obtain pretty much the same information as in vmdumper using the new "vm" option in esxcli

~ # /sbin/esxcli vms vm list | grep -A3 "scofield" | grep Cartel | awk '{gsub("VMX Cartel ID:", "");print $1}'
16880

Obtaining GID

Method1 - Using vscsiStats and esxcfg-info to extract VM's GID

~ # VM_NAME=scofield;VMX_CARTEL=$(/sbin/vscsiStats -l | grep "${VM_NAME}" | awk '{print $4}' | sed 's/,//g');esxcfg-info -r -F perl > /tmp/esxcfg-info.txt;grep -B1 "vm.${VMX_CARTEL}" /tmp/esxcfg-info.txt | head -1 | awk '{print $3}' | sed 's/,//g'
4639

Method2 - Using esxcli and esxcfg-info to extract VM's GID

~ # VM_NAME=scofield;VMX_CARTEL=$(/sbin/esxcli vms vm list | grep -A3 ${VM_NAME} | grep Cartel | awk '{gsub("VMX Cartel ID:", "");print $1}');esxcfg-info -r -F perl > /tmp/esxcfg-info.txt;grep -B1 "vm.${VMX_CARTEL}" /tmp/esxcfg-info.txt | head -1 | awk '{print $3}' | sed 's/,//g'
4639

Method3 - Using esxcli and vsish to extract VM's GID

~ # VM_NAME=scofield;VMX_CARTEL=$(/sbin/esxcli vms vm list | grep -A3 ${VM_NAME} | grep Cartel | awk '{gsub("VMX Cartel ID:", "");print $1}');/sbin/vsish -e cat /sched/memClients/${VMX_CARTEL}/SchedGroupID
4639

Note: You just need to substitute VM_NAME variable with the display name of of the virtual machine you are interested in. There are actually multiple commands being executed in this one line. If your VM has spaces, make sure you put quotes around it

ESX

Obtaining VMX Cartel ID (LWID)

vmdumper - This will list all running VMs including the path to the .vmx configuration file, the display name of your VM and the VMX Cartel ID

[root@himalaya ~]# /usr/lib/vmware/bin/vmdumper -l
wid=29250 pid=-1 cfgFile="/vmfs/volumes/4a48004d-f9af7fa0-5bbf-003048d9586b/STA202G/STA202G.vmx" uuid="42 30 d1 75 c5 3e 81 2a-14 15 1f 86 bb 5b b9 e5" displayName="STA202G" vmxCartelID=29249

vscsiStats - The VMX Cartel ID is also used in identifying the VMs when displaying vscsi information

[root@himalaya ~]# /usr/lib/vmware/bin/vscsiStats -l | grep "STA202G" | awk '{print $4}' | sed 's/,//g'
29249

esxcli - You can obtain pretty much the same information as in vmdumper using the new "vm" option in esxcli

[root@himalaya ~]# /usr/sbin/esxcli vms vm list | grep -A3 STA202G | grep Cartel | awk '{gsub("VMX Cartel ID:", "");print $1}'
29249

Obtaining GID

Method1 - Using esxcli and esxcfg-info to extract VM's GID

[root@himalaya ~]# VM_NAME=STA202G ;VMX_CARTEL=$(/usr/sbin/esxcli vms vm list | grep -A3 ${VM_NAME} | grep Cartel | awk '{gsub("VMX Cartel ID:", "");print $1}');esxcfg-info -r -F perl > /tmp/esxcfg-info.txt;grep -B1 "vm.${VMX_CARTEL}" /tmp/esxcfg-info.txt | head -1 | awk '{print $3}' | sed 's/,//g'
197

Method2 - Using sched-stats to extract GID

[root@himalaya ~]# VM_NAME=STA202G;VMX_CARTEL=$(/usr/lib/vmware/bin/vscsiStats -l | grep "${VM_NAME}" | awk '{print $4}' | sed 's/,//g');/usr/bin/sched-stats -t groups | grep "vm.${VMX_CARTEL}" | awk '{print $1}'
197

Method3 - Using /proc information from sched

[root@himalaya ~]# VM_NAME=STA202G ;grep "${VM_NAME}" /proc/vmware/sched/drm-stats | awk '{print $1}'
197

As you can see, there are more than one way to obtain the same exact values and I am sure there are probably a few others. For command simplicity, I would probably recommend method #3 for ESXi and method #3 for ESX. As with anything, be careful when you using these methods as they are not really supported by VMware unless directed by their support organization.