WilliamLam.com

vMotion' With Style

by // Leave a Comment

I got this idea after catching an interesting tweet by Cody Bunch last Friday:

I had just recently finished a post on How to Ack & Reset vCenter Alarm implementing hidden API method and thought this might work by using a vCenter alarm. After few hours in our skunkwork's lab, I found "a" method to exactly what Cody wished for. If you can not wait, jump straight to the video at the bottom 🙂

The following tools were used:

  • PsExec -Windows utility to perform remote operations
  • SayStatic - Windows text to speech utility
  • A random mp3 audio file found online

The environment consisted of two vESXi 4.1 hosts and Windows XP VM residing on shared storage that was vMotion functional:

You will need to upload some files to your vCenter server and to the desktop in which you want to implement this hack.

Local Desktop Server Setup

1) Download SayStatic to your local desktop ( in the example, it is located on the desktop)

2) Download a playable mp3 file (in the example, it is located on the desktop)

Note: If you decide to change the path to the files above, please make note of the path as it will be used later in the alarm script.

Here is a screenshot of my local desktop:

vCenter Server Setup

1) Download PsTools toolkit and extract the contents to your vCenter Server and make a note of the path (in the example, it is located on the desktop)

2) Create an alarm script and make a note of the path ( in the example, it is located in C:\alarm.cmd)

The alarm.cmd should contain the following:

You will need to edit the script and update at minimum the following variables:
REMOTE_SERVER = This is the hostname or IP address of your local desktop, make sure you keep the double slashes which is needed

REMOTE_USERNAME = This is the username to login to your local desktop and it should be the one you use to login else you will not see anything interesting. Make sure you include both the Domain and username, if your system is not part of a domain, use the local system name else the script will fail.

REMOTE_PASSWORD = The password to the account aove

Note: If you have changed the path of the files on either the local desktop or vCenter Server, you will need edit the remainder variables that specify where to look for the executable's.

Here is a screenshot of vCenter Server desktop:

vCenter Alarm:

Finally, we need to create an alarm in vCenter, you can create this at any level of the vCenter infrastructure.

1) Create an alarm and give it a name and ensure the type is of "Virtual Machines" and monitor type is "Monitor for specific events occurring on this object" (in the example, I call it VMOTIONIN_WITH_STYLE)

2) Now click on the Triggers tab and look for an event type called "VM emigrating", no this is not a typo. VMware apparently has both "VM migrating" and "VM emigrating", apparently the functional alarm is under the name emigrating ... don't ask me why they called it that ;). Make sure the status is set to "alert"

3) Next, click on the Actions tab and set the action to "Run a command" and specify the configuration to the path of the alarm.cmd. In the example, I stored the script in C:\alarm.cmd and then make sure it alarms when it hits "red" or error and then click okay for the alarm to be created.

Now you are ready to go. Instead of explaining and providing screenshots, I thought I show you what this would look like (explanation of what is going on is at the very bottom).

Without further ado, here is a recorded video of this in action:

vMotion' With Style from lamw on Vimeo.

What is actually going on:
When a vMotion is triggered, it will fire off the alarm.cmd script which basically uses SayStatic.exe to remotely execute on your local desktop to announce the virtual machine being vMotioned by capturing the VMware specific environmental variables and then it will also remotely playing the local audio file using Windows Media Player.One caveat that I was not able to solve, was clearing the alarm after the vMotion. You will notice the virtual machines that vMotion and fire off this alarm will stay alerted and will not play again until it has been resetted to green. I thought about creating another alarm to clear this initial alarm but it did not actually clear the alarm.

There you have it, vMotioin' with style ... though cool in concept, I doubt you will last very long with this kicking off on every single vMotion in your environment.

Big thanks to Cody Bunch for the idea! 🙂

resxtop bug in vCLI 4.1 not vMA 4.1

by // Leave a Comment

I recently noticed a thread in the vMA forums regarding an issue using resxtop on vMA 4.1 to view VM disk statistics on an ESXi 4.0 hosts. The thread was started on July 26th 2010 and as far as I could tell, no resolution was ever provided. A recent comment that was left on Oct 14th 2010 by another user experiencing the same behavior got my attention while browsing the VMTN forums. I decided to perform a small test to see if this was in fact an issue and it turns out it maybe a bug in resxtop running on vMA 4.1.

The test environment consists of the following:

  • 1 x vESXi 4.0u2 running 1 VM
  • 1 x vESXi 4.1 running 1 VM
  • 1 x vMA 4.0
  • 1 x vMA 4.1

Here is a screenshot of running esxtop locally within Tech Support Mode (Busybox console) on the ESXi 4.0 hosts and you can see, the VM disk statistics are visible and present:

Here is a screenshot of running esxtop locally within Tech Support Mode (Busybox console) on the ESXi 4.1 hosts and you can see, the VM disk statistics are visible and present:

Now here is a screenshot of running both vMA 4.1 (on top) and vMA 4.0 (on bottom) connecting to an ESXi 4.0 host running a single virtual machine called VM2. I use resxtop to connect to the ESXi 4.0 and select "v" option or VM disk statistics and as you can see from the screenshot, no statistics are being displayed when using vMA 4.1:

I perform the same exact test but now connecting to an ESXi 4.1 host using both vMA 4.1 (on top) and vMA 4.0 (on bottom) and what is actually surprising is, the VM disk statistics shows up for both vMA 4.0 and vMA 4.1:

It looks like something changed in the resxtop binary between vMA 4.0 and vMA 4.1 that causes the the VM disk statistics on ESX(i) hosts running on 4.0 not to be visible. I have not found any VMware KB articles documenting this issue nor found anything in vMA's release notes in which this configuration is not supported. This looks like a bug to me and I will try follow-up with the vMA's product manager to get an official word.

Note: I used ESXi since it was quicker to deploy for this test, but the issue affects both ESX and ESXi 4.0 when using resxtop from vMA 4.1

UPDATE:  After further investigation, I found out the issue is in fact with vCLI 4.1 installation and not with vMA 4.1. To confirm, I spun up a CentOS VM and installed and individually tested vCLI 4.0u2 and vCLI 4.1 and experience the same behavior as in vMA. I have already reported the issue to vMA product manager and hopefully we can get this resolved in either a patch or an updated released.

How to Ack & Reset vCenter Alarm implementing hidden API method

by // 11 Comments

There was a recent question in the VMTN developers forum around automating the acknowledgment and resetting of a triggered vCenter alarm using vSphere SDK for Perl. This is actually a trivial task using the vSphere Client, you would first identify the alarm and acknowledge the alarm by right clicking on the alarm and selecting "Acknowledge Alarm". To reset the alarm, you would right click on the alarm and select "Reset Alarm to Green".

To automate this task using the vSphere SDK for Perl which uses the vSphere API, you would perform the same two API operations. Doing a search, you will find a method called AcknowledgeAarm which should does exactly that, but if you try to search for method to reset an alarm, you will notice no such method exists. This was something I was aware of since vSphere 3.5 API, but never understood why it was kept hidden.

UPDATE - The AcknowledgeAlarm API has been made public as of vSphere 7.x and later

Here is a screenshot of the alarmManager in the vSphere MOB and you will notice no methods pertains to resetting an alarm:

You might ask, how is the vSphere Client performing this operation and what API method is it using? Sadly, this operation is one of the many hidden API methods that VMware choose to hide and not public expose for various reasons.

However, there are several ways of identifying some of these hidden vSphere API methods and properties. When you install the vSphere Client on your workstation, there is a catalog directory (e.g. C:\Program Files\VMware\Infrastructure\Virtual Infrastructure Client\4.1\Catalogs\Default)  that is created based on the version of vSphere and within the vim directory, there are these *.vmsg files that contain information regarding the VIM API (Virtual Infrastructure Management API) and searching within these files, you will find some hidden goodies.

Searching the task.vmsg file and within the alarm section, you will see a method name called setAlarmStatus:

alarm.AlarmManager.setAlarmStatus.label = "Set alarm status"
alarm.AlarmManager.setAlarmStatus.summary = "Sets the status of an alarm for an entity"

As you can see, there are other methods with respect to the alarmManager that is documented, but the method above is not documented in the vSphere API reference. We can take this information and further validate by using the vSphere MOB to see what parameters are required for this method.

By generating the proper URL, you can see the method is exposed and the required parameters to this function:

We can perform one additional confirmation to ensure that the method above is actually the method the vSphere Client is performing when resetting an alarm. The tool that we will use here is Onyx, yep, it is not only useful for PowerCLI but for all vSphere developers and administrators. I created a dummy alarm that would trigger if a VM is powered on or powered off and by running Onyx to capture the API calls when acknowledging and resetting an alarm.

Here is the output from Onyx:

As you can see, the two operations uses the AcknowledgeAlarm and the hidden SetAlarmStatus API method. We now can definitively say that the above hidden API method is being used and now we need to figure out how we can incorporate this method into the existing vSphere SDK for Perl.

In this example, I will be working with the vSphere SDK for Perl found on vMA 4.1, but the same set of changes will apply to vCLI 4.x being installed on a Windows or Linux system. There are two client Perl stubs or Perl Modules that contains prototype and definition of a vSphere API method:

/usr/lib/perl5/5.8.8/VMware/VIM25Stub.pm (prototype definition)
/usr/lib/perl5/5.8.8/VMware/VIM25Runtime.pm (method definition)

We will first edit the VIM25Runtime.pm module and if you are on vMA, you will need to use 'sudo' to edit the file. We will add the SetAlarmStatus entry try similar the AcknowledgeAlarm method:

Next we will edit the VIM25Stub.pm and again it will be very similar to an existing method, but now we must populate the parameters based on what we discovered from the vSphere MOB:

Now we are ready to implement this method in a vSphere SDK for Perl Script. I quickly threw together a script that helps manages your vCenter alarms using the CLI by listing all active and triggered alarms in either a red or yellow state.

Download Script: alarmManagement.pl

Here is an example of listing all triggered alarms that are either in a red or yellow state:

Here is an example of acking and resetting the above alarm and utilizing the new hidden API method that was just implemented:

There you have it, a method that was once hidden is no longer =)

If you are interested in locating other hidden API goodies, take a look at this post on How to browse the internal vSphere APIs