WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

How to Send vCenter Alarm Notifications to SMS & Other Online Services Using IFTTT

02.02.2013 by William Lam // 2 Comments

I just discovered a pretty cool free online service called IFTTT (If That Then This) which allows you to easily create what is known as a recipe that is composed of trigger and an action to send a notification. This can be thought of like a vCenter Alarm with a trigger and action, but in IFTTT's case, a trigger and action can be almost anything such as an email, facebook event, twitter, etc.

I stumbled onto IFTTT while going through some of my unread blog feeds this morning since I could not go back to sleep after randomly waking up at 4am in morning. I was reading an article from Matt Cowger about something called a Pebble which I had not heard of before and it peaked my interest. In reading about Pebble (pretty slick actually), I learned about IFTTT which is one of the notification systems Pebble supports. I decided to give IFTTT a try and to see how easy it would be to setup SMS notifications for a vCenter Server alarm.

Step 1 - Sign up for IFTTT account, you need to ensure the email account that you use to register is the same account that will be used to send the trigger.

Step 2 - Create a new Recipe using email as the trigger and SMS for the action. The process is very straight forward, just follow the wizard and at the end you will enter your phone number for SMS notification and confirm with a code.

Step 3 - Create a vCenter Alarm for the event you wish to trigger off of and set the email address to send to *protected email* as noted in the Recipe.

Step 4 - In my test, I created an alarm for a Powered Off event and went ahead and powered off a VM to generate the alarm.

Step 5 - IFTTT checks the triggers every 15minutes. If you do not wish to wait 15minutes you can force a check by clicking on Check Now button in the Recipe.

Step 6 - If everything was setup correctly, you should have received a text message with the details of the vCenter alarm. In my Recipe, I configured it to only send the Subject which contains everything you would need to know about the vCenter alarm, at least for you to decide whether or not you want to investigate the issue.

The possibilities are pretty much endless in terms of the triggers and actions that you could create with IFTTT, I even created one for my ghettoVCB backup notification. The only downside that I noticed while giving IFTTT a try is that the trigger check is every 15minutes which could be a bit long for things requiring immediate attention, but I also read that there are certain Recipes that supports a "Quick Trigger" which would then execute immediately upon receiving. I think for a free service this is very cool and much easier than setting up your own SMS system. I would recommend giving IFTTT a try and see what cool Recipes you can build and integrate with your VMware or other environments.

I hope my body will let me crash and get some sleep now ...

Categories // Uncategorized Tags // alarm, ESXi, ifttt, notification, sms, text, vSphere

Detecting A Duplicate IP Address For Your ESXi Hosts Using a vCenter Alarm

01.28.2013 by William Lam // 6 Comments

The motivation for this article was a tweet I noticed from Duncan Epping this morning. Per Duncan's tweet, it looks like he may have accidentally assigned an IP Address to one of his virtual machines which was already being used by an existing ESXi host causing a duplicate IP Address error. We probably have all experienced this once in our lives and it can be quite difficult and frustrating to troubleshoot. Similar to a Windows OS, ESXi can also detect a duplicate IP Addresses but instead of a notification window, it is just logged in the VMkernel logs which looks like the following:

2013-01-21T15:52:35.989Z cpu1:2049)Tcpip_Vmk: 112: arp: 00:50:56:bd:3b:2b is using my IP address 172.30.0.213 on vmk1! 

The biggest challenge of course is to identify which ESXi host actually has a conflict and then taking a look at the logs to find the offending MAC Address and shutting them down yourself or with the help of a Network Administrator. Wouldn't it be great if we had an alarm to automatically notify us when a duplicate IP Address is detected? Well I am glad you asked and the answer is YES! 🙂

In addition to logging to the VMkernel logs, ESXi also logs this "observation" in /var/log/vobd.log which stands for the VMkernel Observation. These "observations" can provide critical identifying information in case of an error and is usually used during troubleshooting. In our case, we are seeing an intermittent network connectivity to our ESXi host which is in result of a duplicate IP Address. The really neat thing about these VOBs is that you can create vCenter Alarms when a specific VOB has been detected. I have shown an example of this before in my Detecting ESXi Remote Syslog Connection Error Using a vCenter Alarm article.

We can do exactly the same for detecting a duplicate IP Address for an ESXi host. The first thing we need to do is identify the VOB ID by looking in /var/log/vobd.log:

2013-01-21T15:02:07.513Z: [netCorrelator] 917174784727us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for 172.30.0.83 on interface vmk0, current owner being 00:50:56:bd:3b:2b

We can see the VOB ID for this is esx.problem.net.vmknic.ip.duplicate and this will be used in our vCenter Alarm trigger.

Step 1 - Create a new Alarm and specify a name, the Monitor type will be Hosts and Monitor For will be for a specific event:

Step 2 - Copy the VOB ID that we have identified from above and specify that as our alarm Trigger:

Step 3 - If you wish to receive an email notification or send an SNMP trap go ahead and configure additional actions, else just click next which will just display a vCenter Server alert in the UI.

Now that our alarm has been created, we will want to give this a test drive .... who can we ask? Well it just happens that I have a new user in my environment and I provisioned him a new VM which is already connected to the network. Let's hope he does not try to change the IP Address (because this never happens, right?)

After the user statically assigns the IP Address of an existing ESXi host in the VM, we should see our new alarm trigger in vCenter.

As you can see, we have quickly identified the ESXi host that is impacted and we can then login to DCUI via the console to take a look at the logs to find the offending MAC Address. Hopefully duplicate IP Addresses is not a common problem in your environment but it does happen from time to time and having an alarm to help you quickly narrow down the culprit can be quite useful.

Categories // Uncategorized Tags // alarm, duplicate IP, ESXi, ip address, vob, vSphere

Automatically Remediating SvMotion / VDS Issue Using vCenter Alarms

04.20.2012 by William Lam // 8 Comments

UPDATE 07/13/2012 - vSphere 5.0 Update 1a has just been released which resolves this issue, please take a look here for more details for the patch as this script is no longer required.

In my previous article Identifying & Fixing Virtual Machines Affected By SvMotion / VDS Issue, I provided a script for users to easily identify the impacted VMs as well as a way to remediate them. Though the the issue was only temporarily fixed, as any of the remediated VMs can be re-impacted if they are Storage vMotion again (manually or automatically) by Storage DRS. This meant that users would to re-run these scripts every so often to ensure their environment is not affected by this problem.

I decided to look into a more automated and hands-off approach in which a Storage vMotion of a VM will automatically trigger the execution of the remedation script. I was able to successfully accomplish this by leveraging vCenter Alarms and running a script on the vCenter Server (Here's a cool thing I did with alarms awhile back) .

Disclaimer: This script is not officially supported by VMware, please test this in a development environment before using on production systems.

You can create the alarm at any level of the inventory hierarchy, but I would recommend placing it at least at the datacenter or cluster level. The alarm type will be for a VirtualMachine and it we use "monitor for specific events". For the trigger, we will need to use "VM migrated" and set the status to "Unset" which will not create an alarm icon when it is triggered.

You might wonder why we selected "VM migrated" versus "VM relocated" and this is actually due to the fact that a Storage vMotion starts out just like a vMotion and if you manually perform a vMotion or Storage vMotion, only this event type will be triggered. Due to this single event being triggered by two completely different operations, it has an interesting impact which we will discuss in a bit.

Next we need to create an action for this alarm which will be running a command, you will need to specify the full path to perl.exe (assuming you're using my script which is based on vSphere SDK for Perl and you will need to have vCLI installed on the vCenter Server) as well as the path to the alarm script which in this example is called alarm.pl. Also ensure you set the green->yellow action to execute once.

You will need to create the alarm.pl script on your vCenter Server and here is what it looks like:

#!/usr/bin/perl -w
# William Lam
# http://www.virtuallyghetto.com/

use strict;
use warnings;

my $scriptlocation = "C:\\querySvMotionVDSIssue.pl";
my $server = "localhost"
my $username = "VC-USERNAME";
my $password = "VC-PASSWORD";
my $debug = 0;

###########################
# DO NOT MODIFY PAST HERE #
###########################

my $start1 = "from";
my $start2 = "to";
my $end = ",";

# extract VMware env variables from alarm
my $eventstring = $ENV{'VMWARE_ALARM_EVENTDESCRIPTION'};
my $vmname = $ENV{'VMWARE_ALARM_EVENT_VM'};

my @sourcehost = $eventstring =~ /$start1 (.*?)$end/;
my @destinationhost = $eventstring =~ /$start2 (.*?)$end/;


# Output environmental variables to see what's up
if($debug) {
 open(FILE,">C:\\output.txt");
 foreach my $key (keys %ENV) {
  print FILE $key . "=" . $ENV{$key} . "\n";
 }
 close(FILE);
}

# if the source/destination host is the same, means we had a Storage vMotion instead of vMotion
# and we execute the remediation script on the VM
if($sourcehost[0] eq $destinationhost[0]) {
 `"$scriptlocation --server $server --username $username --password $password --vmname $vmname --fix true"`;
}

You will need to fill in the script location, in this example I have all scripts stored in C:\ and you will also need to populated the credentials which will be used to execute the script.

Earlier we mentioned that both a Storage vMotion and vMotion trigger the same event and because of that, we need to be able to identify when a Storage vMotion actually happens to run the script. The alarm.pl script above will be executed when the alarm is triggered and using the VMware specific environmental variables that is populated from the vCenter Alarm, we can extract from the event description to figure out whether it was a vMotion or Storage vMotion. Once we confirm it is a Storage vMotion, we then execute our remediation script which is from my previous article.

Note: Ensure you download the latest version of of the querySvMotionVDSIssue.pl from the previous article, as it has been updated to handle single remediation and targeted for this use case.

Now to verify that our alarm is functioning as expected, we can perform a manual Storage vMotion of a VM and we should see our alarm.pl execute and then after the Storage vMotion has completed, we should see some VM reconfiguration tasks which is from our remediation script.

So there you have it, you no longer have to worry about running the script every so often to ensure your VMs are not being impacted by the SvMotion / VDS problem. Again, I would like to stress though we are able to automate this remediation, this is not a real solution and VMware is actively working on a fix for this problem.

If you have any questions, feel free to leave a comment.

Categories // Uncategorized Tags // alarm, distributed virtual switch, dvportgroup, dvs, storage drs, svmotion, vds

  • « Previous Page
  • 1
  • …
  • 3
  • 4
  • 5
  • 6
  • 7
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025