Application Awareness HA is not a new feature in vSphere 5, it has actually has been around since vSphere 4.1. With this feature, vSphere HA can monitor heartbeats generated from an application monitor within the guestOS and reboot the virtual machine.
What is actually new in vSphere 5 is the availability of the Application Awareness API for anyone to consume and integrate into their own application and/or script. Prior to this, the API was only exposed to ISV and 3rd party vendors with solutions such as Symantec's ApplicationHA and Neverfail's vAppHA.
The Application Awareness API (will be shorthanded as AAA, going forward) is supported in both Linux and Windows (32/64bit) and can be accessed by installing a package within the guestOS. This package includes the necessary AAA libraries to create your own program/scripts in C, C++, Java and Perl. In addition, the package also includes a pre-compiled binary (vmware-appmonitor) that implements all the AAA methods that can easily be called from within a script or program. AAA uses the VMware Tools as communication channel to the ESX(i) host and you will need to ensure VMware Tools is installed and running. Since the communication is between VMware Tools and the ESX(i) host, there is no reliance on a TCP/IP network for this communication channel.
UPDATE: You can download GuestAppMonitor SDK here.
There are currently 6 AAA methods:
- VMGuestAppMonitor_Enable()
- Enables Monitoring
- VMGuestAppMonitor_MarkActive()
- Call every 30 seconds to mark application as active
- VMGuestAppMonitor_Disable()
- Disable Monitoring
- VMGuestAppMonitor_IsEnabled()
- Returns status of Monitoring
- VMGuestAppMonitor_GetAppStatus()
- Returns the current application status recorded for the application, three possible values:
- green = Virtual machine infrastructure acknowledges that the application is being monitored.
- red = Virtual machine infrastructure does not think the application is being monitored. The
HA monitoring agent will initialize an asynchronous reset on the virtual machine if the status is Red - gray = Application should send VMGuestAppMonitor_Enable again, followed
by VMGuestAppMonitor_MarkActive, because either application monitoring failed, or the virtual machine was vMotioned to a different location
- Returns the current application status recorded for the application, three possible values:
- VMGuestAppMonitor_Free()
- Frees the result of the *_GetAppStatus() call (only required when writing your own program)
Here is the basic workflow for using AAA within your application:
To start using AAA functionality, you will first need to have a vSphere HA enabled cluster and enable the "VM and Application Monitoring" under VM Monitoring.
You have the ability to configure the sensitivity of AAA from Low, Medium and High which correlates to the heartbeat interval and frequency of virtual machine reboots. You also have the option of configuring your own custom policy.
Lastly, you can choose which virtual machines will be included in VM Monitoring and/or Application Monitoring.
Note: It is important to note, that HA will expect an application heartbeat to be generated every 30secs. If HA fails to receive a heartbeat within 30secs, it will transition the appHeartbeatStatus state from green to red. Depending on the configured sensitivity policy, once the heartbeat interval has been violated, HA will then restart the virtual machine. For example, if you have the sensitivity configured to medium and a heartbeat was not received within 30secs, it will change to a red state. If HA still has not received a heartbeat within 60secs of that time, then it will reboot the virtual machine.
Step 1 - Copy the AAA package to Linux system and extract the contents
Step 2 - Change into VMware-GuestAppMonitorSDK/docs/samples/C/ and ensure you have gcc compiler. You may have to change the makefile if you are on a 64bit platform as it is configured by default to point to the 32bit library. When you are ready, just type "make" and you should get compiled binary called "sample" which is the sample C application
Before you run the application, you need to ensure that your shared library path variable LD_LIBRARY_PATH has been updated to include the libappmonitorlib.so. To update the variable, you will run the following command:
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/full/path/to/VMware-GuestAppMonitorSDK/lib64
Step 3 - You can now run the "sample" application which runs in a continuous loop and automatically enables AAA within the virtual machine and sends heartbeats to ESX(i) host. You can press control+C which then brings up three options: stop (s), disable (d), continue (c). The last two options should be pretty straight forward, but if you decide to stop the heartbeating and you don't resume, you will see HA restart the virtual machine based on your cluster configuration.
As you can see, once the heartbeats have not been received within the specified interval, HA will take action and reboot the virtual machine as expected. Here is a detail of the events as seen by vCenter and the HA cluster:
Here is an example of installing AAA on a Windows system and using the pre-compiled vmware-appmonitor binary:
Step1 - Copy the AAA package to Windows system and extract the contents
Step 2 - Launch a cmd prompt and change into C:\Users\Administrator\Desktop\VMware-GuestAppMonitorSDK-457765\VMware-GuestAppMonitorSDK\bin\win64 directory. Depending if you are on a 32/64bit OS, you will need to modify the win{32,64}
Step 3 - Run the vmware-appmonitor.exe which will then provide you with options: enable, disable, markActive, isEnabled, getAppStatus
Note: The options in vmware-appmonitor for both Linux and Windows are exactly the same, this is very nice for consistency and scripting purposes. Just like with the direct use of the API, you need to first run the enable command to turn on Application monitoring and then run the markActive command which sends the heartbeats. You can always check the current heartbeat status by running getAppStatus or check whether AAA is enabled by running isEnabled command
As a reference, here are the paths to the vmware-appmonitor for both Linux and Windows:
- VMware-GuestAppMonitorSDK/bin/bin{32,64}/vmware-appmonitor
- VMware-GuestAppMonitorSDK-457765\VMware-GuestAppMonitorSDK\bin\win{32,64}\vmware-appmonitor.exe
For those of you who are not into programming languages such as C,C++ and Java, here is an example using Perl. In the example, the script simulates the monitoring of an application by checking whether or not a file exists. The script starts off by creating a file that will monitored and then loops for 5 minutes and checks for the existence of the file. Once the 5 minutes are up, the script then disables Application monitoring and exits.
Note: You will need to set the two variables at the top which define the path to the shared library and the vmware-appmonitor binary.
So far we have demonstrated on how to setup AAA within the guestOS and provided a variety of programming/scripting interfaces such as C,C++, Java and Perl to integrate with your own application/script. Now what if we wanted to extract the heartbeat status for all virtual machines that have AAA implemented going through vCenter? You can easily do so by using the vSphere API and querying the appHeartbeatStatus property of your virtual machine.
I wrote a very simple vSphere SDK for Perl script getVMAppStatus.pl that allows you to query a virtual machine connecting to either vCenter or directly to an ESX(i) host to extract the heartbeat status.
Download the getVMAppStatus.pl script here.
The script can return three different status: gray, green or red and the definition for each is defined above.
Now before you jump right in and start leveraging this awesome API in either a custom application or script, you need to understand your application and various ways on detecting that it has failed and when you would like vSphere HA to reboot the virtual machine. Simply checking whether the process is running may or may not be enough.
To get more details on some of the best practices around using the Application Awareness API, I would highly recommend you check out Tom Stephens upcoming VMworld 2011 presentation TEX1928 Implementing Application Awareness in the Web Client and The Uptime Blog for more details about implementing AAA . For now, if you would like to learn more about Application Awareness API, check out last year's VMworld presentation.
New vSphere 4.1 CLI Utilities Marketing Did Not Tell You About Part 1
With the new release of vSphere 4.1, there are new additions to the CLI utilities that administrators can leverage for configurations and troubleshooting. Although, not all were treated equally from an announcement and documentation standpoint.
Here is an example of the new version of vmkfstools and -D option:
2. storageRM is a debugging utility for Storage I/O Control at the host level.
[root@esx4-1 ~]# /usr/lib/vmware/bin/storageRM -h
Default Values:
SleepTime: 4000
Threshold: 35
Gamma: 0.25
Beta-per-host: 4.00
LowerBound: 4
UpperBound: 64
Storage I/O Control-- This tool does flow control at the host
in order to maintain disk I/O latency close to a
threshold and queue sizes converge at values
proportional to the beta parameter.
The following histogram related options are available:
-a, --print the list of all luns, their latency threshold,
queue depths and if Storage I/O Controlis set/unset
-b, --Beta per host value
-d
-f, --force the run without checking version or checksum
-g, --defGamma value for use in control equation
-h, --help will print the usage
-l, --lower bound on the length of lun queue, (default 4)
-n, --no anomaly detection is done
-r, --reset issue queue for all luns to default
-s, --sleep time in ms for periodic execution
-t, --threshold on the latency (in ms), for rate control
-u, --upper bound on the length of lun queue (default 64)
-v, --debug log level value
storageRM Usage:
storageRM [options]
3. net-lbt is a debugging utility for the new Load-Based Teaming feature.
[root@esx4-1 ~]# /usr/lib/vmware/bin/net-lbt -h
Usage: [-d] [-t time] [-v] [-s threshold]
-d run in daemon mode
-t daemon sleep period in seconds, minimum 10 seconds
-v run with verbose logging
-s saturate threshold [10, 100], i.e. 60 for 60% of line rate
4. net-dvs is a debugging utility for Distributed vSwitch.
[root@esx4-1 ~]# /usr/lib/vmware/bin/net-dvs -h
Warning: This is an unsupported command. Use at your own risk.
net-dvs -a [ -P maxPorts] switch_name
net-dvs -d switch_name
net-dvs [ -A | -D ] -p port switch_name
net-dvs [ -s name=value | -u name ] -p port switch_name
net-dvs -l [ switch_name ]
net-dvs -i (init database)
net-dvs [-S | -R | -G ]
net-dvs -T
net-dvs -v "vlanID[;t|p[0-7][;min-max,min-max...]]
net-dvs -V "primaryVID,secondaryVID,i|c|p;primaryVID,secondaryVID,i|c|p..."
net-dvs -m "sid;dname;snaplen;
[oiveld];encapvlan;wildcardsIn,wildcardsOut;dstPort1,dstPort2,...;srcInPort1,srcInport2,...;srcOutPort1,srcOutPort2,...;:sid2;dname2..."
net-dvs dvswitch -k "respool1_id;respool2_id;..."
net-dvs dvswitch -p dvport -K "respool1_id:shares:limit;respool2_id:shares:limit;..."
net-dvs dvswitch -p dvport -z "respool_id"
net-dvs dvswitch -j [activate|deactivate]
net-dvs -L uplink_name1[,uplink_name2,...] -t team_policy_type -p port switch_name
net-dvs dvswitch -H "red|yellow|green:some message" switch_name
net-dvs -o "depth,param|classname;depth,param|classname;... -p port|globalPropList switch_name
net-dvs --mtu mtu_value [-p dvport] switch_name
net-dvs --x 0|1 -p dvport switch_name
net-dvs --vlan vlanID -p dvport switch_name
net-dvs --reset -p dvport switch_name
net-dvs --cap cap_value -p dvport switch_name
net-dvs --states -p dvport switch_name
5. remoteDeviceConnect is a new utility that allows you to mount various remote devices including floppy and USB.
/usr/lib/vmware/bin/remoteDeviceConnect: option requires an argument -- h
VMware remote Device Connect
Usage: /usr/lib/vmware/bin/remoteDeviceConnect OPTIONS filename
Options:
-h
-p
-t
-d
-n
-f
-A : use Authd to connect
-U
-V
-P
Examples:
remoteDeviceConnect -t floppy -d floppy0 -f device /dev/fd0
remoteDeviceConnect -t floppy -d floppy0 -f file image.flp
remoteDeviceConnect -t usb "path:2/1 vid:0x0547 pid:0x2131"
6. sensorD looks to be a debugging utility that can connect to an ipmi device.
[root@esx4-1 bin]# /usr/lib/vmware/bin/sensord
sensord: failed to open ipmi device: No such file or dir
sensord: unsupported hardware
7. statedumper looks to be a debugging utility for output information about the system and its states.
[root@esx4-1 bin]# /usr/lib/vmware/bin/statedumper -h
statedumper [-f filename] [-s off] [-e off] [-b] [-o] [-r] [-x]
The following options are supported:
-e
-f
-o - output entry offsets
-r
-s
-b - filter on branch count, use -s and -e for start/end
-x - dump extra debug data
8. vmkeventd looks to be a utility for capturing VMkernel events
[root@esx4-1 bin]# /usr/lib/vmware/bin/vmkeventd -h
/usr/lib/vmware/bin/vmkeventd: invalid option -- h
Usage: /usr/lib/vmware/bin/vmkeventd [-d]
9. analyze-esx-init-boot.py looks like a debugging utility to analyze the COS boot up logs
[root@esx4-1 ~]# /usr/sbin/analyze-esx-init-boot.py -s -S /vmfs/volumes/datastore1/esxconsole-4c27dd75-a38d-5044-5670-005056927558/logs/sysboot.log -V /var/log/messages
ERROR: Could not find 'cpu 0: early measured tsc speed' in the log
This is one of the very first log messages after boot
POST boot times will not be relevant
Unable to find vsish. Please ensure that you have debugging tools installed and your PATH is correctly setup.
VMKernel: 0.0000 secs
POST Tests: 0.0000 secs
Init Scripts: 0.0000 secs
ESX Boot Time: 0.0000 secs
Hardware/BIOS: 0.0000 secs
Total Boot Time: 0.0000 secs
Serial port output was on
VCF 9.1 - Auditing vCenter Server Connections using the Connection Utilization API
vCenter Server has had the ability to audit vSphere logins, whether through the API or UI, for nearly two decades using vSphere Events, which provide detailed information on who connected, when the login occurred and the client IP address associated with the session.
While looking up something in the latest vSphere 9.1 Automation REST API, I came across a new Connection Utilization API that provides visibility into all HTTP and HTTPS connections established with vCenter Server.
While most organizations deploy vCenter Server on a dedicated management network as a best practice, it does not eliminate the possibility of unexpected or unauthorized connections. Having additional visibility into those connections and the ability to audit them can help organizations quickly identify and investigate suspicious activity.
- « Previous Page
- 1
- …
- 15
- 16
- 17
- 18
- 19
- …
- 38
- Next Page »













