If you followed my previous blog post on How to audit VM reconfigurations and see what exactly changed, you may have concluded that historical VM configuration events may potentially be unavailable by the time you need to perform an audit. The reason for this is that the VM Events tables may have rotated out depending on the retention policy of your vCenter Server Database. This is where we can take advantage of the powerful vCenter Server Alarm feature which would allows us to capture every single VM reconfiguration and store this information outside of the vCenter Server. This not only allows you to reduce the amount of data stored in the vCenter Server Database but it also allows you to efficiently archive this data into a data warehouse or Big Data platform that provides more advanced analytics and reporting capabilities.
Building on top of our previous script, I have created a slightly modified version called Get-VMConfigChangesFromAlarm.ps1. The main difference is instead of passing in a VM object to look for past configuration changes, the script can now pull in information from a triggered VmReconfigureEvent and log the specific changes associated with that VM reconfiguration. The way in which it does this is by querying the following two environmental variables which are set when the vCenter Server alarm is triggered and below is an example of their values:
| Alarm Environmental Variable | Value |
|---|---|
| VMWARE_ALARM_TRIGGERINGSUMMARY | Event: VM reconfigured (9322) |
| VMWARE_ALARM_TARGET_ID | vm-125 |
In the first variable, what we care about is the eventId which is associated with the VM reconfiguration. What I have found is you have to take this ID and subtract 1 to get the event which actually contains the reconfiguration information that we want. The second variable provides us with the MoRef ID of the VM that was configured. Using these two pieces of information, we can then perform an event lookup to pull out the configuration changes that were made to that particular VM.
Here are the steps for creating the vCenter Server Alarm:
Step 1 - Ensure that the latest PowerCLI 6.0 Release 1 is installed on your vCenter Server. I will assume that this is how you wish to execute the PowerCLI script. If not, other options can include sending an SNMP trap to vRealize Orchestrator and have it perform the execution of the script.
Step 2 - Create a new Alarm and fill in the general settings as shown in the screenshots below.
Step 3 - Add the "VM reconfigured" trigger with the status set to "Unset", this will ensure you do not have a notification icon when the alarm is activated.
Step 4 - Select "Run a command" as the action and then paste the full path to where a "wrapper.bat" script will be located in the vCenter Server.
Step 5 - Create the "wrapper.bat" script on the vCenter Server system with the following (adjust the paths to fit your environment):
C:\Windows\System32\cmd.exe /c powershell.exe -noninteractive -noprofile -file C:\Users\primp\Desktop\Get-VMConfigChangesFromAlarm.ps1 >> C:\Users\primp\Desktop\alarm.log
The following snippet will execute the PowerCLI script and any console output will be re-directed to the alarm.log file (which can be helpful in troubleshooting errors in the script itself).
Step 6 - Download the Get-VMConfigChangesFromAlarm.ps1 script and place it on the vCenter Server system and ensure it aligns with the path of the wrapper.bat script. You will also need to edit the script to update the vCenter Server credentials as well as the log file in which the VM configurations will be stored. Currently it will just append to a file called alarm.txt.
If everything was configured successfully, you can now test the alarm by simply editing one of your VMs. If you click under Monitor->Events for the VM, you should see the alarm being triggered and that the script was executed.
If we take a look at our alarm.txt file, we should hopefully see the VM reconfiguration details logged like the following:
ChangeVersion : 2015-08-13T21:10:58.248345Z
DeviceChange : {VMware.Vim.VirtualDeviceConfigSpec}
Files : VMware.Vim.VirtualMachineFileInfo
MemoryMB : 8192
NumCPUs : 2
VMName : Test-VM
Start : 8/13/2015 9:11:17 PM
End : 8/13/2015 9:11:19 PM
State : success
User : VGHETTO.LOCAL\Administrator
Device : VirtualDisk
Operation : edit
Logging the output to a file is just one way of storing this data. I am sure some of you may want to modify the script to forward to a remote syslog collector like vRealize Log Insight for example or directly storing it into a Big Data platform. I will leave that as an exercise for my readers but hopefully this give you idea on how you can automatically archive all VM reconfigurations along with the what, when and who details for auditing and/or reporting purposes in the future.
Lastly, I wanted to give a big thanks to Jonathan Medd who helped me with a problem I was running into when trying to get a PowerCLI script to execute when using a vCenter Server Alarm. It turned out that I had the vCenter Server service configure to run as a local admin and switching it over to a Domain Account resolved my problem.
