WilliamLam.com

Using the vSphere API to remotely generate ESXi performance support bundles

by // 2 Comments

This is a follow-up post from my previous article on how to run a script using a vCenter Alarm action in the vCenter Server Appliance (VCSA). What I had demonstrated was using a pyvmomi (vSphere SDK for Python) script to be triggered automatically to generate a VMware Support Bundle for all ESXi hosts for a given vSphere Cluster. The one caveat that I had mentioned in the blog post was that the solution slightly differed from the original request which was to create an ESXi performance support bundle. However, the vSphere API only supports the creation of generic VMware Support Bundle which may not be as useful if you are only interested in collecting more granular performance stats for troubleshooting purposes.

After publishing the article, I had thought about the problem a bit more and realized there is still a way to solve the original request. Before going into the solution, I wanted to quickly cover how you can generate an ESXi Performance Support Bundle which can be done either directly in the ESXi Shell using something like the following:

vm-support -p -d 60 -i 5 -w /vmfs/volumes/datastore1

or you can actually use a neat little trick which I had blogged about here back in 2011 where you can simply open a web browser and run the following:

https://esxi-1.primp-industries.com/cgi-bin/vm-support.cgi?performance=true&interval=5&duration=60

Obviously, the first option is not ideal as you would need to SSH (generally disabled for good security practices) to each and every ESXi host and then manually run the command and copy the support bundle off of each system. The second option still requires going to each and every ESXi host, however it does not require ESXi Shell or SSH access. This is still not ideal from an Automation standpoint, especially if these ESXi hosts are already being managed by vCenter Server.

However, the second option is what gave me the lightbulb idea! I had recalled a couple of years back that I had blogged about a way to efficiently transfer files to a vSphere Datastore using the vSphere API. The solution leveraged a neat little vSphere API method called AcquireGenericServiceTicket() which is part of the vCenter Server sessionManager. Using this method, we can request a ticket for a specific file with a one time HTTP request to connect directly to an ESXi host. This means, I can connect to vCenter Server using the vSphere API, retrieve all ESXi hosts from a given vSphere Cluster and request a one time ticket to remotely generate an ESXi performance support bundle and then download it locally to the VCSA (or any other place that you can run the pyvmomi sample).

Download the pyvmomi script: generate_esxi_perf_bundle_from_vsphere_cluster.py

Here is the sample log output when triggering this script from vCenter Alarm in the VCSA:

2016-06-08 19:29:42;INFO;Cluster passed from VC Alarm: Non-VSAN-Cluster
2016-06-08 19:29:42;INFO;Creating directory /storage/log/esxi-support-logs to store support bundle
2016-06-08 19:29:42;INFO;Requesting Session Ticket for 192.168.1.190
2016-06-08 19:29:42;INFO;Waiting for Performance support bundle to be generated on 192.168.1.190 to /storage/log/esxi-support-logs/vmsupport-192.168.1.190.tgz
2016-06-08 19:33:19;INFO;Requesting Session Ticket for 192.168.1.191
2016-06-08 19:33:19;INFO;Waiting for Performance support bundle to be generated on 192.168.1.191 to /storage/log/esxi-support-logs/vmsupport-192.168.1.191.tgz

I have also created a couple more scripts exercising some additional use cases that I think customers may also find useful. Stay tuned for those additional articles later this week.

UPDATE (06/16/16) - There was another question internally asking whether other types of ESXi Support Bundles could also be generated using this method and the answer is yes. You simply just need to specify the types of manifests you would like to collect, such as HungVM for example.

To list the available Manifests and their respective IDs, you can manually perform this operation once by opening browser specifying the following URL:

https://192.168.1.149/cgi-bin/vm-support.cgi?listmanifests=true

Screen Shot 2016-06-16 at 5.38.08 AM
To list the available Groups and their respective IDs, you can specify the following URL:

https://192.168.1.149/cgi-bin/vm-support.cgi?listgroups=true

Screen Shot 2016-06-16 at 5.45.56 AM
Here is an example URL constructed using some of these params:

https://192.168.1.149/cgi-bin/vm-support.cgi?manifests=HungVM:Coredump_VM%20HungVM:Suspend_VM&groups=Fault%20Hardware%20Logs%20Network%20Storage%20System%20Userworld%20Virtual&vm=FULL_PATH_TO_VM_VMX

The following VMware KB 2005715 may also be useful as it provides some additional examples on using these additional parameters.

vCenter Server 6.0 Tidbits Part 12: New methods of downloading Support Bundles for VCSA / PSC

by // Leave a Comment

Many of you are probably pretty familiar with the process of generating a VMware Support bundle for vCenter Server when it comes to troubleshooting or filing an SR with VMware GSS by using either the vSphere Web/C# Client UI. With the released of vSphere 6.0 and specifically with the vCenter Server Appliance (VCSA) there are now some additional methods in generating a VMware Support bundle which can come in handy if the vSphere Web Client is not running or if you wish to perform this simply through the command-line on a remote system.

Screen Shot 2015-06-14 at 7.06.39 AM
The first option is simply by pointing your web browser to the following URL of your VCSA:

  • https://192.168.1.60/appliance/support-bundle

vcsa-support-bundle
You will be prompted to login with a local account such as the "root" user which ensures that there is no dependency that SSO must be running which is the case when using the vSphere Web Client. Once authenticated, the VMware Support bundle will be generated and you will then be able to download it onto your system. The VMware Support Bundle is in the format of a compressed tar file which you can use gzip or tar to extract.

The second option is simply using cURL or wget from the command-line which you will also need to provide valid credentials to download. Here is an example of using curl and we will be saving the file as "support-bundle.tar.gz":

curl -k -u root -o support-bundle.tar.gz -O https://192.168.1.60/applmgmt/support-bundle

Once we have downloaded the VMware Support bundle to our desktop, we can then extract it using a variety of tools such as tar for example:

tar -zxvf support-bundle.tar.gz

In situations when you need to quickly resolve a problem by providing support logs to VMware, time is of the essence and being able to quickly generate the necessary support files can help lead to a quick resolution. Hopefully these additional methods of generating a VMware Support Bundle can help save you time when you may need to call upon them.