WilliamLam.com

How to query for MACs on internal vSwitch on ESXi

by // 10 Comments

There was an interesting question this week on the VMTN community forums about querying a vSwitch on an ESX(i) host. The user was trying to locate a particular virtual machine's MAC Address due to an IP conflict that was identified. The internal VMware vSwitch is pretty much closed off as a blackbox. The vSwitch is not exposed like a traditional physical switch in which you can run commands against such as "show mac-address-table" to display the MAC addresses found on the switch.

However, you can still perform a lookup of all the MAC Addresses found on a particular ESX(i)/vCenter host by using the vSphere APIs. You can search for all virtual machines and dump out their associated MAC Addresses and correlate that back to a particular vSwitch. You can easily do this through a script such as using the vSphere SDK for Perl script: getvSwitchMacTable.pl which supports both stand vSwitch and distributed vSwitch or if you prefer a GUI, you can use the popular RVTools. I am sure there is most likely a PowerCLI solution to solving this problem as well.

The solution described above is the proper and most flexible way of solving this problem, but what if you really wanted to query the internal vSwitch and extract out the MAC Addresses that way? Well the answer is, you can so using vsish on ESXi (vsish is not available on ESX unless you have the VMware debugging RPM package installed).

Here are some of things you can view for a given vSwitch using vsish:

~ # vsish -e ls /net/portsets/vSwitch0
ports/
overlays/
uplinks/
type
mtu
unlink
link
destroy
properties
stats

The "ports" section is what we are interested in:

~ # vsish -e ls /net/portsets/vSwitch0/ports
16777217/
16777218/
16777219/
16777220/
16777358/
16777359/

When looking at a particular port, it provides quite a bit of information on what is connected and various metrics/statistics:

~ # vsish -e ls /net/portsets/vSwitch0/ports/16777220/
respool/
e1000/
vmxnet3/
pktSizes/
clusterSizes/
worlds/
coalesceDetailed/
ip
schedTeamUplink
teamUplink
blocked
injectIGMP
txCompCoalesce
txCoalesce
rxCoalesce
controlChain
notifyStats
inputStats
outputStats
vmxnet2clientStats
clientStats
gateway
setPassthru
status
stats

As you can see it is pretty tedious to go through each of the ports and it does not easily allow you to figure out what is exactly connected to the port until you view the "status" property.

I decided to write a tiny script that would allow a user to dump out all the MAC Addresses from the vSwitch(s) found on an ESX(i) host. Not only does it provide this mapping but also what is specifically using a given port whether it is mapped to internal interface or a particular virtual machine.

You can download the script vswitchInfo.sh which runs directly on ESXi's TSM (Tech Support Mode). The script can be called with the "-l" option to provide a high level dump of all MAC Addresses. Once you have identified the particular vSwitch and port, then you can get further details by specifying "-v" for vSwitch name and "-p" for the port number as displayed from the previous execution.

Here is an example output of just listing all MAC Addresses from all vSwitch(s) in an ESXi host:

Here is an example of getting more details on a particular port on a vSwitch:

Here you can see the clientName which is either a VM or interface using the port. You will also notice there is a mapping to set of pNICS that are attached to the vSwitch and various other details that I will let you explore.

You might have noticed the vSwitch port-ids looks kind of familiar? If you did, they actually are, as they part of the "networking" section in esxtop/resxtop output.

Unfortunately with esxtop/resxtop, it does not display the associated MAC Addresses, but now you have a way to easily query for details on the internal ports of a vSwitch.

Note: The second solution falls under the "not supported" category as you might have guessed.

ESXi Migration Worksheet Script

by // 7 Comments

Over the weekend Duncan Epping released a new whitepaper called VMware ESXi 4.1 Operations Guide which documents some of the operational tasks for managing/configuring an ESXi environment. Not only is this a really cool document, but I also found that I was referenced in the whitepaper. This was really neat as it is probably my first reference ever in a whitepaper 🙂

While looking through the other links, I noticed there was another whitepaper written by Duncan's colleague, Kyle and it referenced a host configuration worksheet. When I opened up the document, it was basically a plain PDF form requiring users to manually input some of the important configurations of an ESX host prior to upgrading to ESXi. I thought this would be fine if you only had a few hosts to upgrade, but what if you had several dozens or hundreds of hosts? Manually editing the non-editable PDF document meant you had to use a PDF tool or worse printing it out and writing in the information by hand.

I thought, why not automate it? I decided create a very simple vSphere SDK for Perl script called generateHostConfigurationWorksheet.pl that would allow a user to connect to either a standalone ESX host or to a vCenter server. The script also allows you to specify a list of ESX hosts to extract the information when connecting to vCenter, else by default it will collect information for all your ESX hosts. You also have the option of selecting either an html or csv output.

To run this script, you will need a system that has the vCLI installed or you can use VMware vMA. You will also need to download the script from here. Make sure you set the script to be executable

The script accepts two paraemters:
--output [html|csv] (required)

--hostlist [file-containing-esx-hosts] (optional)

Here is an example of connecting directly to an ESX host:

[vi-admin@tancredi ~]$ ./generateHostConfigurationWorksheet.pl --server vesx41-1.primp-industries.com --username root --output html
Enter password:
Processing vesx41-1.primp-industries.com
Generating vesx41-1.primp-industries.com.html ...

Here is an example of connecting to vCenter and specifying a list of ESX hosts to collect information on:

[vi-admin@tancredi ~]$ ./generateHostConfigurationWorksheet.pl --server reflex.primp-industries.com --username primp --output csv --hostlist hostlist
Enter password:
Processing vesx41-1.primp-industries.com
Generating vesx41-1.primp-industries.com.csv ...

Note: If you like to select the ESX hosts you want to report on, just create a file that has the displayname of your ESX host seperated by a newline

Here is sample report that you can view

You will notice that syslog section is left blank, this is done intentionally as with ESX, syslog configuration is not exposed through the vSphere API like ESXi. This will be something you will need to fill in manually.

I think this worksheet will be useful, but if you are interested in a more extensive report prior to upgrading, I would highly recommend you check out the popular vSphere Health Check script and here is what one of those reports can look like.

CIM monitoring caveat with ESXi

by // 9 Comments

I recently started to play around with the CIM API to monitor the hardware on an ESXi host. Instead of relying on hardware agents/scripts that would normaly run in the traditional Service Console of ESX, CIM is an API that allows you to monitor the health of your ESX or ESXi host. You can see the health status of your host by logging directly into the ESX(i) host using the vSphere Client and clicking on Configuration->Health Status tab.

I decided to start off with a small python script that would run on vMA and using the wbem python module to make a simple connection to query for the ESXi version. Here are the steps to get the following script working:

1. Download pywbem onto vMA

2. Extract the contents of pywbem

tar -zxvf pywbem-0.7.0.tar.gz

3. Install pywbem

sudo python setup.py install

4. The script expects the hostname/IP of your ESX(i) host as it's first argument and the username as the second argument and then you will be prompted for the password

./cim.py himalaya.primp-industries.com root

If everything went according plan, you should see the version of your ESX(i) host printed on the screen.

Next I wanted to create a dedicated service account so that I do not have to use the root account, I thought a read-only role would suffice.

To my surprise, when I ran the script again with this user, I received an unauthorized access error.

At first I thought the user account required an "Administrator" role to perform the operation but after further investigation, I found that the user account must be part of the "root" user group. Even for a read operation, it still needed to be in that that group.

After I made the change using the vSphere Client and re-ran the script, it executed as I expected.

After speaking to someone at VMware regarding this issue, it was confirmed by engineering that this is in fact a software bug and it should not require the user account to be part of the "root" user group to query from the CIM API.

I was still interested in using CIM, but I wanted to lock down the account as much as possible and came up with the following snippet of code which can be included in your ESX(i) kickstart configuration.

The script creates a regular user who does NOT have login access to the ESX(i) host. It then puts the user into the "root" user group and then creates a new role called CIM with a single privilege Host.Cim.CimInteraction and then associates this user with this role.This ensures that the account can only perform read-only operations against the CIM API and does not allow for host logins. Until the bug is resolved, this should be an acceptable work around.

So what type of monitoring can you do with CIM? Well pretty much anything and everything. There is a popular Nagios script that monitors the hardware health of an ESXi host using the CIM API called check_esxi_webem.py that one can implement to alert on your hardware components.

The script currently expects three arguments: hostname/IP of ESX(i) host, username and password on the command line (this can be changed with minor modifications). If you run it using those defaults, you will either get an OK or WARN/ERROR which will include additional information about the component that is alarming.

If you would like to get more details on the components being checked, you can pass in a fourth parameter called "verbose" and the script will provide more information on what is being checked.

If you are not big on python, there is also a Perl SDK for CIM/WSMAN as part of the vCLI installation and if you are using vMA, you can find some great examples under /usr/share/doc/vmware-vcli/samples/WSMan

The checksensorhealth.pl is definitely one to take a look at, here is an example output:

If you are interested to learn more about CIM, take a look at the these resources:
CIM SDK
7 Part series on ESXi Chronicles blogs about CIM and hardware monitoring