WilliamLam.com

A Hidden vSphere 5.1 Gem - Forwarding Virtual Machine Logs (vmware.log) to Syslog Part 2

by // 7 Comments

In Part 1 I showed how you can forward virtual machine logs to ESXi syslog using an advanced virtual machine setting that was introduced in vSphere 5.1. A caveat with this solution is that the ESXi syslog file contains both system logs as well as virtual machine logs which is not very ideal from an isolation perspective. With virtual machine logs being quite verbose, if you are not forwarding logs to a remote syslog server, important system events can easily be rotated out of the local logs.

To work around this caveat, we can create a new logger specifically for handling virtual machine logs within the ESXi syslog client. You can view the existing logger types by looking in /etc/vmsyslog.conf.d directory. You will need to create a new logger configuration file which I named vmx.conf and it should contain the following:

[vmsyslog-logger]
# unique id for this logger
id = vmx
# description of this logger
descr = VMX Logs
# idents this logger is interested in
idents = vmx
# output file (e.g. foo == /var/log/foo.log)
file = vmx
# file logger class
fclass = FileLoggerSyslog
# network logger class
nclass = NetworkFilterSyslogTimestamp

Here is a screenshot of of my configuration file and noticed the highlighted text in yellow is what needs to be modified:

Note: Ensure that idents property matches the vmx.log.syslogID string specified for your virtual machines. This also means you will not be able to specify the virtual machine's name for the advanced setting, but will need to keep it generic so it can be filtered by the logger.

Once you have saved the vmx.conf configuration file, you will need to reload the ESXi syslog client for the changes to go into effect by running the following ESXCLI command:

esxcli system syslog reload

You now should see a new log file in /var/log called vmx.log which will contains only virtual machine logs:

If your ESXi host is forwarding its logs to vCenter Log Insight, you can easily create a filter for the keyword "vmx" in the log source or whatever string you decided to set it to if you are not using the default.

One final caveat to be aware of now is that the custom syslog logger (vmx.conf) will not persist after a system reboot. To preserve this file, you can either automatically re-create the file during bootup and reload syslog client using this article here OR create a custom VIB using this article here.

Emulating an SSD Virtual Disk in a VMware Environment

by // 32 Comments

I continue to be amazed everyday at all the awesome features and challenges being tackled by our VMware Engineering organization and yesterday was another example of that. There was a question that was posed internally about emulating an SSD device for a Nested ESXi environment running in VMware Fusion. I figure this would be an easy answer and pointed the user to a blog article I had written a few years ago on how to fake an SSD device in ESXi using SATP claim rules via ESXCLI. It turns out, one of the engineers knew of a better way of emulating an SSD Virtual Disk that can be consumed beyond just Nested ESXi VMs but also for any other guestOSes that supports SSD devices.

So why would you want to emulate an SSD device? Well for a vSphere environment, you may want to try out the new Swap to Host Cache feature from a functional perspective to see how it would work. You might be developing a script to enable this feature and having a "fake" SSD device would allow you to create such a script and test it. For other guestOSes, maybe you want to see how the system would react to an SSD device, perhaps drivers or configurations maybe needed and you would like to run through those processes before installing a real SSD device.

So the solution is actually quite simple and it is just an advanced setting in the Virtual Machine's configuration file (VMX) which can also be appended to using either the vSphere Web Client, vSphere C# Client or the vSphere API. This setting is only supported on Virtual Machines that is running virtual hardware 8 or greater. To configure a specific virtual disk to appear as an SSD, you just need to add the following:

scsiX:Y.virtualSSD = 1

where X is the controller ID and the Y is the disk ID of the Virtual Disk.

This configuration presents to the guestOS the mediumRotationRate field of the SCSI inquiry pages 0xB1 and sets the value to 1 and the guests will then report it as a solid-state device. As you can see, this can benefit more than just running Nested ESXi, you can also do various testing on other guestOSes that you require a "fake" SSD device.

Note: Though you can emulate an SSD device, it is no substitute for an actual SSD device and any development or performance tests done in a simulated environment should also be vetted n a real SSD device, especially when it comes to performance.

It is also important to note that reporting of an SSD device will highly depend on the guestOS, here is a high level table on how some of the common guestOSes recognize SSD devices.

GuestOS SSD Reporting
Windows 8 IDE, SCSI and SATA disks can be recognized as SSDs
Windows 7 IDE and SATA disks can be recognized SSD, but SCSI as mechanical
Linux (Ubuntu & RHEL) IDE, SCSI and SATA disks can be recognized as SSDs
Mac OS X SATA can be recognized as SSDs, but IDE and SCSI as mechnical

Here is a screenshot of a Nested ESXi host with an emulated SSD device:

Here is a screenshot of the new Windows 8.1 Preview with an emulated SSD device:

Note: Though I demonstrated this using vSphere, this also works for VMware Fusion (tested this personally), Workstation and Player. The only requirement is that you are running virtual hardware 8 or greater and that your guestOS supports reporting SSD device.

From a Nested ESXi perspective, I will definitely be using this method instead of using ESXCLI to go through the SATP claim rules, this is much easier to remember. I would also like to thank Regis Duchesne for sharing this tip and Srinivas Singavarapu and the virtual devices team for developing this awesome feature. You guys ROCK!

How to Run VMware's New Fling VisualEsxtop on Mac OS X

by // 12 Comments

The folks over at VMware Labs continue to pump out awesome new Flings and just yesterday, they released another one called VistualEsxtop, which is an enhanced version of esxtop and resxtop. It is no surprise, the Fling was developed by VMware Engineers working in the Performance group such as Priya Sethuraman, Zhelong Pan, Haiping Yang and Joanna Guan, some of which who helped develop the original esxtop utility.

VisualEsxtop can connect to directly to an ESX(i) host or a vCenter Server and provides support going all the way back to ESXi 3.5 or vCenter Server 4.0. Here is a summary of all the features for VisualEsxtop:

Features

  1. Live connection to ESX host or vCenter Server
  2. Flexible way of batch output
  3. Load batch output and replay them
  4. Multiple windows to display different data at the same time
  5. Line chart for selected performance counters
  6. Flexible counter selection and filtering
  7. Embedded tooltip for counter description
  8. Color coding for important counters

While reading the instructions, I noticed VisualEsxtop is supported on both Windows and Linux and is loaded by a simple .bat or .sh script. So I wondered if it could run on Mac OS X? Well, it turns out the script uses a utility called readlink which does not operate the same on Mac OS X and will thrown an error. However, since VisualEsxtop is just a Java application, you can manually load the VisualEsxtop and the necessary library files.

To do so, you just need to change into the visaulEsxtop directory and run the following command:

java -cp lib/vtop-ui.jar com.vmware.vtop.VTopMain

Note: For ease of use, you can even create a shell script which executes the command so you do not have to type it out each time.

You should now see visualEsxtop launch after executing the above command:

One really cool feature of VisualEsxtop is the use of color coating for important counters and issues. In the screenshot below, you can see I have a VM that is dropping packets and it is automatically highlighted for me.

I would highly recommend you check out VisualEsxtop and add this to your toolbelt of tools for troubleshooting and diagnosing performance issues in a vSphere environment! If you have any feedback or questions for the engineers, please leave them in the comment section of the VisualEsxtop Fling.