WilliamLam.com

Emulating an SSD Virtual Disk in a VMware Environment

by // 32 Comments

I continue to be amazed everyday at all the awesome features and challenges being tackled by our VMware Engineering organization and yesterday was another example of that. There was a question that was posed internally about emulating an SSD device for a Nested ESXi environment running in VMware Fusion. I figure this would be an easy answer and pointed the user to a blog article I had written a few years ago on how to fake an SSD device in ESXi using SATP claim rules via ESXCLI. It turns out, one of the engineers knew of a better way of emulating an SSD Virtual Disk that can be consumed beyond just Nested ESXi VMs but also for any other guestOSes that supports SSD devices.

So why would you want to emulate an SSD device? Well for a vSphere environment, you may want to try out the new Swap to Host Cache feature from a functional perspective to see how it would work. You might be developing a script to enable this feature and having a "fake" SSD device would allow you to create such a script and test it. For other guestOSes, maybe you want to see how the system would react to an SSD device, perhaps drivers or configurations maybe needed and you would like to run through those processes before installing a real SSD device.

So the solution is actually quite simple and it is just an advanced setting in the Virtual Machine's configuration file (VMX) which can also be appended to using either the vSphere Web Client, vSphere C# Client or the vSphere API. This setting is only supported on Virtual Machines that is running virtual hardware 8 or greater. To configure a specific virtual disk to appear as an SSD, you just need to add the following:

scsiX:Y.virtualSSD = 1

where X is the controller ID and the Y is the disk ID of the Virtual Disk.

This configuration presents to the guestOS the mediumRotationRate field of the SCSI inquiry pages 0xB1 and sets the value to 1 and the guests will then report it as a solid-state device. As you can see, this can benefit more than just running Nested ESXi, you can also do various testing on other guestOSes that you require a "fake" SSD device.

Note: Though you can emulate an SSD device, it is no substitute for an actual SSD device and any development or performance tests done in a simulated environment should also be vetted n a real SSD device, especially when it comes to performance.

It is also important to note that reporting of an SSD device will highly depend on the guestOS, here is a high level table on how some of the common guestOSes recognize SSD devices.

GuestOS SSD Reporting
Windows 8 IDE, SCSI and SATA disks can be recognized as SSDs
Windows 7 IDE and SATA disks can be recognized SSD, but SCSI as mechanical
Linux (Ubuntu & RHEL) IDE, SCSI and SATA disks can be recognized as SSDs
Mac OS X SATA can be recognized as SSDs, but IDE and SCSI as mechnical

Here is a screenshot of a Nested ESXi host with an emulated SSD device:

Here is a screenshot of the new Windows 8.1 Preview with an emulated SSD device:

Note: Though I demonstrated this using vSphere, this also works for VMware Fusion (tested this personally), Workstation and Player. The only requirement is that you are running virtual hardware 8 or greater and that your guestOS supports reporting SSD device.

From a Nested ESXi perspective, I will definitely be using this method instead of using ESXCLI to go through the SATP claim rules, this is much easier to remember. I would also like to thank Regis Duchesne for sharing this tip and Srinivas Singavarapu and the virtual devices team for developing this awesome feature. You guys ROCK!

How to Create an SE Sparse (Space-Efficient) Disk in vSphere 5.1

by // 8 Comments

You probably may have heard, that with the upcoming release of vSphere 5.1, a new virtual machine disk format will be introduced called called SE Sparse (Space-Efficient). One of it's features is to provide the ability to reclaim unused blocks from within the guestOS. I would highly recommend you check out a recent blog post vSphere 5.1 Storage Enhancements – Part 2: SE Sparse Disks by Cormac Hogan for more details about the new SE Sparse disk format as well as other storage improvements in vSphere 5.1.

As Cormac points out, this new disk format will initially be leveraged by VMware View (in a future release from my understanding), as there are additional integrations required to use this feature than just using the new SE Sparse disk format. Having said that, the SE Sparse disk format is a feature of the vSphere 5.1 platform and with that, you do have the ability to create an SE Sparse disk.

Disclaimer: This is for educational purposes only, this is not officially supported by VMware. Please test this in a development environment before using it on actual systems.

There are two methods in which you can create an SE Sparse disk, directly on the ESXi Shell of an ESXi 5.1 host or remotely connecting to an ESXi 5.1 host.

Option 1 - Using vmkfstools on ESXi Shell 

Though it may not be documented, you can easily create a new VMDK with the new SE Sparse disk format by running the following command (10GB disk in this example):

vmkfstools -c 10g -d sesparse WindowsXP.vmdk

Here is a screenshot of new SE Sparse disk descriptor file to prove we have successfully created a new VMDK using the new format:

Option 2 - Using vSphere 5.1 API w/modified remote version of vmkfstools

As mentioned, the SE Sparse disk format is a feature of the vSphere 5.1 platform and as so, you can also leverage the vSphere 5.1 API to create a new VMDK using the virtualDiskManager and specifying the new SeSparseVirtualDiskSpec.

Note: Even though the vSphere API reference mentions the ability to set grain size via grainSizeKb property, I have found that it is not possible and just leaving it blank will automatically default to 1024K (1MB) which might be a system default for now.

You can download the modified version of the remote vmkfstools called vmkfstools-lamw which requires the the installation of vCLI 5.1 or vMA 5.1.

Here is an example of creating the same 10GB VMDK using the new SE Sparse disk format:

./vmkfstools-lamw --server 172.30.0.187 --username root -c 10G -d sesparse "[datastore1] WindowsXP.vmdk"

After you have created your new SE Sparse disk, the next logical step is assign it to a virtual machine. Since this is a new feature in vSphere 5.1, you will need to use the new vSphere Web Client to perform the operation as the legacy vSphere C# Client is not aware of this new disk type. You will also need to ensure that the virtual machine is running the latest ESXi 5.1 compatibility and later (virtual hardware version 9).

Once you have added our newly created disk from the datastore, it should now show up in the vSphere Web Client as Flex-SE for the disk type.

Additional Resources:

 

How to Query VM Disk Format in vSphere 5

by // 5 Comments

Prior to vSphere 5, it was not trivial to identify the particular disk format for a given virtual machine's disk. Using the vSphere Client, you would see a virtual machine's disk be displayed as either thin or thick. The problem with this is that the "thick" format can be either:

  • zeroedthick - A thick disk has all space allocated at creation time and the space is zeroed on demand as the space is used
  • eagerzeroedthick - An eager zeroed thick disk has all space allocated and wiped clean of any previous contents on the physical media at creation time. Such disks may take longer time during creation compared to other disk formats.

Users would not be able to distinguish the exact type using the vSphere Client or the vSphere 4 APIs. With the release of vSphere 4, VMware did introduce a new property in the vSphere 4 API called eagerlyScrub which was supposed to help identify whether a virtual disk was allocated as an eagerzeroedthick disk. Unfortunately there may have been a bug with the property as it never gets modified whether a disk is created as zeroedthick or eagerzeroedthick.

The only method that I was aware of to truly figuring out the disk format would be to manually parse the virtual machine's vmware.log file to identify the disk type which I wrote a script for in 2009.

During the vSphere 5 beta, I had noticed the vSphere Client UI now properly displays all three virtual machine disk format: zeroedthick (displayed as flat), thin and eagerzeroedthick (displayed as thick).

Seeing that VMware now displays the three different formats, I wanted to see if it was possible to extract this using the vSphere 5 APIs and not have to rely on the hack of reading the vmware.log files. It turns out that the eagerlyScrub property is now functioning properly when a VMDK is provisioned or has been inflated/converted to the eagerzeroedthick format. I wrote a simple vSphere SDK for Perl script called getVMDiskFormat.pl which allows you to extract the disk formats of all virtual machines connecting to either vCenter or directly to an ESX(i) host.

The script allows for two types of output: console (directly on the console) or csv (creates .csv file)

If you select csv output, by default it will be stored in a file called "vmDiskFormat.csv". You also have the option of specifying the filename by using the --filename flag and providing a name of your choosing.

You can then load the csv file into excel and easily sort through the various disk format types.

All this is already included in the latest version of the VMware vSphere Health Check Report 5.0 if you want a centralize report that includes virtual machine disk format.