WilliamLam.com

Exploring the vSphere Flash Read Cache (vFRC) APIs Part 1

by // 2 Comments

I am sure by now you have all probably heard about some of the new storage features introduced in vSphere 5.5 such as Virtual SAN (VSAN) and vSphere Flash Read Cache. In the coming weeks, I will be working on a series of articles that will be looking at these new features from an Automation perspective and demonstrate how you can automate and manage these configurations using the vSphere APIs. This of course will then allow you to automate using your favorite scripting/programming language such as PowerShell, Perl, Java, .NET, Ruby, etc.

In this first article, I will be exploring the new vSphere Flash Read Cache APIs and I will be primarily focusing on consuming vSphere Flash Read Cache for your virtual machines and not on the actual ESXi host configurations. In a subsequent articles, I will look at the necessary APIs to configure vSphere Flash Read Cache for your ESXi 5.5 host. For those of you who are not familiar with this feature I would recommend you check out Duncan Epping's introduction article to vSphere Flash Read Cache also known as vFRC as well as the new What's News vSphere Flash Read Cache whitepaper by Rawlinson Rivera.

To enable vFRC for a particular virtual machine, you will need to use the virtual machine ReconfigVM_Task() method and specify the particular virtual disk you wish to enable vFRC on. There is new property in the vSphere 5.5 API called vFlashCacheConfigInfo under the VirtualDisk object which you will need to configure.

These are the 5 properties:

  • reservationInMB
  • blockSizeInKB
  • cacheConsistencyType
  • cacheMode
  • vFlashModule

Technically speaking, you only need to specify the reservationInMB property as the rest of the properties have system defaults. However, at a minimum you will probably want to configure reservationInMB and blockSizeInKB where the valid values are 4-1024 and the system default is 4KB.

For cacheConsistencyType, even though the vSphere API mentions both "strong" and "weak" type, only "strong" is supported/configurable and this means that the cache data will be consistent upon a crash. If you try to configure it to "weak", you will get a not supported error.

For cacheMode, even though the vSphere API mentions both "write_thru" and "write_back", only "write_thru" is supported/configurable and this means that when writes are written to the cache, they are then de-staged to the underlying storage sub-system. If you try to configure it to "write_back", you will get a not supported error.

The last property vFlashModulespecifies the specific vFRC module to be used and at this current time, only "vfc" is valid and this is also a system default and does not need to be specified.

To demonstrate these new vFRC VM APIs, I created a sample vSphere SDK for Perl script called vflashVMMgmt.pl which can run against a vCenter Server or a standalone ESXi 5.5 host.


The script supports three operations: query, enable and disable.

To enable vFRC for a particular VM, you will need to use the "enable" operation and specify two required options (--disk and --reservation) and --blocksize is optional with default being 4KB. Here is an example configuring vFRC with 8KB blocksize & 1GB reservation:

./vflashVMMgmt.pl --config .vcenter55-1 --vmname TestVM --disk "Hard disk 1" --operation enable --blocksize 8 --reservation 1024

You can query whether a VM has vFRC enabled by using the "query" operation and specify --disk option for a particular VMDK. Here is an example:

./vflashVMMgmt.pl --config .vcenter55-1 --vmname TestVM --disk "Hard disk 1" --operation query

To disable vFRC for a particular VM, you can use the "disable" operation which disables vFRC by setting the reservationInMB property to 0. Here is an example:

./vflashVMMgmt.pl --config .vcenter55-1 --vmname TestVM --disk "Hard disk 1" --operation disable

Hopefully this has given you a good overview of the new vSphere Flash Read Cache APIs from a virtual machine perspective and the necessary information to enable or disable this feature. In the next part of the series, I will take a look at the new vSphere APIs that are required to setup and configure vFRC for your ESXi host, so stay tuned!

How to quickly setup and test VMware VSAN (Virtual SAN) using Nested ESXi

by // 48 Comments

Last week at VMworld 2013, VMware announced the release of vSphere 5.5 which includes a variety of exciting new features.  One of the most anticipated feature introduced in this release is VMware Virtual SAN (VSAN) which will be available initially as a public beta. One question that I heard repeatedly throughout the VMworld conference was whether it would be possible to test VSAN in a nested ESXi environment? The answer is absolutely! This is a great way to learn about VSAN and how works from a functional perspective before procuring the necessary hardware.

Disclaimer: Running VSAN in a nested ESXi environment is not officially supported nor is it a replacement for actual testing on actual physical hardware.

Before getting started, I would highly recommend you check out the following resources from my good friend Cormac Hogan which includes a detailed VSAN walk through as well what looks to be an awesome series of articles on how VSAN works:

Requirements:

  • Environment running either vSphere 5.1 or 5.5 and access to the vSphere Web Client.

Configuration:

Nested ESXi VM configured with the minimal resources:

  • 2 vCPU
  • 5GB Memory (ESXi 5.5 now requires a minimum of 4GB vs 2GB as with previous releases but VSAN requires minimum of 5 with recommended 6)
  • 2GB Disk for ESXi 5.5 installation
  • 4GB Disk for an "Emulated" SSD
  • 8GB Disk for HDD

Easy Method:

Instead of having you go through the process of building a Nested ESXi VM with all the prerequisites that includes steps from here and here. I have pre-built a VSAN Nested ESXi VM template (217Kb) that you can just download and import into your environment and being the installation process.

Download either:

and connect to your vCenter Server 5.1 or 5.5 using the vSphere Web Client and import the OVF into your environment (do not use the vSphere C# Client as the import does not persist VHV configuration). Once you have imported the VM, you can then mount the ESXi 5.5 ISO and begin the installation. All three VMDKs have been thin provisioned and you can change the capacity during deployment.

Slightly Harder Method:

If you wish to build the Nested ESXi VM yourself, then you can follow these instructions:

Step 1 - Create a new VM and when you get to the compatibility screen, select either "ESXi 5.1 or greater" or "ESXi 5.5 or greater" depending on the version of vSphere you are running

Step 2 - For the GuestOS select "Other" and "Other (64-bit)"

Step 3 - We will need to customize the following virtual hardware configuration:

  • Change vCPU to 2
  • Click on CPU drop down and enable "Expose hardware assisted virtualization to the guest OS"
  • Change Memory to 4GB
  • Change the initial VMDK to 2GB or whatever value you wish to use for ESXi installation
  • Add second VMDK with 4GB or whatever value you wish to use for "emulated" SSD
  • Add third VMDK with 8GB of whatever value you wish to use for the HDD
  • Click on the VM Options tab at the top and select the "Advanced" drop down box. We will need to add the following entry scsi0:1.virtualSSD = 1 For more details please refer to this article

Step 4 - Click okay to provision the VM and once it has been deployed you will need to re-configure the guestOS to "VMware ESXi 5.x" using the vSphere C# Client for vSphere 5.1 or vSphere Web Client for vSphere 5.5. At this point, you will have the same VM image as in the Easy Method and you are now ready to install ESXi 5.5

When you install ESXi 5.5, you should see the following three disks as shown in the screenshot below, ensure you install ESXi on the 2GB disk:

Prior to enabling VSAN on the particular vSphere Cluster, make sure you enable the new VSAN traffic type on one of your VMkernel interfaces for each of your ESXi hosts, this is required for VSAN communication.

If all the prerequisites have been met, you can now easily enable VSAN by simply checking the VSAN box when editing the vSphere Cluster. In just a few minutes you should see diskgroups automatically created (assuming you selected Automatic mode) consuming both the emulated SSD and HDD and the creation of the vsanDatastore which will be available on all ESXi hosts within that vSphere Cluster.

You can also use the same method for emulating an SSD running in a Nested ESXi to functional test the new VMware Flash Read Cache (vFRC) feature.