WilliamLam.com

Exploring the vSphere Flash Read Cache (vFRC) APIs Part 3

by // Leave a Comment

To conclude our 3-part blog series in exploring the vSphere Flash Read Cache (vFRC) APIs, in this last article we will take a look at the vSphere APIs required to live migrate a running Virtual Machine which has been configured with vSphere Flash Read Cache. If you perform this migration using the vSphere Web Client, you will see that you are now presented with a new option in the migration wizard on whether or not to migrate the vFRC cache.

If you choose to migrate the vFRC cache, you have the option to migrate all of the Virtual Machine's vFRC cache or you can specify this on a per VMDK basis by selecting the "Advanced" option. By choosing to migrate the vFRC cache, the migration will of course take longer as it needs to not only transfer the running Virtual Machine, but also the vFRC cache.

To automate this operation from an API perspective, you will need to use the Virtual Machine RelocateVM_Task method. As part of creating the migration spec, there is a new property called migrateCache which can now be specified on each VMDK.

To demonstrate the use of this vSphere API, I have created a vSphere SDK for Perl sample script called migratevFRCVM.pl which accepts the name of the Virtual Machine to migrate, the name of the ESXi host to migrate to and an optional parameter on whether or not to migrate the vFRC cache. By default, the script will migrate the vFRC cache if no option is selected.

Here is an example of migrating a VM along with its configured vFRC cache:

./migratevFRCVM.pl --config .vcenter55-1 --vmname Test-VM --dst_vihost vesxi55-10.primp-industries.com --migrate_cache true

Note: I have simplified the script to apply the --migrate_cache to ALL VMDKs, but as mentioned earlier you can specify this on a per-VMDK basis so you can migrate the vFRC cache for specific virtual disks if you choose to.

Hopefully you have enjoyed this series and now have a better understanding of the vSphere Flash Read Cache APIs. Be sure to check out the other two articles if you have not already.

Exploring the vSphere Flash Read Cache (vFRC) APIs Part 2

by // Leave a Comment

Continuing from Part 1 of Exploring the vSphere Flash Read Cache (vFRC) APIs, we will now explore the necessary vSphere APIs to setup and configure vFRC on your ESXi host. There are two workflows in which you can create your Virtual Flash Resource, the first is simply adding all valid SSDs as you would when using the vSphere Web Client which automatically creates a VFFS (Virtual Flash File System) which is used to manage the underlying SSD devices. The second workflow is to start with a single SSD and to manually create the VFFS volume which then allows you to extend the VFFS by adding additional SSD devices. We will be going over both set of workflows and the necessary vSphere APIs required to perform these operations.

To automate the configuration of vFRC on your ESXi hosts, you will need to access both vFlashManager and storageSystem along with the following vSphere API methods:

  • QueryAvailableSsds
  • ConfigureVFlashResourceEx_Task
  • DestroyVffs
  • FormatVffs
  • HostConfigureVFlashResource
  • ExtendVffs

To demonstrate the functionality of these vSphere APIs, I have created a vSphere SDK for Perl sample script called vflashHostMgmt.pl and it supports the following operations: query, listssd, add, format, extend and destroy

Workflow 1 - Add all valid SSD devices

To configure a Virtual Flash Resource for your ESXi host, you will need to use the vSphere Web Client and click on the "Add Capacity" button and select all valid SSD devices for that particular ESXi host as seen in the screenshot below.

To automate the same workflow, we first need to be able to identify the list of available SSD devices that could be used for either vFRC or even VSAN. There is a nice vSphere API method under the storageSystem called QueryAvailableSsds which has been implemented in the script as the "listssd" operation.

Here is an example execution of the "listssd" operation:

./vflashHostMgmt.pl --config .vcenter55-1 --vihost vesxi55-4.primp-industries.com --operation listssd

As you can see from the output we have three available SSD devices matching the vSphere Web Client output. To add these SSD devices and create your Virtual Flash Resource, you will need to use the "add" operation within the script that accepts a comma seperated list of the SSD device paths as shown in the above output. Next we need to call the vFlashManager's ConfigureVFlashResourceEx_Task method which thnan accepts an array of SSD device paths to automatically configure and add the Virtual Flash Resource.

Here is an example execution of the "add" operation:

./vflashHostMgmt.pl --config .vcenter55-1 --vihost vesxi55-4.primp-industries.com --operation add --disk /vmfs/devices/disks/naa.6000c297de55bcf0471f311abc865449,/vmfs/devices/disks/naa.6000c2992cfbf14a2d827303c48632fa,/vmfs/devices/disks/naa.6000c2989357b5d31eb20256e39f9338

We can confirm that our Virtual Flash Resource was successfully created by running the "query" operation.

Here is an example execution of the "query" operation:

./vflashHostMgmt.pl --config .vcenter55-1 --vihost vesxi55-4.primp-industries.com --operation query

From the output we can see a VFFS was automatically created for us including its name and UUID and it contains the three SSD devices we added in earlier. We can also confirm by logging into our vSphere Web Client and we should see the same output as well.

In preparation for the next workflow, we can easily destroy our VFFS which is the same operation within the vSphere Web Client by selecting the "Remove All" button. To do so, we need to use the storageSystem's DestroyVffs method. In the script, this has been implemented as the "destroy" operation.

Here is an example execution of the "destroy" operation:

As you can see workflow 1 is pretty straight forward if you have an ESXi host that contains all the SSD devices you wish to add to your Virtual Flash Resource. In workflow 2, we will take a look at starting with a single SSD and manually creating the VFFS which can then be extended OR if you have an existing Virtual Flash Resource and would like to extend it, the set of APis shown in workflow 2 will aide in that use case.

Workflow 2 - Create VFFS using single SSD device / Extend VFFS

When going through workflow 1, the VFFS volume is automatically created for the user and is not something on would need to think about unless you would like to extend an existing VFFS. In this workflow we start out by adding a single SSD device which will require the creation of VFFS volume and then we will then extend that VFFS with additional SSD devices so we end up in the same end state as workflow 1.

To create a VFFS, you will need to use the FormatVffs API method which accepts a single SSD device and VFFS label and then using the HostConfigureVFlashResource API method to mount the VFFS volume to the ESXi host. This has been implemented as the "format" operation which is similar to the "add" operation but require an additional --vffs parameter which denotes the VFFS volume label.

Here is an example execution of the "format" operation:

./vflashHostMgmt.pl --config .vcenter55-1 --vihost vesxi55-4.primp-industries.com --operation format --vffs vghetto-vffs --disk /vmfs/devices/disks/naa.6000c297de55bcf0471f311abc865449

As part of the result, it will return the VFFS UUID which is required when extending a VFFS. You can also get this information by using the "query" operation which we can also see the label that we have assigned our VFFS.

To add additional SSD devices to our existing VFFS using either workflow 1 or 2, you will need to use the ExtendVffs API method which requires the VFFS UUID and the SSD device you wish to add. This has been implemented as the "extend" operation within the script.

Here is an example execution of the "extend" operation:

./vflashHostMgmt.pl --config .vcenter55-1 --vihost vesxi55-4.primp-industries.com --operation extend --vffs_uuid 527fc6e6-249cdb69-d502-005056adfa73 --disk /vmfs/devices/disks/naa.6000c2992cfbf14a2d827303c48632fa

We can confirm our changes by using the "query" operation as well as looking at our Virtual Flash Resource using the vSphere Web Client. We should see the two SSD devices that we have added to our VFFS.

 

In Part 3 of exploring the vSphere Flash Read Cache (vFRC) APIs, we will take a look at migrating a virtual machine which has vFRC configured and the options we have in terms of either migrating or dropping the vFRC cache.

Exploring the vSphere Flash Read Cache (vFRC) APIs Part 1

by // 2 Comments

I am sure by now you have all probably heard about some of the new storage features introduced in vSphere 5.5 such as Virtual SAN (VSAN) and vSphere Flash Read Cache. In the coming weeks, I will be working on a series of articles that will be looking at these new features from an Automation perspective and demonstrate how you can automate and manage these configurations using the vSphere APIs. This of course will then allow you to automate using your favorite scripting/programming language such as PowerShell, Perl, Java, .NET, Ruby, etc.

In this first article, I will be exploring the new vSphere Flash Read Cache APIs and I will be primarily focusing on consuming vSphere Flash Read Cache for your virtual machines and not on the actual ESXi host configurations. In a subsequent articles, I will look at the necessary APIs to configure vSphere Flash Read Cache for your ESXi 5.5 host. For those of you who are not familiar with this feature I would recommend you check out Duncan Epping's introduction article to vSphere Flash Read Cache also known as vFRC as well as the new What's News vSphere Flash Read Cache whitepaper by Rawlinson Rivera.

To enable vFRC for a particular virtual machine, you will need to use the virtual machine ReconfigVM_Task() method and specify the particular virtual disk you wish to enable vFRC on. There is new property in the vSphere 5.5 API called vFlashCacheConfigInfo under the VirtualDisk object which you will need to configure.

These are the 5 properties:

  • reservationInMB
  • blockSizeInKB
  • cacheConsistencyType
  • cacheMode
  • vFlashModule

Technically speaking, you only need to specify the reservationInMB property as the rest of the properties have system defaults. However, at a minimum you will probably want to configure reservationInMB and blockSizeInKB where the valid values are 4-1024 and the system default is 4KB.

For cacheConsistencyType, even though the vSphere API mentions both "strong" and "weak" type, only "strong" is supported/configurable and this means that the cache data will be consistent upon a crash. If you try to configure it to "weak", you will get a not supported error.

For cacheMode, even though the vSphere API mentions both "write_thru" and "write_back", only "write_thru" is supported/configurable and this means that when writes are written to the cache, they are then de-staged to the underlying storage sub-system. If you try to configure it to "write_back", you will get a not supported error.

The last property vFlashModulespecifies the specific vFRC module to be used and at this current time, only "vfc" is valid and this is also a system default and does not need to be specified.

To demonstrate these new vFRC VM APIs, I created a sample vSphere SDK for Perl script called vflashVMMgmt.pl which can run against a vCenter Server or a standalone ESXi 5.5 host.


The script supports three operations: query, enable and disable.

To enable vFRC for a particular VM, you will need to use the "enable" operation and specify two required options (--disk and --reservation) and --blocksize is optional with default being 4KB. Here is an example configuring vFRC with 8KB blocksize & 1GB reservation:

./vflashVMMgmt.pl --config .vcenter55-1 --vmname TestVM --disk "Hard disk 1" --operation enable --blocksize 8 --reservation 1024

You can query whether a VM has vFRC enabled by using the "query" operation and specify --disk option for a particular VMDK. Here is an example:

./vflashVMMgmt.pl --config .vcenter55-1 --vmname TestVM --disk "Hard disk 1" --operation query

To disable vFRC for a particular VM, you can use the "disable" operation which disables vFRC by setting the reservationInMB property to 0. Here is an example:

./vflashVMMgmt.pl --config .vcenter55-1 --vmname TestVM --disk "Hard disk 1" --operation disable

Hopefully this has given you a good overview of the new vSphere Flash Read Cache APIs from a virtual machine perspective and the necessary information to enable or disable this feature. In the next part of the series, I will take a look at the new vSphere APIs that are required to setup and configure vFRC for your ESXi host, so stay tuned!