WilliamLam.com

AHCI (vmw_ahci) performance issue resolved in ESXi 6.5 Update 1

by // 44 Comments

For customers who had SATA controllers that consumed the VMware Advanced Host Controller Interface (AHCI) driver found that after upgrading to ESXi 6.5, the disk performance for those devices were significantly impacted. Basic operations such as cloning or uploading an OVF/OVA would literally double if not triple in time. In fact, I too had observed this same behavior when I had upgraded my Intel NUC (not an officially supported platform) to ESXi 6.5. One thing I had noticed at the time when others were reporting simliar issues was that their HW platforms were also not on the VMware HCL, so I was not sure if this was limited to only home-lab environments?

In any case, I and others eventually stumbled onto this blog article by Sebastian Foss who I believe may have been the first to identify a workaround which was to simply disable the new AHCI Native Driver which loads by default and forcing it fall back to using the legacy AHCI driver which made the issue go away after a reboot. Although the folks who had reported seeing simliar issue were all using hardware platforms that were not officially on the VMware HCL, I decided to still file an internal bug and hoped someone could take a look to see what was going on.

With the release of ESXi 6.5 Update 1, I am happy to report the observed performance issues with the Native AHCI driver have now been resolved! I have been running on earlier release of ESXi 6.5 Update 1 build for couple of weeks now and have not seen any of the problems I had before. For those interested, the official fix went is in version 1.0.0-37vmw or greater of the vmw_ahci driver.

You can easily verify for this by running the following ESXCLI command to retrieve the version of your vmw_ahci driver:


If you had disabled the Native AHCI driver, you will definitely want to re-enable it. You can check if its been disabled by running the following ESXCLI command and checking the second column to see if it shows "false":

esxcli system module list | grep vmw_ahci

If the Native AHCI driver is disabled as shown in the previous command, then you can re-enable it by running the following ESXCLI command:

esxcli system module set --enabled=true --module=vmw_ahci

Once you have re-enabled the driver, you will need to reboot for the changes to go into effect.

New Nested ESXi 6.x Content Library 

by // 16 Comments

A few years back I had showed how you could create and host your own 3rd Party vSphere Content Library which allows customers to decouple their content from the underlying vSphere environment and centralizing their content and making it available to number of vCenter Servers by simply just having an HTTP(s) endpoint. The other huge benefit is being able to take advantage of the existing web content tools for optimizing delivery or retrieval whether that is replication, caching, etc. and not relying a single vCenter Server for providing Content Library publication. In addition to showing how to create your own content libraries, I also had built my own 3rd Party vSphere Content Library which contains a variety of my Nested ESXi Templates (empty VM shells) running on Amazon S3 which can be consumed by anyone as long as you are running vCenter Server 6.0 or newer.

Although the empty Nested ESXi Templates were quite useful for myself and customers, it would have also been nice to include my pre-built Nested ESXi Virtual Appliances which I had recently updated to support vSphere 6.0 Update 3 and vSphere 6.5d (vSAN 6.6). Thanks to Dana Nourie, who runs our wildly popular VMware Flings Program, was kind enough to help me with the content hosting and now anyone can also subscribe to my Nested ESXi VA's and automatically have the content sync down using the vSphere Content Library feature.

UPDATE 1 (07/31/17) - The Nested ESXi Content Library has been updated to include the latest ESXi 6.5 Update 1 VA. If you are already subscribing to the library, it should have already pulled down the content (or at least the metadata which you can then force synchronization) or you can simply subscribe to the library and have access to the latest ESXi VA.

UPDATE 2 (05/07/18) - The Nested ESXi Content Library has been updated to include the latest ESXi 6.5 Update 2 VA. If you are already subscribing to the library, it should have already pulled down the content (or at least the metadata which you can then force synchronization) or you can simply subscribe to the library and have access to the latest ESXi VA.

To get started, just create a new vSphere Content Library and enter the following subscription URL: https://download3.vmware.com/software/vmw-tools/lib.json 


You can either download the content immediately or only when you need to use it. I recommend the former since its only two images which totals up to a whopping 1GB 😉

Once the creation of the Content Library has been completed, you should see the following two Nested ESXi VAs in the library which are now ready for deployment!


For more information about the Nested ESXi 6.0u3/6.5d VA's and how they work, please have a look at this blog post here. For more information about the Nested ESXi Templates and how to subscribe to the 3rd Party vSphere Content Library, please have a look at this blog post here.

Managing ESXi Embedded Host Client settings

by // Leave a Comment

There was a question the other day about managing ESXi Embedded Host Client (EHC) settings which you can find by click on the logged-in username and navigating to the "Settings" section as shown in the screenshot below. Customers can manage things like the default VM Console used whether that is the HTML5 VMRC or the Standalone VMRC to auto-refresh and even sharing usage information to help improve the product back to VMware.


In addition to configuring the EHC settings within UI, you can also manage them via automation using the vSphere API and any one of your favorite vSphere SDK/CLIs. The EHC settings are exposed as a set of ESXi Advanced Settings as shown in the screenshot below. These settings are applied on a per-ESXi basis and NOT on a per-user basis.


Below is a table that summarizes the 7 different EHC settings which you can programmatically query to retrieve their current value, default value and whether they have been overridden simliar to what the UI provides today.

Key Description Default Value
UserVars.HostClientCEIPOptIn  Whether or not to opt-in for CEIP in Host Client: 0 for ask 1 for yes 2 for no 0
UserVars.HostClientDefaultConsole  Default console type in Host Client webmks
UserVars.HostClientEnableMOTDNotification  Whether or not to enable MOTD notification on login for Host Client 1
UserVars.HostClientEnableVisualEffects  Whether or not to enable visual effects for Host Client 1
UserVars.HostClientSessionTimeout  Default timeout for Host Client sessions in seconds 900
UserVars.HostClientShowOnlyRecentObjects  Whether or not to show only recent objects in Host Client 1
UserVars.HostClientWelcomeMessage  Welcome message displayed on login in Host Client  Welcome to {{hostname}}

Note: The language settings is based on your browser settings which you can overridden, but it looks like we may not have exposed that as a configurable setting via automation.