WilliamLam.com

Homelab considerations for vSphere 7

by // 107 Comments

With the vSphere 7 Launch Event just a few days away, I know many of you are eager to get your hands on this latest release of vSphere and start playing with it in you homelab. A number of folks in the VMware community have already started covering some of the amazing capabilities that will be introduced in vSphere and vSAN 7 and I expect to see that ramp up even more in the coming weeks.

One area that I have not seen much coverage on is around homelab usage with vSphere 7. Given this is a pretty significant release, I think there are some things you should be aware of before you rush out and immediately upgrade your existing homelab environment. As with any vSphere release, you should always carefully review the release notes when they are made available and verify the hardware and its underlying components are officially on the VMware HCL, this is the only way to ensure that you will have a good and working experience.

Having said that, here are just a few of the observations that I have made while running pre-GA builds of vSphere 7 in my own personal homelab. This is not an exhaustive list and I will try to update this article as more information is made available.

Disclaimer: The following considerations below is based on my own personal homelab experience using a pre-GA build of vSphere 7 and it does not reflect any official support or guidance from VMware. Please use these recommendation at your own risk.

[Read more...]

ESXi 5.5 introduces a new Native Device Driver Architecture Part 2

by // 4 Comments

Following up from Part 1 where I provided an overview of the new Native Device Driver architecture introduced in ESXi 5.5, we will now take a deeper look at how this new device driver model works in ESXi. A new concept of driver priority loading is introduced with the Native Device Driver model and the diagram below provides the current ordering of how device drivers are loaded.

As you can see OEM drivers will have the highest priority and by default Native Drivers will be loaded before "legacy" vmklinux drivers. On a clean installation of ESXi 5.5 you should see at least two of these directories: /etc/vmware/default.map.d/ and /etc/vmware/driver.map.d/ which contains driver map files pertaining to Native Device and "legacy" vmklinux drivers.

Here is a screenshot of the map files for both of these directories on an ESXi host:

The following inbox Native Drivers are included in default installation of ESXi 5.5:

Device Device Driver Name
Emulex 10GBe NIC elxnet
Emulex FC lpfc
LSI Megaraid lsi_mr3
LSI mptsas lsi_msgpt3
Micron SSD mtip32xx_native
QLogic FC qlnativefc
SAS/SATA rste
vmxnet3 & graphics vmkernel

As I mentioned earlier, Native Drivers by default will always load before vmklinux drivers, however if you need to perform some troubleshooting, one option is to disable the specific driver in question by using ESXCLI which is applicable to both Native Drivers as well as vmklinux drivers.

To do so, run the following ESXCLI command:

esxcli system module set --enabled=false --module=[DRIVER-NAME]

ESXi 5.5 introduces a new Native Device Driver Architecture Part 1

by // 12 Comments

With a new release of vSphere, many of us are excited about all the new features that we can see and touch. However, what you may or may not notice are some of the new features and enhancements that VMware Engineering has made to the underlying vSphere platform to continue making it better, faster and stronger. One such improvement is the introduction of a new Native Device Driver architecture in ESXi 5.5. Though this feature is primarily targeted at our hardware ecosystem partners, I know some of you have asked about this and I thought it might be useful to share some of the details.

Note: If you are a hardware ecosystem partner and would like to learn more, please reach out to your VMware TAP account managers.

If we take a look back at the early days of ESX, VMware made a decision to use Linux derived drivers to provide the widest variety of support for storage, network and other hardware devices for ESX. Since ESX and specifically the VMkernel is NOT Linux, to accomplish this we built a translation (shim) layer module called vmklinux which sits in between the VMkernel and drivers. This vmklinux module is what enables ESX to function with the linux derived drivers and provides an API which can speak directly to the VMkernel.

Here is a quick diagram of what that looks like:

So why the change in architecture? Since the stability, reliability and performance of these device drivers are importantly critical to ESX(i) second to the VMkernel itself. There is actually a variety of challenges with this architecture in addition to the overhead that is introduced with the translation layer. The vmklinux module must be tied to a specific Linux kernel version and the continued maintenance of vmklinux to provide backwards compatibility across both new and old drivers is quite challenging. From a functionality perspective, we are also limited by the capabilities of the Linux drivers as they are not built specifically for the VMkernel and can not support features such as hot-plug/ To solve this problem, VMware developed a new Native Device Driver model interface that allows a driver to speak directly to the VMkernel and removing the need for the “legacy” vmklinux interface.

Here is a quick diagram of what that looks like:
What are some of the benefits of this new device driver model?
  • More efficient and flexible device driver model compared to vmklinux
  • Standardized information for debugging/troubleshooting
  • Improved performance as we no longer have a translation layer
  • Support for new capabilities such as PCIe hot-plug

This new architecture was developed with backwards compatibility in mind as we all know it is not possible for our entire hardware ecosystem to port their current drivers in one release. To that extent, ESXi 5.5 can run a hybrid of both “legacy” vmklinux drivers as well as the new Native Device Driver. Going forward, VMware will be primarily investing in the Native Device Driver architecture and encourage new device drivers to be developed using the new architecture. VMware also provides an NDDK (Native Driver Development Kit) to our ecosystem partners as well as a sample Native Device Driver which some of you may have seen in the release of vSphere 5.1 with a native vmxnet3 VMkenel module for nested ESXi.

Hopefully this has given you a good overview of the new Native Device Driver architecture and in part 2 of the article I will go into a bit more details on where to find these drivers, which vendor supports this new architecture today and how they are loaded.