Search Results for: intel NUC

Quick Tip - iPerf now available on ESXi

03.15.2016 by William Lam // 25 Comments

The other day I was looking to get a baseline of the built-in ethernet adapter of my recently upgraded vSphere home lab running on the Intel NUC. I decided to use iPerf for my testing which is a commonly used command-line tool to help measure network performance. I also found a couple of articles from well known VMware community members: Erik Bussink and Raphael Schitz on this topic as well which were quite helpful. Erik's article here outlines how to run the iPerf Client/Server using a pair of Virtual Machines running on top of two ESXi hosts. Although the overhead of the VMs should be negligible, I was looking for a way to benchmark the ESXi hosts directly. Raphael's article here looked promising as he found a way to create a custom iPerf VIB which can run directly on ESXi.

I was about to download the custom VIB and I had just remembered that the VSAN Health Check plugin in the vSphere Web Client also provides some proactive network performance tests to be run on your environment. I was curious on what tool was being leveraged for this capability and in doing a quick search on the ESXi filesystem, I found that it was actually iPerf. The iPerf binary is located in /usr/lib/vmware/vsan/bin/iperf and looks to have been bundled as part of ESXi starting with the vSphere 6.0 release from what I can tell.

UPDATE (10/19/23) - As of ESXi 8.x, you may run into following error when attempting to start running iperf:

iperf3: running in appDom(30): ipAddr = ::, port = 5201: Access denied by vmkernel access control policy

To workaround this, you can run the following command:

esxcli system secpolicy domain set -n appDom -l disabled

and re-enable (use "enforcing" as the value) once you have completed your iperf test both on the server and client ESXi host.

UPDATE (09/20/22) - As of ESXi 7.0 Update 3 20036589 (possibly earlier) and later, you no longer need to make a copy of the iperf3 utility. You can simply run it from /usr/lib/vmware/vsan/bin/iperf3 and you also do NOT have to lower the security by changing ESXi Advanced Setting execInstalledOnly to FALSE

UPDATE (10/02/18) - It looks like iPerf3 is now back in both ESXi 6.5 Update 2 as well as the upcoming ESXi 6.7 Update 1 release. You can find the iPerf binary under /usr/lib/vmware/vsan/bin/iperf3

One interesting thing that I found when trying to run iPerf in "server" mode is that you would always get the following error:

bind failed: Operation not permitted

The only way I found to fix this issue was to basically copy the iPerf binary to another file like iperf3.copy which it would then allow me to start iPerf in "server" mode. You can do so by running the following command on the ESXi Shell:

cp /usr/lib/vmware/vsan/bin/iperf3 /usr/lib/vmware/vsan/bin/iperf3.copy

Running iPerf in "Client" mode works as expected and the copy is only needed when running in "server" mode. To perform the test, I used both my Apple Mac Mini and the Intel NUC which had ESXi running with no VMs.

I ran the iPerf "Server" on the Intel NUC by running the following command:

/usr/lib/vmware/vsan/bin/iperf3.copy -s -B [IPERF-SERVER-IP]

Note: If you have multiple network interfaces, you can specify which interface to use with the -B option and passing the IP Address of that interface.

I ran the iPerf "Client" on the Mac Mini by running the following command and specifying the address of the iPerf "Server":

/usr/lib/vmware/vsan/bin/iperf3 -n 800M -c [IPERF-SERVER]

I also disabled the ESXi firewall before running the test, which you can do by running the following command:

esxcli network firewall set --enabled false

Here is a screenshot of my iPerf test running between my Mac Mini and Intel NUC. Hopefully this will come in handy for anyone needing to run some basic network performance tests between two ESXi hosts without having to setup additional VMs.

The future of the ESXi Embedded Host Client

03.04.2016 by William Lam // 14 Comments

As many of you know, the ESXi Embedded Host Client project is something that is very near and dear to my heart. I have always felt that we needed a simple web interface that customers can just point their web browser to an ESXi host after a new installation and be able to quickly get started. One of the biggest benefit in addition to simplicity is that it is also very intuitive from a user experience standpoint which I believe is very important in a world where things can quickly get complex. In addition, it can also provide an interface for basic troubleshooting and support greenfield deployments where vCenter Server has not been deployed yet.

It has truly been amazing to follow the Embedded Host Client development from the initial idea to the first prototype built by VMware Engineers Kevin Christopher and Jehad Affoneh to its current implementation lead by Etienne Le Sueur and the ESXi team. I have really been fortunate to have had the opportunity to be so involved in this project. It is hard to imagine that in just little over 6 months, we have had had 5 releases of the Embedded Host Client Fling, all of which, produced with high quality development and rich feature sets.

You can click on the links below to get more details about each release.

08/11/15 - EHC Fling v1 released
08/26/15 - EHC Fling v2 released
10/23/15 - EHC Fling v3 released
12/21/15 - EHC Fling v4 released
02/07/16 - EHC Fling v5 released

I think its an understatement to say that customers are genuinely excited about this project as well, just look at some of the comments left on the Flings page here. Interestingly, this excitement has also been felt internally at VMware as well and I think this goes to show that the team has built something really special that affects anyone who works with VMware's ESXi Hypervisor.

So where to do we go from here? Are we done? Far from it ...

For those of you who follow me on Twitter know that I had recently refreshed my personal vSphere home lab from a Apple Mac Mini to latest Intel NUC running the yet to be release VSAN 6.2 (vSphere 6.0 Update 2). I was pleasantly surprised to see that ESXi Embedded Host Client (EHC) is now included out of the box with ESXi! Although this has been said by a few folks including myself, it is another thing to actually see it in person 🙂

Although the VMware Flings program is a great way to share and engage with our customers to get early feedback, it may not always be a viable option. As some of you may know, Flings are not officially supported and this sometimes prevents some of our customers from engaging with us and really putting the Flings through its paces. By making EHC out of the box, not only are we officially supporting it but it will also make it easier for customers to try out this new interface.

UPDATE (03/04/16) - It looks like I made a mistake and that the ESXi Embedded Host Client will NOT be released as a "Tech Preview" as previously mentioned but rather it will be officially GA'ed with vSphere 6.0 Update 2. EHC is a fully supported feature of ESXi.

Although EHC is very close to parity with the vSphere C# Client, it is still not 100% there. We will continue to improve its capabilities and if you have any feedback when trying out the EHC, do not hesitate and leave feedback or file a Feature Request through GSS. For those looking to live on the "edge" a bit more, we will still continue to release updates to the EHC Fling but if you want something that is stable, you can stick with the stock EHC included in ESXi 6.0 Update 2. We will still ship the legacy Windows vSphere C# Client, so you will not be forced to use this interface. However, it is no secret that VMware wants to get rid of the vSphere C# Client and that EHC is the future interface to standalone ESXi hosts.

One feature that I know that many of you have been asking about is Free ESXi. Well, I am please to say that support for Free ESXi has been added in the latest version of EHC included with the upcoming ESXi 6.0 Update 2 release and below is a screenshot demonstrating that it is fully functional.

Lastly, I just want to say that EHC has really morphed beyond just a "simple UI" for managing standalone ESXi hosts and has also enabled other teams at VMware to do some really amazing things and create new experiences with this interface. As I said earlier, this is just the beginning 😀 Happy Friday!

Here are some additional cool capabilities provided by EHC

Working USB Ethernet Adapter (NIC) for ESXi

03.01.2016 by William Lam //

As part of upgrading my personal vSphere home lab from an Apple Mac Mini to an Intel NUC (more on this in a future blog), I have been researching to see if there are other alternatives for adding an additional network adapter. The Intel NUC only includes a single built-in ethernet adapter which is similar to the Mac Mini. However, the NUC also lacks additional IO connectors like a Thunderbolt port which the Mac Mini includes and can support a Thunderbolt to Ethernet adapter. I think this is probably the only downside to the Intel NUC platform which has been similar feedback that I have seen from other vSphere home labbers who currently use or would like to use the NUC. Perhaps, with the next update of the NUC platform code named "Skull Canyon", the rumored Thunderbolt 3 / USB-c connector may make things easier as some of the existing vendors who produce Thunderbolt to ethernet adapter also use common drivers like the Broadcom tg3 which have historically worked with ESXi.

One option that has been suggested by many folks over the years was to see if a USB based ethernet adapter could be used to provide additional networking to ESXi? Historically, the answer had been no because there were no known device drivers that would work with ESXi. I had even looked into this a few years ago and although I ran into some folks who seemed to have made it work, I was never able to find the right USB ethernet adapter to personally confirm myself. It was only until last week, I decided to start fresh again and after a bit of Googling I came across an old VMTN thread here where VMTN user AK_____28 mentioned he had success with the StarTech USB 3.0 to Gigabit Ethernet NIC Network Adapter and using a custom compiled driver that was posted over here by another user named Trickstarter.

UPDATE (02/12/19) - A new VMware Native Driver for USB-based NICs has just been released for ESXi 6.5/6.7, please use this driver going forward. If you are still on ESXi 5.5/6.0, you can continue using the existing driver but please note there will be no additional development in the existing vmklinux-based driver.

UPDATE (03/29/16) - Please have a look at this updated article here which includes ESXi 5.5 and 6.0 driver.

Disclaimer: In case it is not clear and apparent, you should never install any unknown 3rd party software on your ESXi host as it can potentially lead to instability issues or worse open yourself to a security hole. The following solution is not officially supported by VMware, please use at your own risk.

I decided to bite the bullet and give this solution a try and purchased the USB ethernet adapter from Amazon here.

There are two modules that needs to be downloaded, extracted and loaded onto ESXi. I have included the links below for your convenience:

ax88179vz026.gz
usbnetvz026.gz

As the VMTN thread mentioned, you can load using either the vmkload_mod or ESXCLI. Here are the two commands that I used in the following order:

vmkload_mod /vmfs/volumes/mini-local-datastore-1/usbnetvz026
vmkload_mod /vmfs/volumes/mini-local-datastore-1/ax88179vz026

When I tried to initially load either of the modules, I would always get the following error:

vmkwarning.log:2016-02-28T21:54:54.531Z cpu6:374787)WARNING: Elf: 2041: Load of <usbnetvz026> failed : missing required namespace <com.vmware.usb#9.2.1.0>

As you can imagine, I was pretty bummed to see this since I was afraid that something like this would happen. I was not sure if the device I had purchased no longer worked or if was the drivers? I saw that these modules were initially compiled for ESXi 5.1 (at the time, that was the latest version) and the only difference was that I was using a much newer version of ESXi, specifically 6.0 Update 1. I decided to install the latest version of ESXi 5.1 Update 3 and tried the process again and to my surprise, the modules loaded without errors. I suspect that this was a hard dependency on the namespace version which was version 9.2.1.0 and the latest version is now 9.2.3.0.

After successfully loading the two modules, I ran the following command:

esxcfg-nics -l

to verify that ESXi did in fact did claim the USB ethernet device and as you can see from the screenshot below, it did indeed!

Next up, I needed to verify basic connectivity and added the new uplink to my existing vSwitch. You must use the following ESXCLI command (esxcfg-vswitch command does not work apparently for non vmnicX devices)

esxcli network vswitch standard uplink add -u vusb0 -v vSwitch0

Once added, I hopped over to the vSphere C# Client to see if the device is now showing up under the Network Adapters tab, which it is.

Finally, the last test was to make the vsb0 (this is how ESXi names the device) device the active connection while moving my existing vmnic0 to stand-by. Networking connectivity continued to function and I was even able to transfer an ISO image over the USB ethernet adapter without any issues.

So it looks like it is possible to get a USB based ethernet adapter to function with ESXi, at least with the specific model listed above (PCI ID 0b95:1790). The challenge now is to see if there is a way to build an updated version of the drivers targeted at the latest ESXi 6.0 release. From what I have been able to follow on the forum here, it looks like there was also some amount of non-trivial code changes that were required to get the driver to function. If true, without those changes, it can difficult to re-compile the driver. I have reached out to the original author to see if he might be able to share the changes he made to the driver code. In the mean time, if folks are interested in giving the build process a try, Trickstarter did a great two part write up on how to setup your build environment and compile an example driver.

Although the write up is targeted at ESXi 5.x, you can download the equilvenet packages for ESXi 6.0 which includes the ESXi Open Source Disclosure Package as well as the VMware Toolchain which is required and used to compile the source code. I have provided the direct download links below.

You can also find the latest version of the USB ethernet adapter ax88179 ASIX driver here. I have also attempted to compile just the driver but have already ran into some issues. I have not had time to dig further, so not sure how far I will be able to get. Any tips or tricks others may have for compiling against ESXi 6.0, feel free to share them and I will give them a shot when I get some time!