WilliamLam.com

How to recover ESXi installed on USB device after disabling vmkusb module?

by // Leave a Comment

I have to say, this is one of the more interesting challenges that I have come across in quite some time. A user was looking for assistance after they accidentally disabled the vmkusb module, which is the USB driver for ESXi and allows it to communicate with USB devices that are connected to the system.

The vmkusb module also plays a very critical role if you have ESXi installed on a USB device, as the driver is required for proper functionality such as being able to save the ESXi state and configurations to the USB device. So what happens when you disable the vmkusb module and you reboot the ESXi host, which is also installed on a USB device?

Well, everything continues to work including VMs since ESXi by design runs in memory after the initial boot from the USB device. However, any configuration changes made after that is lost after a system reboot including the attempt to re-enable the vmkusb module since ESXi is unable save any of the settings to the USB device. Fortunately, I was able to help the user out as I had a few ideas on how we could fully recover from this type of scenario and hence the blog post.

Hopefully a lesson can be learned here, do not make changes or disable things that you are not familiar with 🙂

[Read more...]

Emulating a Virtual USB storage device using Nested ESXi

by // Leave a Comment

My buddy Alan Renouf had pinged me earlier today and asked whether it was possible to emulate a USB storage device that could aide him in the testing the installation of ESXi from a USB device but without having to use a real USB device. I honestly was not aware of any mechanisms that would allow for this and I normally would just passthrough a real USB device to a Nested ESXi VM for this type of testing purposes.

While thinking about his question, I also recall we had made some enhancements to our Virtual USB interface that would allow user to back it using a disk file. While searching further, I came to learn that not only was this possible, but it was also a common method for testing USB-based installation without the hassle of messing with physical hardware. It turns out you can just present a Virtual Disk (VMDK) to a VM running ESXi (Nested ESXi) and through a special driver, it will recognize the device as a USB storage device!

I definitely wish I had learned about this earlier and it goes to show, all the hard engineering efforts made by our VMware Engineers to make testing and using our software as easy as possible even without needing real physical hardware 😀

[Read more...]

Creating a VMFS datastore greater than 2TB on a USB device in ESXi

by // 7 Comments

I recently had an inquiry from one of our Technical Account Managers (TAM) with an interesting question from their customer. They were looking to manually migrate VMs from one environment to another and because these were isolated and secured environments, they were looking to use an encrypted USB device that would be formatted with VMFS. While researching this topic, they came across several mentions in the community that VMFS on USB has a 2TB limitation, which was not going to work for them.

I personally have never tried nor had the need to ever create a VMFS datastore that was greater than 2TB on a USB device, but I have certainly heard simliar claims in the past and this finally piqued my interests. I reached out to a few folks within VMware Engineering that works on our USB stack and came to learn that is no such limitation when using VMFS on USB. In fact, they also pointed out that some of the reported errors in the community was most likely due to hardware issues rather than capacity of the underlying USB device. This was actually great news and of course I wanted to verify for myself before replying back.

First off, VMFS on USB is NOT officially supported by VMware, so I just want to make sure that is clear. With that said, this is a fairly common practice within the community, especially for VMware Homelabs which I have also demonstrated this capability as early as 2011 and most recently in 2015 for use with vSAN and in 2020 for a vSAN Witness with ESXi on Arm. While most USB storage devices, especially those found in the consumer space are not durable enough for Enterprise usage, it does NOT mean you can not have a reliable USB storage device. I actually wrote about this topic a few years back where you can use inexpensive M.2 enclosures to house an NVMe device that can then be connected via USB/USB-C to have a more reliable storage medium that can also be cost effective.

[Read more...]