WilliamLam.com

Emulating a Virtual USB storage device using Nested ESXi

by // Leave a Comment

My buddy Alan Renouf had pinged me earlier today and asked whether it was possible to emulate a USB storage device that could aide him in the testing the installation of ESXi from a USB device but without having to use a real USB device. I honestly was not aware of any mechanisms that would allow for this and I normally would just passthrough a real USB device to a Nested ESXi VM for this type of testing purposes.

While thinking about his question, I also recall we had made some enhancements to our Virtual USB interface that would allow user to back it using a disk file. While searching further, I came to learn that not only was this possible, but it was also a common method for testing USB-based installation without the hassle of messing with physical hardware. It turns out you can just present a Virtual Disk (VMDK) to a VM running ESXi (Nested ESXi) and through a special driver, it will recognize the device as a USB storage device!

I definitely wish I had learned about this earlier and it goes to show, all the hard engineering efforts made by our VMware Engineers to make testing and using our software as easy as possible even without needing real physical hardware 😀

[Read more…]

Creating a VMFS datastore greater than 2TB on a USB device in ESXi

by // 6 Comments

I recently had an inquiry from one of our Technical Account Managers (TAM) with an interesting question from their customer. They were looking to manually migrate VMs from one environment to another and because these were isolated and secured environments, they were looking to use an encrypted USB device that would be formatted with VMFS. While researching this topic, they came across several mentions in the community that VMFS on USB has a 2TB limitation, which was not going to work for them.

I personally have never tried nor had the need to ever create a VMFS datastore that was greater than 2TB on a USB device, but I have certainly heard simliar claims in the past and this finally piqued my interests. I reached out to a few folks within VMware Engineering that works on our USB stack and came to learn that is no such limitation when using VMFS on USB. In fact, they also pointed out that some of the reported errors in the community was most likely due to hardware issues rather than capacity of the underlying USB device. This was actually great news and of course I wanted to verify for myself before replying back.

First off, VMFS on USB is NOT officially supported by VMware, so I just want to make sure that is clear. With that said, this is a fairly common practice within the community, especially for VMware Homelabs which I have also demonstrated this capability as early as 2011 and most recently in 2015 for use with vSAN and in 2020 for a vSAN Witness with ESXi on Arm. While most USB storage devices, especially those found in the consumer space are not durable enough for Enterprise usage, it does NOT mean you can not have a reliable USB storage device. I actually wrote about this topic a few years back where you can use inexpensive M.2 enclosures to house an NVMe device that can then be connected via USB/USB-C to have a more reliable storage medium that can also be cost effective.

[Read more…]

Disable passthrough for specific USB devices for use with ESXi

by // 7 Comments

There was another interesting question that I saw recently asking whether you can use a USB device for an ESXi Datastore while still maintaining the ability to passthrough other USB devices to a VM? This was interesting because my understanding was that you had to choose between one of the two options because ESXi required the USB Arbitrator service to be disabled for USB storage (not officially supported by VMware) and thus disabling USB passthrough for all of your VMs.

I decided to reach out to our USB expert within VMware Engineering, Songtao, who you may know as developer for the popular USB Network Native Driver for ESXi and the Community Networking Driver for ESXi Fling for this thoughts on this question. It turns out my assumption has actually been incorrect and there is actually a way to easily exclude specific USB devices from VM passthrough which can then be used by ESXi.

[Read more…]