Emulating a Virtual USB storage device using Nested ESXi

My buddy Alan Renouf had pinged me earlier today and asked whether it was possible to emulate a USB storage device that could aide him in the testing the installation of ESXi from a USB device but without having to use a real USB device. I honestly was not aware of any mechanisms that would allow for this and I normally would just passthrough a real USB device to a Nested ESXi VM for this type of testing purposes.

While thinking about his question, I also recall we had made some enhancements to our Virtual USB interface that would allow user to back it using a disk file. While searching further, I came to learn that not only was this possible, but it was also a common method for testing USB-based installation without the hassle of messing with physical hardware. It turns out you can just present a Virtual Disk (VMDK) to a VM running ESXi (Nested ESXi) and through a special driver, it will recognize the device as a USB storage device!

I definitely wish I had learned about this earlier and it goes to show, all the hard engineering efforts made by our VMware Engineers to make testing and using our software as easy as possible even without needing real physical hardware 😀

Step 1 - Convert your desired ESXi ISO image into the IMG format. For macOS users, you can use the hdiutil command-line utility and the syntax is the following:

hdiutil convert VMware-VMvisor-Installer-7.0U3d-19482537.x86_64.iso -format UDRW -o esxi70u3d.img

Step 2 - Transfer the ESXi IMG file to your ESXi host datastore

Step 3 - Create a new VM and select Other / VMware ESXi 7.0 or later as the GuestOS and configure the desired compute resources. You will also need to add a USB Controller (3.1) before completing the VM creation wizard. In my example, I have named the VM Nested-ESXi-Virtual-USB and will be referencing the name throughout the instructions.

Depending on your use case, you may or may not need create an additional Virtual Disk because our Virtual USB storage device can also be used, which also mimics the behavior you get with a physical USB storage device that contains the ESXi installer.

Step 4 - SSH to your physical ESXi host and then create an empty VMDK using vmkfstools utility with the desired name and size. The path to the VMDK should ideally be in the root directory of the VM that you had created in the previous step.

vmkfstools -c 6G /vmfs/volumes/esxi-local-datastore/Nested-ESXi-Virtual-USB/usb.vmdk

Step 5 - We now need to copy the contents of our ESXi IMG file into our VMDK (flat file) using the dd utility and the syntax to do so is the following:

dd if=/vmfs/volumes/esxi-local-datastore/esxi70u3d.img of=/vmfs/volumes/esxi-local-datastore/Nested-ESXi-Virtual-USB/usb-flat.vmdk conv=notrunc

Step 6 - Next, we need to edit the VMX file for our VM and append the following which will add the Virtual USB storage device pointing to our VMDK file (usb.vmdk)

usb_xhci:0.present = "TRUE"
usb_xhci:0.deviceType = "disk"
usb_xhci:0.fileName = "usb.vmdk"
usb_xhci:0.readOnly = "FALSE"
usb_xhci:0.speed = "16"
usb_xhci:0.parent = "-1"
usb_xhci:0.port = "0"

Step 7 - Finally, we need to reload the VM configuration since we have manually edited the VMX file using the following command:

vim-cmd vmsvc/reload $(vim-cmd vmsvc/getallvms | grep 'Nested-ESXi-Virtual-USB' | awk '{print $1}')

The only thing left to do now is to power on our VM! If you look at the vSphere UI, you will also notice a new entry that shows our VM is now configured with our newly added Virtual USB storage device as shown in the screenshot below.

If you are using the default settings when creating your Nested ESXi VM, the Firmware will automatically default to EFI. This is important because both the CD-ROM and Network boot is before USB Devices. You will need to hit the ESC key while the VM is powering up to enter the EFI boot menu to select booting from the USB Device.

Note: Only EFI firmware is supported with Virtual USB Storage, using BIOS will not work.

At this point, you should now see the ESXi Installer boot up normally which is coming directly from the VMDK.

Once the ESXi Installer has fully booted up, we can also go into the ESXi Shell (Alt+F1) and confirm that ESXi now sees our VMDK as a USB Storage Device by using any of the following commands:

• vdq -q
• lsusb
• localcli storage core device list