WilliamLam.com

Creating a VMFS datastore greater than 2TB on a USB device in ESXi

by // 12 Comments

I recently had an inquiry from one of our Technical Account Managers (TAM) with an interesting question from their customer. They were looking to manually migrate VMs from one environment to another and because these were isolated and secured environments, they were looking to use an encrypted USB device that would be formatted with VMFS. While researching this topic, they came across several mentions in the community that VMFS on USB has a 2TB limitation, which was not going to work for them.

I personally have never tried nor had the need to ever create a VMFS datastore that was greater than 2TB on a USB device, but I have certainly heard simliar claims in the past and this finally piqued my interests. I reached out to a few folks within VMware Engineering that works on our USB stack and came to learn that is no such limitation when using VMFS on USB. In fact, they also pointed out that some of the reported errors in the community was most likely due to hardware issues rather than capacity of the underlying USB device. This was actually great news and of course I wanted to verify for myself before replying back.

First off, VMFS on USB is NOT officially supported by VMware, so I just want to make sure that is clear. With that said, this is a fairly common practice within the community, especially for VMware Homelabs which I have also demonstrated this capability as early as 2011 and most recently in 2015 for use with vSAN and in 2020 for a vSAN Witness with ESXi on Arm. While most USB storage devices, especially those found in the consumer space are not durable enough for Enterprise usage, it does NOT mean you can not have a reliable USB storage device. I actually wrote about this topic a few years back where you can use inexpensive M.2 enclosures to house an NVMe device that can then be connected via USB/USB-C to have a more reliable storage medium that can also be cost effective.

[Read more...]

Disable passthrough for specific USB devices for use with ESXi

by // 11 Comments

There was another interesting question that I saw recently asking whether you can use a USB device for an ESXi Datastore while still maintaining the ability to passthrough other USB devices to a VM? This was interesting because my understanding was that you had to choose between one of the two options because ESXi required the USB Arbitrator service to be disabled for USB storage (not officially supported by VMware) and thus disabling USB passthrough for all of your VMs.

I decided to reach out to our USB expert within VMware Engineering, Songtao, who you may know as developer for the popular USB Network Native Driver for ESXi and the Community Networking Driver for ESXi Fling for this thoughts on this question. It turns out my assumption has actually been incorrect and there is actually a way to easily exclude specific USB devices from VM passthrough which can then be used by ESXi.

[Read more...]

How to passthrough USB Keyboard/Mouse HID and CCID devices to VM in ESXi?

by // 66 Comments

About a month back I had received an interesting tidbit from Darius Davis (VMware Engineer) after helping a customer solve an interesting problem and Darius thought this could be a useful blog post to share. Funny enough, a couple of weeks after that conversation, a simliar issue was being faced by another customer and luckily I was able to share with them the solution and also validate the specific configuration that was needed.

The customer that Darius was helping out had two VMs running on ESXi which they wanted to configure several passthrough devices. In addition to a PCI passthrough of a GPU, they also wanted to passthrough independent USB keyboard and mouse to each individual VM. PCI passthrough to a VM is nothing new but passing through a USB keyboard/mouse also known as Human Interface Devices (HID) to a VM is generally not expected. The physical ESXi host just assumes these type of USB devices are meant for it to consume.

In addition to HID USB devices, there are also Chip Card Interface Devices (CCID) USB devices like a smart card reader which customers may also want to passthrough to a VM. The latter use case was what I ended up helping the customer out with. To passthrough HID/CCID USB devices, the following steps are required which will include changes to the ESXi host.

Step 1 - Add the following two VM Advanced Settings for all USB CCID/HID devices that you wish to enable passthrough:

HID USB Devices:

usb.generic.allowHID = "TRUE"
usb.quirks.device0 = "0xXXXX:0xYYYY allow"

CCID USB Devices:

usb.generic.allowCCID = "TRUE"
usb.quirks.device0 = "0xXXXX:0xYYYY allow"

where 0xXXXX = vendorId and 0xYYYY = deviceId (e.g 0x03f0:0x0024) which was retrieved from the previous step

To list all USB devices and get their vendor and device ID, you can use the lsusb command found within the ESXi Shell and below is an example listing out both my USB Mouse and Keyboard.

lsusb -v | grep -E '(^Bus|HID)'
Bus 001 Device 001: ID 0e0f:8003 VMware, Inc. Root Hub
Bus 002 Device 001: ID 0e0f:8003 VMware, Inc. Root Hub
Bus 001 Device 003: ID 8087:0026 Intel Corp.
Bus 001 Device 004: ID 0781:5591 SanDisk Corp. Ultra Flair
Bus 001 Device 002: ID 05fe:0011 Chic Technology Corp. Browser Mouse
    iConfiguration          4 HID Mouse
        HID Device Descriptor:
          bcdHID               1.00
Bus 001 Device 005: ID 046d:c31d Logitech, Inc. Media Keyboard K200
        HID Device Descriptor:
          bcdHID               1.10
        HID Device Descriptor:
          bcdHID               1.10

Step 2 - We need to make the USB arbitrator service aware of these USB device quirks by adding the usb.quirks.deviceN string to /etc/vmware/config file. In my example above, I want to enable passthrough for both my Mouse and Keyboard device, so the following entries would be appended:

usb.quirks.device0 = "0x05fe:0x0011 allow"
usb.quirks.device1 = "0x046d:0xc31d allow"

Step 3 - Lastly, we need to append the following string to the ESXi boot option to disable the VMkernel from claiming HID USB devices.

CONFIG./USB/quirks=0xXXXX:0xYYYY::0xffff:UQ_KBD_IGNORE

where 0xXXXX = vendorId and 0xYYYY = deviceId (e.g 0x03f0:0x0024)

The easiest way to append this to the boot option is by editing /bootbank/boot.cfg rather than manually typing this during the initial boot up (SHIFT+O)

Note1: This is not required for CCID devices or mouse devices as they are not claimed by ESXi

Note2: The USB quirks are given in sets of five parameters - vendorID:deviceID:minRevision:maxRevision:quirkName If you wish to specify multiple devices, you will need to ensure all five parameters are included. Here's an example for specifying two USB devices: CONFIG./USB/quirks=0x05fe:0x0011::0xffff:UQ_KBD_IGNORE:0x046d:0xc31d::0xffff:UQ_KBD_IGNORE

Step 4 - A system reboot of your ESXi host will be required for the changes to go into effect. Once your ESXi host is available, you can use the vSphere H5 Client or Embedded ESXi Host Client to attach the USB devices. For vCenter Server, click on "Add New Device" and select "Host USB Device" and for ESXi, click on "Add other device" and select "USB Device".