I recently had an inquiry from one of our Technical Account Managers (TAM) with an interesting question from their customer. They were looking to manually migrate VMs from one environment to another and because these were isolated and secured environments, they were looking to use an encrypted USB device that would be formatted with VMFS. While researching this topic, they came across several mentions in the community that VMFS on USB has a 2TB limitation, which was not going to work for them.
I personally have never tried nor had the need to ever create a VMFS datastore that was greater than 2TB on a USB device, but I have certainly heard simliar claims in the past and this finally piqued my interests. I reached out to a few folks within VMware Engineering that works on our USB stack and came to learn that is no such limitation when using VMFS on USB. In fact, they also pointed out that some of the reported errors in the community was most likely due to hardware issues rather than capacity of the underlying USB device. This was actually great news and of course I wanted to verify for myself before replying back.
First off, VMFS on USB is NOT officially supported by VMware, so I just want to make sure that is clear. With that said, this is a fairly common practice within the community, especially for VMware Homelabs which I have also demonstrated this capability as early as 2011 and most recently in 2015 for use with vSAN and in 2020 for a vSAN Witness with ESXi on Arm. While most USB storage devices, especially those found in the consumer space are not durable enough for Enterprise usage, it does NOT mean you can not have a reliable USB storage device. I actually wrote about this topic a few years back where you can use inexpensive M.2 enclosures to house an NVMe device that can then be connected via USB/USB-C to have a more reliable storage medium that can also be cost effective.
I was able to get a hold of a 4TB USB storage device and using the following instructions, which have been confirmed by Engineering, I was successful in creating a VMFS datastore that was greater than 2TB running on a USB device using ESXi 7.0 (GA) as well as the latest ESXi 7.0 Update 3c release.
Step 1 - Identify the specific USB storage device that you wish to use for your VMFS datastore, so that we can disable this specific USB device from passthrough. This is handy because you may not want to completely disable USB passthrough for other USB devices and previously, the only method was disabling the USB Arbitrator service in ESXi, which was not ideal.
esxcli hardware usb passthrough device list
Make a note of the Bus, Dev, VendorID and ProductID for the USB device that you wish to disable passthrough for.
Step 2 - Run the following ESXCLI command and replace the Bus, Dev, VendorID and ProductID with the values from Step 1 to disable passthrough for the USB device so that ESXi can now claim it.
esxcli hardware usb passthrough device disable -d 2:5:1058:2621
Step 3 - Next, we need to identify the device name for our USB device which should be in the form of "mpx.vmhbaX:CX:TX:LX". To do so, run the following command and if you have multiple USB storage devices, you can refer to the Size column to confirm you have found the correct device.
vdq -q
Step 4 - Replace the DEVICE_PATH variable with the name of the USB storage device and run the following 5 command which will partition and automatically create a VMFS 6 datastore using the entire device.
DEVICE_PATH="/vmfs/devices/disks/mpx.vmhba32:C0:T0:L0"
partedUtil mklabel ${DEVICE_PATH} msdos
END_SECTOR=$(eval expr $(partedUtil getptbl ${DEVICE_PATH} | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1)
/sbin/partedUtil "setptbl" "${DEVICE_PATH}" "gpt" "1 2048 ${END_SECTOR} AA31E02A400F11DB9590000C2911D1B8 0"
/sbin/vmkfstools -C vmfs6 -b 1m -S usb-datastore ${DEVICE_PATH}:1
Once the operation has successfully completed, you can now navigate to your vSphere Client or ESXi Embedded Host Client (if you do not have vCenter Server) and you should be able to see our newly created VMFS datastore!
Another great article William.
Thank you for taking the time and efort to post. Greatly apreciated.
Problem with Step 4:
# DEVICE_PATH="/vmfs/devices/disks/mpx.vmhba34:C0:T0:L0"
# partedUtil mklabel ${DEVICE_PATH} msdos
Error: Input/output error during read on /dev/disks/mpx.vmhba34:C0:T0:L0
Error: Input/output error during write on /dev/disks/mpx.vmhba34:C0:T0:L0
WriteNewPtable: Unable to commit to device /vmfs/devices/disks/mpx.vmhba34:C0:T0:L0
Any ideas please?
P.S. Some of your images are very small and difficult to read.
Tried:
# esxcfg-advcfg -g /Disk/PreventVMFSOverwrite
Value of PreventVMFSOverwrite is 1
# esxcfg-advcfg -s 0 /Disk/PreventVMFSOverwrite
Value of PreventVMFSOverwrite is 0
# esxcfg-advcfg -g /Disk/PreventVMFSOverwrite
Value of PreventVMFSOverwrite is 0
If it makes a difference, this drive came straight out of an older ESX host.
Can you try another USB device to ensure its not a hardware issue (which is actually quite common with consumer grade USB).
Regarding the photos, you can simply right click and grab URL link which will give you a larger image
Thank you for a great explanation.
I have mounted an 18TB HDD via USB to my Intel NUC7i3BNH and after I plugged it I realized that the software cannot see it. I was in trouble until I found this tutorial. You saved my life and money 🙂
In my example system found an external hard drive after point no. 2. Next I made a partition via GUI.
Many thanks in advance!
Hi, the procedure works. but it seems that after a ESXI reboot i need to redo the procedure as the USB datastore doesnt show, so im scared that it will remove the data
Hi, are there plans for a similar recipe to expand a 2TB VMFS datastore on an USB device with free capacity? Using the ESXi8 webclient actually results in an issue "Failed to expand VMFS datastore local - Failed to update disk partitions for /vmfs/devices/disks/mpx.vmhba33:C0:T0:L0." Such a feature for VMFS version 6.x and above would be nice.
Hello.
Thanks for he article. A quick question, I get a Connection timed out error when trying this. Any ideas as to why it could be?
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs6 file system on "mpx.vmhba49:C0:T0:L0:1" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "externaldatastore".
Failed to create VMFS on device mpx.vmhba49:C0:T0:L0:1
Usage: vmkfstools -C [vmfs5|vmfs6|vfat] /vmfs/devices/disks/vml... or,
vmkfstools -C [vmfs5|vmfs6|vfat] /vmfs/devices/disks/naa... or,
vmkfstools -C [vmfs5|vmfs6|vfat] /vmfs/devices/disks/mpx.vmhbaA:T:L:P
Error: Connection timed out
This worked for me to create VMFS 4TB USB drive. However, ESXI 8.0 will not allow me to create a datastore. The options is greyed out. Appreciate you putting this out there nonetheless.
ESXi version:
8.0.2
ESXi build number:
22380479
Well, the method above worked as intended. I am not able to create a datastore because the datastore was already created via this Mr. Lam's method. vCenter is showing that I have 3.73TB as "usb-datastore". Thank you very much.
Hi William,
I want to comment on this, because it may be useful to someone:
Some USB 3.0 drives can't be recognized by ESXi. I've tested it from 5.5 to 8.0, and every time the same result: vmhba3x is detected (dmesg), but it doesn't appears in "/dev/disks/". Finally, I discovered that this is true for all TOSHIBA external USB 3.0 drives. So, don't use any disk with VID "0480" in ESXi.
I don't know the root cause of this. Perhaps some blacklist involved? Or a faulty firmware? Unsupported UASP protocol? I don't know it. And perhaps this could be true with other brands. However with several WD disks and USB cases (NVMe adapters) I don't have any problem.