WilliamLam.com

Removable M.2 NVMe SSD PCIe enclosure by Icy Dock

by // 6 Comments

My homelab is a constant experiment and hardware components are moved around for various testing, especially when it comes to networking and storage. When needing to move around an M.2 NVMe SSD, complexity of taking apart a system will vary on the platform but generally it is inconvenience. When I came to learn that Icy Dock, a manufacturer of storage enclosures, will be releasing a removable M.2 NVMe SSD tray that is connected to PCIe expansion slot, I knew I had to get my hands on it.


The good folks over at Icy Dock were kind enough to send me an early evaluational unit of the upcoming MB840M2P-B which is now available for $69. will be released in August and should retail for around $80 USD (final prices are still TBD). The use case above may not apply to most folks and is probably unique to my specific hardware usage but I think this is still a very interesting solution that is still useful to be aware of if you are your own homelab whitebox and have a spare PCIe slot. Icy Dock also produces many other types of storage enclosures that you might find interesting based your own needs.

For my setup, I installed the MB840M2P-B into my Intel NUC 9 Pro, which is definitely not easy to take apart. This is especially true for the two M.2 which is attached to the NUC Element but even more painful to get to the 3rd M.2 which is located under the baseboard. For my specific use case, this was well worth using up one of the PCIe slots on the NUC 9 Pro! This enclosure can also be added to the new 2019 Mac Pro which is another platform that Icy Dock sees benefiting from this solution.

[Read more...]

Thunderbolt 3 enclosures with (Single, Dual & Quad) M.2 NVMe SSDs for ESXi

by // 15 Comments

Thunderbolt 3 (TB3) and eventually USB 4 is a really fascinating technology and I believe it still has so much untapped potential, especially when looking at Remote/Branch Office (ROBO), Edge and IoT types of deployments. TB3 was initially limited to Apple-based platforms, but in the last couple of years, adoption has been picking up across a number of PC desktop/laptops including the latest generations of Intel NUCs which are quite popular for vSphere/vSAN/NSX Home Labs. My hope with USB 4 is that in the near future, we will start to see servers with this interface show up in the datacenter 🙂

In the mean time, I have been doing some work with TB3 from a home lab standpoint. Some of you may have noticed my recent work on enabling Thunderbolt 3 to 10GbE for ESXi and it should be no surprise that the next logical step was TB3 storage. Using a Thunderbolt interface to connect to external storage, usually Fibre Channel is something many of our customers have been doing for quite some time. In fact, I have a blog post from a few years back which goes over some of the solutions customers have implemented, the majority use case being Virtualizing MacOS on ESXi for iOS/MacOS development. These solutions were usually not cheap and involved a sizable amount of infrastructure (e.g. storage arrays, network switches, etc) but worked very well for large vSphere/MacOS based environments.

[Read more...]

Quick Tip - Crucial NVMe SSD not recognized by ESXi 6.7 & 7.0

by // 105 Comments

If you own or have recently purchased Crucial NVMe SSD such as CT1000P1SSD8 (1TB M.2 NVMe SSD) or CT500P1SSD8 (500GB M.2 NVMe SSD), please be aware that these devices may no be recognized by ESXi after upgrading to the latest release. Thanks to Pete Lindley, (OCTO for End-User Computing), who reached out last week regarding the observation as well as a workaround for the problem. This was also quite timely as I recently purchased a Crucial M.2 NVMe SSD and would have also ran into this problem.

It turns out these Crucial devices were working fine while running on ESXi 6.5 Update 2 but was no longer recognized in latest release of ESXi 6.7 Update 2. It is unclear whether support for these SSDs were removed intentionally or unintentionally, but in either case, these devices are not officially on VMware's Hardware Compatibility List (HCL).

UPDATE (07/29/20) - Over the past few months, I have had a number of folks share feedback that using the trick mentioned below for ESXi 7.0, they have had success of ESXi detecting their NVMe SSD. I wanted to share some of the model and/or vendors that folks have reported success with. I will keep this list updated, so feel free to leave a comment below.

UPDATE (06/13/20) - Thanks to reader Dave, it looks like this trick also works with ESXi 7.0 but the filename has changed. Simply copy nvme.v00 VIB from the ESXi 6.5 Update 2 and replace it on ESXi 7.0 system (either live under /bootbank or part of the installer) but rename the file to nvme_pci.v00 which is the new filename for NVMe driver.

UPDATE (05/23/19) - After speaking with a few folks who took a closer look, the issue is due to the fact that we added support for NVMe 1.3 spec in latest ESXi 6.7 Update 2 release, but because these are "consumer" devices, they did not conform to the latest specification and hence the driver is unable to claim the device. This is another good reminder when using components not on VMware HCL, this is always a risk from a home lab perspective. In general, I know Samsung and Intel NVMe SSD usually works quite well without issues but always good to do some research. I think Engineering is looking to see if there are other workarounds for the future, but for now, you can use the workaround below.

The easy workaround that Pete found was to simply replace the NVMe driver from ESXi 6.7 Update 2 (1.2.2.27-1vmw.670.2.48.13006603) with one found in ESXi 6.5 Update 2 (1.2.1.34-1vmw.650.2.50.8294253). To so do, simply copy nvme.v00 to /bootbank from either an existing ESXi 6.5 Update 2 system or directly from the ISO. Please note, any future updates or patches to the ESXi host will most likely override the updated driver.