A question that I frequently receive is whether ESXi supports Thunderbolt-based storage devices. This is especially interesting for folks running ESXi on an Apple Mac Mini due to the limited number of IO connections the Mac Minis' have. If you look on VMware's HCL, you will not find any supported Thunderbolt Storage devices nor are there any that are being actively tested with ESXi, at least as far as I know.
Having said that, generally speaking from an ESXi host point of view, the Thunderbolt interface is just seen as an extended PCIe bus. This means that whatever storage device is connected on the other end can work with ESXi as long as there is a driver in ESXi that can communicate with that devices. This is analogous to having a RAID card and having the proper device driver on ESXi to see its storage.
Even though VMware is not actively testing Thunderbolt-based storage devices, there are a few folks out in the community who have and have been successful. I wanted to share these stories with the community for those that might be interested in this topic and hopefully others who have had similar success can also share more details about their setup.
UPDATE (09/12/16) - ESXi Thunderbolt Driver to Fibre Channel Storage from ATTO
Disclaimer: All solutions listed below are from the community and decisions to purchase based on these solutions will be at your own risk. I hold no responsibility if the listed solutions do not work for whatever reason.
Solution #1 - Pegaus R6 Thunderbolt Storage Enclosure
This was the first Thunderbolt storage device that I had ever seen confirmed publicly to work with ESXi after installing a STEX driver VIB. You can find more details here.
Solution #2 - Sonnet Echo Express III-R Rackmount Thunderbolt 2 Expansion Chassis & RacMac Mini Enclosure
This next solution was recently shared with me from Marc Huppert who has recently expanded his home lab. Marc combined a Thunderbolt expansion chassis with a Mac Mini chassis to exposed Fibre Channel storage to his Mac Minis. You can find more details here.
Solution #3 - xMac Mini Server Enclosure
I came across this solution while searching online which also uses another Mac Mini Thunderbolt expansion chassis connected to Fibre Channel based storage. You can find more details here.
Solution #4 - Sonnet xMac Pro Server Enclosure
Thanks to Joshua for sharing his solution. You can find more details in the comments here.
Solution #5 - LaCie Rugged Thunderbolt drives
Thanks to Philip for sharing his solution. You can find more details in the comments here.
Solution #6 - ARC-8050T2 Thunderbolt 2 RAID
Thanks to Jason for sharing his solution. You can find more details in the comments here.
Solution #7 - Another Sonnet xMac Pro Server Enclosure + EMC VNX
Thanks to Johann for sharing his solution. You can find more details here.
Solution #8 - LaCie Little Big Disk Thunderbolt 2 with 2013 Mac Pro w/ESXi 6.0
Thanks to Thomas for sharing his solution. You can find more details here.
Solution #9 - Sonnet Echo Express III with Mac Pro 6,1 and ATTO ExpressSAS H680 w/ESXi 6.0
Thanks to Grasshopper for sharing details here and here.
Solution #10 - OWC ThunderBay 4 RAID 4-Bay External Drive w/Dual Thunderbolt 2
Thanks to Gregg Green for sharing his use of the Thunderbay with Mac Mini 2012
If there are other Thunderbolt-based storage devices that you or others have had success with ESXi, feel free to leave a comment with details and I will add it to the post. If there are any Thunderbolt storage device vendors that would like to send me a demo unit, I would be more than happy to give the system a test to see if it works with ESXi 🙂
We have a Sonnet Echo Express SE II enclosure with an Atto Celerity FC-82EN FC HBA and an Intel dual-port gigabit Ethernet card. It's connected to a Mac mini (Late 2012) running ESXi 5.5 and it's working great. Our SAN storage system is an HP StorageWorks P2000 G3 FC. It's all working marvellously (including attaching SAN volumes directly to OS X VMs using RDM mapping). The Atto FC card, the Intel NIC and the SAN are all VMware-certified, so the only unofficial parts are the Mac mini and the Sonnet enclosure.
Note that the Atto Thunderlink FC-1082 (and its successors) do not work with ESXi. While hardware-wise it is essentially an FC-82EN with a Thunderbolt bridge chip, to this date Atto does not provide an ESXi driver for the Thunderlink and the FC-82EN driver does not fully recognise it due to differing PCI IDs.
mkuron, what model of Intel dual-port gig Ethernet card are you using? Also, have you had success with your setup and ESXi 6.5? I am considering building a similar setup with a Sonnet Echo Express III-D and ESXi 6.5.
Bob, we've changed our setup a bit in the almost three years since I wrote that comment. At the time, we had an Intel Ethernet Server Adapter I350-T2.
Our Sonnet Echo Express SE II has been moved to a Mac mini running macOS 10.12 directly on the hardware, with the Atto FC-82EN and a Sonnet Presto Gigabit PCIe Pro. Our other Mac mini has had ESXi upgraded to 6.5 in the meantime and has a Sonnet Echo Express SEL containing a Qlogic Sanblade QLE2462 (you can get these pre-owned for less than $20) and an Apple Thunderbolt Ethernet adapter daisy-chained.
William, I assume 🙂 That the Pegaus R4 will work as well?
Thank you,
Jim
awesome! i figured that any thunderbolt expansion chassis would work and i plan on using a sonnet xmac mini but its great to hear that the pegasus works?
we have also used a Akitio Thunder2 PCIe Box with an intel 10 GBE card with great success...
We have recently started building out our Mac VMware "farm" where we are using the latest MacPro6,1's with a Sonnet xMac Pro Server while utilizing the PCIe Bridge to connect Dual QLogic QLE-8440 Converged Network Cards. We have been very successful with utilizing the PCIe Bridge to connect any I/O devices that have been approved and are on the VMware HCL.
I am eager to try the Pegasus R6 though for at home 🙂
Maybe this worked in 2015, but now in 2017 with ESXi 6.5 we have had no luck with this. While our FC card is listed as supported with VMware, it's not with Sonnet and we could never get it to show up in VMware.
http://sonnettech.com/support/charts/thunderbolt/index.html
Very excited by the prospects of what can be done with Thunderbolt devices this year, will be watching this article/comments closely. Thank you for pulling this info together, William!
We've had luck with external Thunderbolt drives like the LaCie Rugged Thunderbolts after installing the V-Front SATA-XAHCI VIBs to support them (https://vibsdepot.v-front.de/wiki/index.php/Sata-xahci), useful on the Mac Pros with ESXi for testing.
The Areca-supplied SAS RAID driver (arcmsr) recognizes the ARC-8050T2 Thunderbolt 2 RAID connected to my Macmini6,2 running ESXi 5.5 U2. Hot-plugging devices on the Thunderbolt bus reliably PSODs the server, but otherwise, I've been using in my home lab for over a month without incident. In case it matters, my drive configuration consists of a single datastore filling a 12TB RAID 6 volume on 8 2TB HGST Ultrastar SATA drives (model HUS724020ALE640).
Note that hot-swapping *drives* within the enclosure works fine, as do related operations such as rebuilding and extending RAID sets via the array's out-of-band management port; PSODs only appear when I try to hot-plug *Thunderbolt* devices.
The Areca driver also recognizes the ARC-8050 Thunderbolt 1 RAID connected to a Mac Mini 6,2 running ESXi 6.0 U2. The driver can be found here; http://www.areca.com.tw/support/s_vmware/vmware.htm If you are new to installing drivers in ESXi, I used the directions found here; http://9to5it.com/install-drivers-esxi-host/
I had never installed any drivers in ESXi before and it was very simple. I copied 1 vib file from the driver package to a local datastore and ran the following command via SSH; esxcli software vib install -v /vmfs/volumes/localstore/folder/driver.vib it installed quickly and after a reboot the Areca Thunderbolt RAID was visible. I'm planning to upgrade to ESXi 6.5 soon and will post the results. I'm assuming it will work fine.
the Pegasus J4 also works out of the box via thunderbolt 1 on a Mac Mini 5,3 on esxi 5.5.1
i can confirm this also works with Mac Mini 6,2 running esxi 6.0.0 u2
also works on a Mac Mini 6,2 running esxi 6.5 (why did Pegasus kill this product?)
Any clue how to get "LaCie 2big Thunderbolt™ 2" to work? Thanks!
https://www.lacie.com/us/products/product.htm?id=10624
Waw! Really interesting! I figured out what I needed! Thanks! Greets, Storage Primrosehill Ltd.
anyone have a lacie 8 big rack to test?
https://www.lacie.com/us/products/product.htm?id=10622
I had one briefly to try out however I didn't have any luck getting it to connect/run with ESXi. It looked like a custom Vendor/Device ID so a new VIB would need to be created once a suitable driver is able to work with it under GNU/Linux.
thanks philip -- always looking for the inexpensive yet fast 1U raid 🙂
Yes, I was hoping to get it running but I ran out of time. If someone (in the world) is able to make it run with GNU/Linux and have the drivers all working it should be possible to compile & create the suitable VIB for ESXi. You probably want to jump to the 12Big with iSCSI instead to avoid direct-attach issues at this point.
I've got a 1TB G-Technology - G-DRIVE mobile working using the V-Front SATA-XAHCI VIBs mentioned in an earlier comment. (https://vibsdepot.v-front.de/wiki/index.php/Sata-xahci) Thank you!
Is there ANY way to spin down a single external Thunderbolt drive? I highly value the silence of my Mac Mini 6,2, but the external drive is ruining this. The drive is mounted in ESXi 6 as a datastore for ghettoVCB backups only. I would love to have it spin down when not in use. I realize this isn't something that is even considered for ESXi, but we're all a fringe case, right? I found a thread that mentioned stopping the smartd process might allow the drive to spin down, but it had no effect. Any suggestions are welcome!
Oran, I have the same drive but can't get it to be seen in the vSphere client. I have ran the above vib on my MacPro (that is how it sees the ssd in the MacPro) but can't seem to get it to work. Can you tell me the steps you took to get the G-Tech to work?
Chris, I installed the VIB as described at http://www.v-front.de/2013/11/how-to-make-your-unsupported-sata-ahci.html, which you've apparently already done. Maybe the drive you have has an updated PCI ID? There is a command on that page that will show you the ID. If it doesn't match what is in the VIB, you can email the author to have it added.
As an aside, I ended up returning that drive and abandoning direct attached Thunderbolt, since I could never get the drive to spiin-down when not in use. My Mac Mini has Firewire, so I passed it through using VMDirectPath I/O to a Windows Server VM that acts as an iSCSI target for vSphere. The end result was a high performance datastore that spins down (Windows Server power management) when not in use. The Thunderbolt port refused to pass through.
If it doesn't work the way you want it to, sometimes you just have to be creative!
Is there anyone out there that has gotten an external SSD to work via thunderbolt?
Hi there!
Any news with ESXi 5.5U2 or 6.0 and LaCie 8big Rack Thunderbolt 2? It could be very interesting for me...
Thanks,
Migo 🙂
Has anyone been able to get a Drobo 5D to work with ESXI. I have a OSX 10.10 VM on a Mac Mini V6,2. i would like to attach the drobo 5d to the OSX 10.10 VM on the ESXI 6 Mac Mini host
Hi do you have a Solution for the Drobo 5d? Same situation for me
No I wish. I can't get it recognize at all.
Hi,
I got a LaCie Little Big Disk Thunderbolt 2 working on a 2013 Mac Pro (ESXi 6.0) using http://www.v-front.de/2013/11/how-to-make-your-unsupported-sata-ahci.html (no RAID however).
stupid question - but with the pegasus - is it just acting as passthrough to the raw disks or are you able to utilize the RAID controller?
Hi,
I need some help on installing ESXi 6.0 on a Mac Pro with a Promise Pegasus2 R4.
The installation only sees the internal SSD, but not the Promise Pegasus2 R4 which is connected via Thunderbolt.
Could it be a problem with the version of ESXi? I see that Rhinofart Jun 6, 2014 1:08 PM (in response to telecastle) made his solution work with a Pegasus R6 Thunderbolt enclosure via a MacMini Server on ESXi 5.1U2, but then didn't work with ESXi 5.5 anymore.
Thanks, Marc
Did you ever get your Promise Pegasus2 R4 working under VMware ESXi 6.x? I'm trying to set up my first ESXi setup and was hoping to use this storage array.
Hey! Awesome blog; I've been doing a lot of research into migration to a virtual environment for our handful of servers.
Currently running a late 2012 i7 Mini in an Sonnet xMac Mini enclosure with an ATTO ExpressSAS R680 attached to a RAID50 and a 10Gb PCIe Ethernet. Currently acts as our Profile Manager, Time Machine server, DNS, and Open Directory.
I can say that 10.10.4 isn't the most friendly with Server. Really wish we could have rolled back to 10.8 (We have a "design raid" thats a Mini running 10.8.x that's rock solid).
Guys i have a question.
I apologize guys and hope you're able to help me out,
I currently have a Mac Mini 6,2 with esxi6 host and would like to inquire in how do i get my OS X 10.10 GUEST to detect my SOFTRAID Raid 5 setup..... I tried Googleing to no success.. Any help....Thanks
The TFC-1600MM supplies Chassis SNMP confirming and also uphold slot based utilization for the TFC-1600 16-Bay Fibre Converter Chassis Unit.
Quick use of cooling fan as well as energy resource condition states. Net browser based control within Ethernet slot even Order Line Software control through RS-232 serial slot.
Have anyone tried connecting two macs to the same thunderbolt storage and see if can be used as a shared storage ?
I'm pretty certain that is not possible. However this device make make it an option but I'm really not sure: https://www.attotech.com/products/thunderbolt-desklink-devices/thunderlink/thunderbolt-2-to-16gb-fibre-channel/TLFC-2162-D00
Some great information on this website - thanks William.
We have quite a number of trash can Mac Pro's which we have maxed out (12 core, 128GB RAM and 2TB onboard SSD's). Our team uses these boxes as highly portable ESXi servers that can taken on a flight as carry-on and quickly configured according to workload required. We have traditionally used a 2 bay NAS for supplementary storage, but recently have had a requirement to use much faster storage which is also light and portable.
After extensive research, we have settled on the Promise Pegasus M4 Thunderbolt unit. This is a 4x 2.5" bay Thunderbolt 2 unit which we have loaded up with 4x Samsung EVO 850 Pro SSD's (2TB).
Performance wise, we are seeing 1200MB/s write and 1400+MB/s read rates with the unit attached to a Windows 2012r2 VM.
Although Promise do not support their Thunderbolt boxes in ESXi, I have established that the unit is correctly detected and performs incredibly well using their SuperTrak EX series driver (version 4.07.0000.75).
Thanks William, the content on this site is very detailed and great with tons of power user tips on ESXi!
I am having issues getting the Pegasus driver mentioned here (SuperTrak EX scsi-stex-4.07.0000.75-1OEM.500.0.0.472560.x86_64.vib) to work with the latest ESXi 6.5. This driver worked great for me (as Paul mentioned) in ESXi 6.0 but when I try to install it in 6.5 it gives the following error:
[DependencyError]
VIB Promise_bootbank_scsi-stex_4.07.0000.75-1OEM.500.0.0.472560 requires com.vmware.driverAPI-9.2.0.0, but the requirement cannot be satisfied within the ImageProfile.
VIB Promise_bootbank_scsi-stex_4.07.0000.75-1OEM.500.0.0.472560 requires vmkapi_2_0_0_0, but the requirement cannot be satisfied within the ImageProfile.
Since the driver is dated 2011 it's not all that surprising the dependencies are growing out of date. I've googled and seen a number of other old .vib files are affected by the same dependency deprecation (HP, Dell, and the like) but not sure if I can find or install the old dependencies or if I'm doing something wrong. Anyone have success getting Thunderbolt Pegasus RAID arrays to work with ESXi 6.5? Paul, you haven't tried to upgrade to 6.5 yet have you?
Yes Todd - we did attempt the upgrade to 6.5 but hit the same issue and to date have not found any resolution. Unfortunately that means we are stuck at 6 and miss out on the other enhancements found in 6.5
Would love to figure this one out - I'm sure there has to be a solution to this - I just haven't been able to find it yet!
I'm in the same boat with ESXi 6.5 and thunderbolt storage. The old driver doesn't seem to be able to install.
hey - stupid question (maybe) but if you did use the pegasus raid on esxi- you would lose access to being able to configure the raid via utility correct? how would you know of issues etc? thx!
Correct - in previous versions of ESX there was a CLI tool that you could apparently use to manage / monitor RAIDS - not sure if it would have worked with the Pegasus line as we never used it.
At the moment, we build the RAIDs using them direct attached to Macs and the Pegasus utility. For monitoring, we can pick up drive failures or unit issues on the box themselves (drive failure - red LED). Not as nice as being able to monitor via ESX.
In theory, you could passthrough the physical device to a windows or Mac VM and manage that way but not something we have every tried or needed to do.
right - thats what i would have thought - but if you passthrough you cant use for native datastore - so defeats the purpose. thx for confirming! 🙂
Has anybody tried to do this without a Mac, I mean with standard server hardware like our Supermicro server and a Thunderbolt controller card? We would like to connect our Pegasus Raid to out Supermicro ESXi server. Any suggestions for such a Thunderbolt controller card?
Just responding here, to update the post.
I've been able test the following thunderbolt 3 devices and ESXi 7.0 u3 recognizes the SATA controllers properly, and works with passthrough if you VMkernel.Boot.disableACSCheck to True in your advanced settings. I ended up testing both because I realized that they contain different SATA controllers, (Asmedia / Jmicron respectively) and neither are listed on the VMware HCL.
OWC Thunderbay 4 Thunderbolt 3 (yes that's the product name)
OWC Thunderbay 8
Hope this helps someone in the future!
@Adrien, hello! I have a question. My interpretation of your message is that there are two functional scenarios. 1) ESXi host provisioning SATA drives of the thunderbolt device, and 2) pass-through. Is that correct?
My understanding of scenario 2 is that an OWC Thunderbay enclosure would present to a guest operating system for exclusive contention by that guest. For scenario 1, would the individual SATA interfaces then be divisible up across multiple guest machines? I presume that, for example, this would mean that each of the 8 slots in a ThunderBay 8 could be assigned to a different guest operating system (i.e. somehow either individually or in aggregate)
Any thoughts would be appreciated