I know many of you have been anxiously waiting for ESXi to be fully functional on the latest Apple Mac Mini (2018), unfortunately I do not have any news to share with you on that front. Without help from Apple, we are still challenged with Apple's new T2 chip, which prevents us from accessing the underlying NVMe device.
UPDATE (04/27/20) - Marvell (formally Aquantia) has just released an official Native ESXi Driver for their AQtion based network adaptors which you can find here and for the complete list of supported devices using this driver, please refer to the VMware HCL here.
Having said that, I do have some exciting news regarding the built-in 10Gigabit Ethernet adaptor on the 2018 Mac Mini. The 10GbE adaptor uses an Aquantia chipset, this is also the same chipset used in Apple's high end iMac Pro which was released earlier in the year. Over the past few weeks, I have been working with the Aquantia team and they have successfully ported over their open source Atlantic driver to a VMKlinux driver for ESXi, which they have published here. Although the local NVMe device can not be used to run any VMs, with the network fully enabled, customers could take advantage of this model and connect to IP-based storage to start leveraging the new Mac Mini platform.
The easiest way to incorporate the driver into the latest ESXi release is to use Image Builder within the vSphere H5 Client UI, below are the step-by-step instructions.
Step 1 - Login to vCenter Server and vavigate to the Auto Deploy plugin and enable the Image Build service if you have not already.
Step 2 - Import the Aquantia and the respective ESXi Offline Bundle zip file into Image Builder.
Step 3 - Create a new Custom Software Depot by clicking on the "New" option on the right hand corner, provide a name and this will be used to hold our new Image Profile.
Step 4 - Select the newly created Custom Software Depot and create a new Image Profile by providing a name and vendor label.
Step 5 - Make sure the Acceptance Level is configured as "Community supported" as the Aquantia driver has been created using the VIB Author Fling. Filter by the depot and select "net-atlantic" to include the NIC driver and then specify the ESXi Depot to include the ESXi Image.
Step 6 - To create our custom ESXi ISO, select the Image Profile and then click Export and then click next to begin. If you are using ESXi 6.7 Update 1, you should have a total of 142 packages including the Aquantia Driver. Once the export has completed, you can click on the Download button to retrieve the ISO and then you can use your favorite USB-imaging tool to create your ESXi installer, I use Unetbootin for this.
Here is a screenshot of ESXi 6.7 Update 1 running on the 2018 Mac Mini model w/10GbE built-in NIC 😀
I also had a chance to run a quick iPerf test (ESXi includes iPerf3) between my Macbook Pro (connected to an external 10GbE NIC) and the Mac Mini and below, you can find the results using 20 parallel threads:
| Direction | Summary |
|---|---|
| TX | [SUM] 0.00-10.00 sec 9.37 GBytes 8.05 Gbits/sec sender |
| RX | [SUM] 0.00-10.02 sec 10.2 GBytes 8.73 Gbits/sec receiver |
Note: One issue that I did run into, which is being investigated, was that I was not able to get a link up while connecting to my home network which is 1GbE-based with an Ubiquiti AmpliFi. When booting into MacOS, the link is immediately picked up, so I do not believe this is related to my physical network and Aquantia had also confirmed a similar behavior. A workaround, as I do not have 10GbE network infrastructure at home is to set the interface to 100mbps (yup, you read that right). To do so, simply login to ESXi Shell and run the following command: esxcfg-nics -s 100 -d full vmnic0 and the link should automatically be enabled after you make this change. I suspect this is not an issue if you are connected to a 10GbE switch, but if you are running into this, feel free to drop a note about your setup so we can better understand the issue.
UPDATE (04/14/19) - Thanks to Mike Roy, I can also confirm ESXi can now be installed on the Apple iMac Pro which uses the exact same Aquantia chipset. Please be aware, the same constraints for the 2018 Apple Mac Mini also applies to the iMac Pro, please see this blog post here for more details.
I want to give a huge thanks to the Aquantia Engineering teams, they have been fantastic to work with and big thanks to Francis Augusto Medeiros-Logeay who helped connect us up. I look forward to seeing how customers will be leveraging the new 10GbE Mac Minis given the current constraints. If you have any feedback, feel free to leave a comment here or directly on Aquantia's Github repo.
Isn't the vmklinux api deprecated and due to be removed in the next numbered release of vsphere?
Hi Nathan,
Yes, VMKlinux has been deprecated and will be completely removed in the next major vSphere release (https://blogs.vmware.com/vsphere/2019/04/what-is-the-impact-of-the-vmklinux-driver-stack-depreciation.html). We'd love for Aquantia to also produce a Native Driver, however they are currently not part of VMware's TAP program which provides access to the SDK to build such a driver. We're hoping they'll be joining the program soon, so we can ensure this driver will continue to work when we remove VMKlinux support.
Thank you William! Pleasure working with the VMWare team
Great work William and Aquantia Team.
Now all we need is the Local Disk to work. Please work with Apple on this.
Hi Steve,
Thanks for the note, Aquantia team did most of the hard work 🙂 We'd love to get the NVMe device enabled on the Mac Mini as well, but without Apple's participation, it's going to be nearly impossible.
MacStadium team is grateful for all the work you do, William! Thanks for the continued push to get VMware working better and better on Apple's most recent hardware. We'll get moving to test this in our labs ASAP.
@Aquantia team -- happy to help out however we can! Our customers are excited to run VMware clouds on Mac minis.
These guides have been amazing. I can get my mac mini's to boot ESXI 6.7 update 2 with the Aquantia drivers but I cannot get them to connect to VCenter ( I get odd "Failed to read request" errors in the ESXi logs ) so they just timeout and can't be added as actual hosts. I haven't seen any examples of anyone running them in a Vcenter installation, curious if anyone has gotten that far.
For Anyone finding this today and running into the issue, as Max alluded to there was a strange timeout occuring when trying to pull the Mac Mini's into vcenter (also when loading the esxi web UI on Windows). We managed to get the hosts added properly to Vcenter and the windows Web UI started working using the Aquantia 1.6 drivers over the 1.4, I believe the patch in 1.5 which included "fix for a lot of TCP retransmissions caused cutting padding of TCP ACK packets" solved the issue. If you have a host running 1.4 you can scp the VIB onto the box and run the update with esxcli and it will work on Windows WebUI and vcenter.
I'm a bit late to the party but I picked up an OWC 10G TB3 adapter and tried both the OEM and Github versions of the driver. Unfortunately both still have the same issues (retransmission errors) and cause all sorts of network problems.
Above all 6 steps are really working in my system that's why this blog post is helpful for me.
After 3-4 hour research, got the tool which can easily update the all windows driver with the newest driver. This driver updater tool is very small in file size but can be updated every windows driver.
https://blogs.innovanathinklabs.com/drivers/16-best-free-driver-updater-software-for-windows-10-8-7/
I was surprised to find that aquantia nics are still not supported even in 7.0
The link for the Marvell driver seems broken, is it possible to get that somewhere?
Thanks,
-Chris
Link works for me. Please check your connection or try incognito/another machine
Hey William!
I got a Sonnet Solo10G for my 2 NUC10i7FNK2's (running ESXi 7.0) and I installed the recommended ESXi driver but the NIC isn't getting recognized. I don't know what to do and these NICs were kind of expensive, I'm out of ideas.
Can you give me some pointers? Is there something else that I need to do?
Thanks!
Hey Guys,
same problem here with Sonnet Solo10G on a a Mac Pro 2013. NICs not getting recognized. Unfortunately VMware homepage seams to be in maintenance mode, so I can't download the recommended driver.
Cheers
Let me edit my last Comment 🙂
I successfully installed this driver:
https://github.com/Aquantia/AQtion-esxi/releases/download/v2.2.3.0-esxi1.6/net670-atlantic-v2.2.3.0-esxi1.6.x86_x64-offline-bundle.zip
After a restart the NIC gets recognized with 10Gbit uplink to the switch.
Oh, and it only works for ESXi 6.7, in 7.0 the dependency of vmkapi_2_3_0_0 is not satisfied.
Just tried to upgrade from 6.7.0 U3 to 7.0 with the driver installed and it failed:
"VIB Aquantia_bootbank_net-atlantic_v2.2.3.0-esxi1.6 requires vmkapi_2_3_0_0, but the requirement cannot be satisfied within the ImageProfile."
Hi Phil,
Please use the official Marvel driver which supports both 6.7/7.0, I'm able to see the download now https://my.vmware.com/en/web/vmware/downloads/details?downloadGroup=DT-ESXI67-MARVELL-ATLANTIC-1020&productId=742
how to get the 10gb driver ti install iso image, vcenter server not installed for now? would like to have a 7.0b ready to install image, may someone can help?
Image Builder (UI/PowerCLI) requires vCenter Server, you can always deploy a VC really quickly to author the image as that's the "easiest" method and it doesn't have to be licensed to use this feature. If you really can't deploy VC in your env, you can look to community tools like PS-Customizer, but this is not something we test, so YMMV
The file can be downloaded, but the first step of importing failed. I had to go back to the AQtion-esxi github site and download that 2.2.3.0 version, but not sure it will works or not for ESXi7.0 version.
And the error is something like "There is no item named \'index.xml\' in the archive"
One of our research expert find the best software list which can update all the driver software. And also help to update device driver automatically. https://www.techpout.com/best-driver-updater-software-for-windows/
Hi William-
I posted in the Mac Pro 2019 update area too, but my only issue today is related to the Aquantia nics and there were clues here I missed. I am definitely having the problem where they don't connect on my home gigabit Netgear unmanaged switch, but I think they don't work at all in ESXi 7.x.
I am using the aquantia drivers with ESXi 7.0c; if I understand correctly however, that can't work based on the discussion about the TAP program. LOL I had trouble getting us into the TAP program at my last company so they may be facing similar issues.
So, my guess is that the aquantia drivers CANT work in ESXi 7? Please confirm.
When I check esx.conf, I can see that the pnic mac addresses are incorrect. I tried to change them of course that didn't persist across reboots (a little bit of work https://www.williamlam.com/2011/08/how-to-persist-configuration-changes-in.html left to do about that).
I also attempted to change them to 100Gbit based on your info (esxcfg-nics -s 100 -d full vmnic0) and this article https://kb.vmware.com/s/article/1004089 but I was getting an error message from your command at first. It executes now without error but it doesn't result in a connection. I have tried editing esx.conf and restarting the management network but esxcli-nics -l continues to show they have the wrong mac address.
My guess is I need to go back to 6.7 and navigate the waters of the PSOD.
I had these same issues - root cause was how I built the custom image
Try to follow these steps:
1. download ESXi 6.7u3 offline bundle (https://my.vmware.com/en/web/vmware/downloads/details?downloadGroup=ESXI67U3&productId=742&rPId=60244) & Atlantic driver (VMware certified) from VMware website (https://my.vmware.com/en/web/vmware/downloads/details?downloadGroup=DT-ESXI67-MARVELL-ATLANTIC-1020&productId=742)
2. Import both ESXi 6.7u3 offline bundle and VMware certified Atlantic driver to VMware image builder
3. Create a new custom software depot
4. Navigate to the offline bundle you imported to the image builder, select this specific image "ESXi-6.7.0-20190802001-standard" and press "CLONE" ( - that was the part I was struggling with, using this is key and makes the process very easy!!! )
5. In the "Clone image profile" : provide a new name for the image, select the "software depot" (created is step #3), add the Atlantic driver (all the rest of the ESXi drivers should be pre selected by the system) and finish the process (144 files in total)
Note: since Atlantic driver is VMware certified it doesn't matter which acceptance level you choose.
6. Navigate to Software Depot created in #3
7. Export & download the image as ISO
8. Run Rufus (v3.13) to create a bootable image
After running these steps I was able to build a the custom image without warnings, install ESXi without PSOD and sync the MacPro NICs in 1G.
Don't forget to run patch ESXi670-202004002 (https://docs.vmware.com/en/VMware-vSphere/6.7/rn/esxi670-202004002.html) in order to run VMs on the MacPro 🙂
I would like to thank William for sharing his knowledge and support.
Using the file: nvme-community-driver_1.0.1.0-1vmw.670.0.0.8169922-offline_bundle-17658145.zip
with ESXi670-202004002.zip containing version 6.7.0 Update 3 (Build 16075168).
I'm also using the NVMe driver, and the disk has been stable running multiple VM's. But the network link keeps going up and down on the Cisco 3560 switch:
Mar 16 22:46:52: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to up
Mar 16 22:47:47: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to down
Mar 16 22:47:48: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to down
Mar 16 22:47:51: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to up
Mar 16 22:47:52: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to up
Mar 16 22:53:33: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to down
Mar 16 22:53:34: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to down
Mar 16 22:53:37: %LINK-3-UPDOWN: Interface GigabitEthernet0/9, changed state to up
Mar 16 22:53:38: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/9, changed state to up
I don't see many errors on cisco:
d20core-cx#show int g0/9 | include error
23 input errors, 13 CRC, 0 frame, 0 overrun, 0 ignored
0 output errors, 0 collisions, 3 interface resets
I see no errors on esxi:
# esxcli network nic stats get -n vmnic0
NIC statistics for vmnic0
Packets received: 274883358515
Packets sent: 227637018516
Bytes received: 411813364031
Bytes sent: 227921939990
Receive packets dropped: 0
Transmit packets dropped: 0
Multicast packets received: 111669153538
Broadcast packets received: 68719477460
Multicast packets sent: 4294967296
Broadcast packets sent: 111669149717
Total receive errors: 0
Receive length errors: 0
Receive over errors: 0
Receive CRC errors: 0
Receive frame errors: 0
Receive FIFO errors: 0
Receive missed errors: 0
Total transmit errors: 0
Transmit aborted errors: 0
Transmit carrier errors: 0
Transmit FIFO errors: 0
Transmit heartbeat errors: 0
Transmit window errors: 0
Any ideas?
I'm running into a similar issue mentioned above:
"Note: One issue that I did run into, which is being investigated, was that I was not able to get a link up while connecting to my home network which is 1GbE-based with an Ubiquiti AmpliFi."
I am able to get a link but it's painfully slow. The 10gb adapter is connected to an unmanaged 1gb switch. I tried disabling TSO and LRO but throughput did not change. I'm using latest 7.0U3d release on a 2018 Mac Mini with the 10gb adapter.
Old thread here but worth seeing if anyone else has upgraded their mac pro 2019 with any releases past 6.7 u3 + 202004002 patch?
I tried with the latest 6.7 build
ESXi 6.7 EP 23 ESXi670-202201001 2022/01/25 19195723
as an esxcli based upgrade but it broke network.
Then I booted from the 19195723 usb based installer (built same as above system) and attempted fresh install (preserve VMFS) and that did also fail to connect.
Then I reinstalled the original 6.7 u3 + 202004002 patch. It worked perfectly.
I then booted from the latest 6.7 build on usb, and said to "upgrade"; unfortunately the upgrade failed in the middle. But I sense the upgrade does not work.
----
The behavior is the same for newer 6.7 builds as it is for 7.x builds:
-Network drivers are detected, and show as connected.
-No other equipment can see the Nics
-Both Nics get a network mac address starting with "cc:2d:b7"; however, when I boot to mac or windows on boot camp these cards have a mac address starting with "E4:50:EB". I find this really strange;
BUT the "cc:2d:b7" based mac addresses DO work on the 6.7 U3 build.
So something changed that broke these drivers in mac pro 2019 in later 6.7 builds; I can't speak for the newer mac minis and how those are working.
I know this is a long shot but happy to provide 1 or 2 machines if that will help with debugging.
Does anyone have a copy of the drivers. The vmware support page is down and the broadcom support page will not be up until 5/6/2024. Thanks