With the help from Aquantia, we now have an ESXi driver to enable the built-in 10GbE adaptor for both the Apple 2018 Mac Mini and the iMac Pro. Although this was exciting news for our VMware/Apple customers, I was actually more excited for what this development meant for the larger VMware Community when it comes to 10GbE accessibility.
Many Enterprise customers have already been using Thunderbolt 2/3 to access their 10 Gigabit infrastructure, usually over SFP+ but Ethernet based options also exists such as the Sonnet solution which I had shared last year. This is especially common for VMware customers who virtualize Apple MacOS on vSphere for MacOS/iOS development and the use of Thunderbolt enables ESXi to connect to the underlying storage and networking infrastructure, which traditionally has been either Fibre Channel and/or IP-based storage running over a 10Gig link.
When you start looking at 10GbE accessibility for VMware home labs which could potentially apply to remote office/branch office (ROBO) and Edge/IoT environments, the cost and the complexity of the setup is something that many folks have to consider. There are definitely some creative options out there, most recently Chad Moon shared his solution using a Thunderbolt 3 to PCIe expansion chassis with his Intel NUCs which will run you about $230 per setup or you can be a true hardware hacker like Jack Harvest and use one of the M.2 slots in the Intel Skull Canyon NUC and connect that to PCIe 10GbE SFP+ card with a custom 3D printed chassis to hide everything for just $43.68 🙂
However, I am guessing that most folks including myself, probably prefer a more plug-in-play option that just works out of the box. This is why I was excited for the Aquantia development because there are a number of Thunderbolt 3 (TB3) to 10GbE adapters on the market that can now be enabled with the new ESXi driver. These devices are pretty small and best of all, it enables you to significantly expand your networking capabilities if you have a system with a TB3 port such as an 7th/8th Gen Intel NUC or Skull Canyon NUC or the Hades Canyon NUC which actually has two TB3 ports. For what you get, this is relatively inexpensive, especially if you are setting up a 2-Node vSAN Cluster with Direct Connect!
UPDATE (04/27/20) - Marvell (formally Aquantia) has just released an official Native ESXi Driver for their AQtion based network adaptors which you can find here and for the complete list of supported devices using this driver, please refer to the VMware HCL here. This driver is supported for both ESXi 6.7 and 7.x
I was very fortunate to have been able to get my hands on several of these TB3 to 10GbE devices for testing purposes, a huge thanks goes out to OWC for the Akitio T3-10G and Dirk Hohndel for the OWC TB3 to 10GbE device.
Once you install the Aquantia ESXi Driver which can be downloaded from here, you can plug in the device into a Thunderbolt 3 port and ESXi should automatically detect the network adaptor.
Here is a screenshot of the latest release of ESXi 6.7 Update 2 running on an Intel Hades Canyon NUC with both the Akitio and OWC TB3 to 10GbE
Below is a table of all known TB3 to 10GbE devices which will work with the Aquantia ESXi Driver, basically any PCIe device using the AQC 107 chipset will be supported.
Vendor | Model | Interface | Cost | Confirmed with ESXi |
---|---|---|---|---|
OWC | Thunderbolt 3 10G Ethernet Adapter | RJ45 | $149 | Confirmed |
Akitio | T3-10G | RJ45 | $199 | Confirmed |
Sonnet | Sonnet Solo 10G | RJ45 | $179 | Confirmed |
Sonnet | Sonnet Thunderbolt AVB Adapter | RJ45 | $149 | Confirmed |
QNAP | QNA-T310G1T | RJ45 | $151 | Confirmed |
QNAP | QNA-T310G1S | SFP+ | $176 | Confirmed here |
CalDigit | Connect 10G | RJ45 | $149 | Confirmed here |
Note: I am still waiting on the arrival of the last two devices, but I should be able to confirm their functionality once they arrive.
UPDATE (04/20/19) - I recently had to setup a new Intel Hades Canyon NUC for testing additional TB3 to 10GbE adaptors as I needed to return the previous unit. I ran into several issues and with the help from some Intel folks, I was able to update the system to the latest Thunderbolt 3 Firmware. Below are the steps for the Hades Canyon, but they should also be applicable for other Intel NUCs with a Thunderbolt 3 port.
Step 1 - Go into the BIOS (F2 during boot up) and make sure BOTH the Thunderbolt Controller and Thunderbolt Boot settings is checked as shown in the screenshot below. These settings are required to use the TB3 ports. Also, if you are using ESXi with the Aquantia Driver, you will need to disable Secure Boot or the system will not boot up ESXi.
Step 2 - Install Windows 10 on the NUC as the firmware utility is only supported on Windows.
Step 3 - Download and install the "legacy" Thunderbolt 3 driver from here. I initially had used the one found on the Hades Canyon download page from Intel, which links to the DCH version pulled down from the Microsoft Store. It turns out, if you are using that version, when trying to run the firmware utility, you will get the following error:
Error: error 0x208 sdk_service_not_found thunderbolt(tm) service doesn't exist
Step 4 - Download and install the Thunderbolt 3 Firmware Utility from here and then flash the device. You may run into an issue if you have something plugged in, so remove any devices and then perform the update operation. Once the system has been updated, you can install ESXi and any of the TB3 to 10GbE adaptors should now automatically be picked up. Also, its a good practice to update to the latest BIOS update, as of writing this, the current BIOS for Hades Canyon is v54.
Lastly, Aquantia also makes a standard PCIe to 10GbE card for $89 which can be added to any white box setup that has a free PCIe slot. Thanks to Stanislav for sharing the good news on getting the card working with his ESXi 6.5 setup.
just tested the driver with PCIe NIC from Aquantia for PC AQC-107 and ESXi 6.5.0 detect this NIC and also detect option of 10Gbit. We will see if I am able to transfer 10gig/s.... once I have other 10gig end 🙂 pic.twitter.com/7QitP6KHuS
— Stanislav Musil (@stmusil) April 14, 2019
In my opinion, this space is only going to get more interesting and with USB 4.0 just around the corner which converges on the Thunderbolt 3 interface, I think it will be exciting to see how connectivity evolves for servers running both within the datacenter as well as on the Edge!
Do you (or anyone else reading) happen to know which chipset is in the Monoprice adaptor?
Have you tested any of the cards on Mohave by any chance? Tempted but see complaints online of sleep/wake crashes.
Probably easiest if you ask Monoprice 🙂 ... but given the OWC is the cheapest and its confirmed to work, why not just grab that?
Found the answer about the Monoprice adapter--it uses a Tehuti TN9710P (it looks to be a white-labeled StarTech TB310G). You can get the driver direct from Tehuti, look for the tn-40xx VIB. StarTech also has them but not listed for the TB3 NIC, you have to look for the PCIe version (PEX10000SFP). Drivers are unsigned so you'll have to bypass signature checking and set acceptance level to CommunitySupported.
It is never late to say THANK YOU! I had this monoprice (bought used) parked for months. Cursed the seller, Ebay. monoprice and Darwin for my frustration...and then I Found this post...LOL
Sweet! Nice to see some elegant solutions coming out -- I, too, am very excited for the non-Intel-taxed TB3 (aka, USB 4).
Two minor corrections:
- The 3D printed 10Gbe expansion ring is for the *skull* canyon, not the Hades Canyon (although, if I can get Intel to donate one to me, maybe I'll make a Hades Canyon expansion ring too).
- The article mentions "Skull/Hades Canyon NUC which has two TB3 ports", which is true to form for the Hades Canyon - but alas, the Skull Canyon is equipped with only a single TB3 port.
Good read. I love reading this stuff.
Thanks for the corrections, fixed!
But not compatible with QNA-T310G1S (Chipset AQC100) & Skull Canyon NUC
Tested yesterday 🙁
I'll ask the obvious (because I really wanted that to work); Did you enable Thunderbolt within the BIOS of the Skull Canyon? (Does it even light up the Eth port?)
enable T3 on bios : of course yes 🙂
I can see it on "lspci" :
0000:02:00.0 Bridge PCI bridge: Intel Corporation DSL6340 Thunderbolt 3 Bridge [Alpine Ridge 2C 2015]
Class 0604: 8086:1576
And no light on the SFP+ port 🙁 (FYI, QNA-T310G1S is not an Eth version but a SFP+ port)
I have tried the 2 vib files :
AquantiaAQN107-2.0.22.017.x86_64.vib & net670-atlantic-v2.2.3.0-esxi1.4.x86_x64.vib :
esxcli software vib list | grep -i aqua
AquantiaAQN107 2.0.22.017 Aquantia CommunitySupported 2019-04-19
net-atlantic v2.2.3.0-esxi1.4 Aquantia CommunitySupported 2019-04-19
Nothing on the "esxcli system module list"
Even if I tried to "--enabled" the module
So, I'm waiting & hoping a new vib file 🙂
Did you insert a module or direct attach cable into the SFP+ slot? Perhaps the driver does not work properly without a PHY?
I have tried the both solution DAC & SFP+ RJ45 module : no light on the QNA-T310G1S.
But when I tried on my other NUC with Windows all is working fine.
It does work with the AQC100 with the drivers released in May.
net670-atlantic-v2.2.3.0-esxi1.6.x86_x64.vib
dbezrukovaquantia released this on May 28
I picked up QNAP QNA-T310G1S connected via 10Gtek DAC to my UBNT ES-16-XG and it's connected at 10Gb.
Got it running on 2 NUC 8i7BEH
NUC10i7FNH does not recognize AQC100 as PCI device in ESXI 6.7. So, both "atlantic" and "aquantia" not loading. "thunderbolt3 boot" option is not present in BIOS. TB3 firmware and BIOS - the latest. Windows works fine.
NUC10i7FNH successfully recognized AQC100 after firmware upgrade of AQC100 from 3.1.53 to 3.1.109. lspci -v for it:
0000:06:00.0 Network controller Ethernet controller: Marvell Felicity Network Adapter [vmnic1]
Class 0200: 1d6a:80b1
You have to allow "thunderbolt3 boot" in bios
Also all wen't wrong with 6.7 U2. I can't no longer use it for NFS storage
I'm trying to update my NUC7i7BNH and intel left the .bin file out of the firmware update tool package (https://downloadcenter.intel.com/download/27679/Thunderbolt-3-Controller-Firmware-Update-Tool-for-Intel-NUC-Kit-NUC7i5BN-NUC7i7BN?product=95065). Anyone have that file? I've contacted intel but they are soooo slow to get it to me..
Hi, Mike, that .zip contains FW v33 .bin inside
I have the PCIE Sonnet Presto 10G working in ESX 6.5 now. I used the net670-atlantic-v2.2.3.0-esxi1.5.x86_x64.vib
I have noticed that VLAN support doesn't work. Network works fine on the native vlan, but wont when it is being tagged. Anyone else notice this?
ESXITROY, the latest net670-atlantic-v2.2.3.0-esxi1.6 driver supports for VLANs.
will the aqc-107 support 802.1q trunking in ESXi 6.7 with this driver?
Any idea if this is supposed to work with NUC8i7BEH ? If i have windows 10 installed, the 10G OWC is discovered. When running ESXi no luck. Also i could not find any firmware update package for TH3 on intel website for NUC8i7beh.
Has anyone maybe more luck with this on NUC8*beh series?
Ok, ignore my question... Although there was no firmware driver update for TH3 for nuc8*Beh, it looks to work , after installing latest driver from https://github.com/Aquantia/AQtion-esxi/releases
I have successfully added the following NIC to my NUC8i7BEH:
https://www.amazon.com/CalDigit-Connect-10G-Thunderbolt-Ethernet/dp/B07RH7VPDF/ref=olp_product_details?_encoding=UTF8&me=
Thank you!
Thanks for sharing Brian! I've updated the page to include this adapter
I have purchased a Sonnet SOLO 10G - installed it into Intel NUC7i7BNH Core i 7
Have installed the vib net670-atlantic-v2.2.3.0-esxi1.6.x86_x64.vib
The adapter is recognised as VMNIC1 with the speed set to 1000gb as currently I have it plugged into 1GB netgear switch.
I can ping the adapter locally on the ESXi server - but I cannot ping anything else - it just errors...
Anyone any ideas please ...
I'm just testing if this will work before purchasing more adapters ands a 10GB switch.
Help appreciated.
Sorry - Yessthe Thunderbolt stuff is switched on in the BIOS and both the ports light up green and yellow. As i said I cannot ping anything connected through the 1GB switch ...
This adapters give nog problems with iSCSI ? Thanks for the nice paper :>)
Is the Akitio previous generation do also work, the Thunder2 to 10g ? And i guess it will be with the git driver, or those are only for the thunder 3.0 ..
thanks
I got the Monoprice Thunderbolt 3 10G Ethernet adapter working this evening. It does not use the Aquantia driver--internally it appears to be identical to the StarTech TB310G, which is using a Tehuti TN9710x-TB3 board. That board has a TN4010 NIC, a Marvell PHY for 10GBASE-T connectivity, and an Alpine Ridge TB3 controller. Tehuti publishes a driver VIB for the TN-40xx series NICs on their website, and StarTech has it available for download as well. It's unsigned, so I had to bypass signature checking and set the host to CommunitySupported, but I have it confirmed as working on an I7HVK system.
Thanks for sharing the details Jack!
Has this been tested with older NUC's like the Intel BOXNUC7I7BNH Core i7 7th?
Yes, without issues for more than year 🙂
Hi, thank you for all your work on this. Ive been using it with 6.7 for over a year. Now im looking to upgrade to 7.0 and wondered if you are going to support this version too please? Thank you.
No. Since the aquantia is vmklinux port which is no longer supported on vsphere.
I am dealing with the same issue now.
Probably will have to go to different solution 🙁
We have been working with Aquantia/Marvell to have them build a Native Driver for ESXi, they are currently going through the certification process and we hope to have some news to share on that front. Please stay tuned
are you serious? Is there any more info? I just bought pcie thunderbolt 3 enclosure and ordered intel sfp+ card.
Should I cancel it? 🙂
What do you think is the timeframe?
Thank you very much for bringing hope 🙂
Cheers
Unfortunately, I don't have any timelines but just know its being worked on as we speak
Do you know if someone can became tester for that? I and move my load to different server and have one with ACQ107 ready.
Thanks
Happy to share this morning that Marvell (formally Aquantia) has just released their Native Driver for ESXi
DL: https://my.vmware.com/en/web/vmware/details?downloadGroup=DT-ESXI67-MARVELL-ATLANTIC-1020&productId=742
VCG: https://www.vmware.com/resources/compatibility/search.php?deviceCategory=io&details=1&keyword=atlantic&page=1&display_interval=10&sortColumn=Partner&sortOrder=Asc
This should apply to both 6.7/7.0 as there's no changes
Thank you William!!!
Will test it once I unpack after moving!
Hi there, any chance Jan you got it running ? After upgrade to esxi 7.0 15843807, its not not working for me anymore. i got nuc8i7beh. I tried the driver from https://my.vmware.com/en/web/vmware/details?downloadGroup=DT-ESXI67-MARVELL-ATLANTIC-1020&productId=742 Installed, but the card is not getting detected.
Ehh.. again 😉 sorry for making confusing. LAter on i read someone's comment, that during the bios f/w upgrade, the TH3 setting can changed. That's what also happened for my nucs. One has to go back to bios and check the boot settings for TH3, they got disabled after bios upgrade. I can report that on latest esxi 7.0 nuc8i7beh , nuc8i5beh everything for for the OWC TH3 10gbit card.
Did anyone test the native Marvell Atlantic driver with Jumbo Frames (MTU > 1500bytes)?
I have TCP connectivity issues when changing the MTU to a value greater than 1500bytes.
ICMP is working just fine.
Adapter: Sonnet Solo 10G
Great work. Thank you.
Has anyone tested the Thunderbolt 3 driver from Aquantia on the new ESXI 7?
vmklinux drivers are no longer supported in vSphere 7 and doesn't even install/function. Only Native Drivers will be supported. With that said, we've been working with Aquantia/Marvell to have an official Native Driver, so once that's available, you will be able to use that going forward
Thanks for the quick response.
I will hold doing the upgrade until I some good news in this forum
Happy to share this morning that Marvell (formally Aquantia) has just released their Native Driver for ESXi
DL: https://my.vmware.com/en/web/vmware/details?downloadGroup=DT-ESXI67-MARVELL-ATLANTIC-1020&productId=742
VCG: https://www.vmware.com/resources/compatibility/search.php?deviceCategory=io&details=1&keyword=atlantic&page=1&display_interval=10&sortColumn=Partner&sortOrder=Asc
This should apply to both 6.7/7.0 as there's no changes
Thanks for the great news!! just installed the vid and it is working as expected
I'm currently experiencing an odd issue with ESXi with my Intel NUC Hades Canyon (NUC8I7HVK). ESXi 6.7 fails to list the Promise SANLink2 10GbE Intel X540 (ixgben) Network Ports via a Thunderbolt3 to Thunderbolt2 Apple Adapter; however, when I use the same HW configuration (with the TB3-to-TB2 adapter) on Windows 10, the Intel X540 network ports are properly detected + configurable.
https://communities.vmware.com/message/2943957
This same device works on TB2 or TB1 hosts with the native driver on ESXi as the Intel X540 AT2 is part of the HCL. In addition, I've used this device on multiple ESXi deployments and they works great. I know the TB3-to-TB2 is the culprit but don't understand why. In addition, I have an Apple Thunderbolt to Gigabit Ethernet Adapter (Broadcom Corporation NetXtreme BCM57762) connected via the same Thunderbolt chain and the network port get detected with the tg3 inbox driver. It appears that ESXi doesn't know how to address the Intel X540 with the ixgben driver when the TB3-to-TB2 adapter is the intermediary between the host and card.
Anyone have any ideas why ESXi fails to list the Intel X540 ports as available configurable ports when using a TB3-to-TB2 adapter on ESXi? Again, the same HW configuration works fine on Windows 10 while using an Apple TB3-to-TB2 adapter on the Intel NUC Hades Canyon (NUC8I7HVK). Thanks!
Also, before throwing in the towel and going with the TB3 options above, I need a dual 10GbE so I can MPIO to my storage via iSCSI.
I know Sonnet has one:
https://www.sonnettech.com/product/twin10g-tb3/overview.html
Anyone tried using this model by any chance with ESXi 6.7? I know the Sonnet Solo 10G works but just thought I'd ask to reconfirm before going with a TB3 option.
I have tried on my NUC Skull Canyon the latest driver as published above but still no luck with the Sonnet Solo 10GB adapter which seems to have a AQC107S. I used the vib and installation said successfully applied. Any ideas?
Have you removed the old vib? What about restarting the host?
Same here for a NUC10i7FNK2 and Solo10G. Can't get it working with the latest vib under esxi 7.0. It just won't see the NIC no matter what I do.
Mine was working fine with and suddenly it stopped working. I am running 7.0.0 (Build 16324942) with MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM.670.0.0.8169922-offline-bundle-16081713.zip
QNA-T310G1S
Check Thunderbolt 3 security settings in the BIOS. Should be Basic/Legacy.
After upgrading my BIOS, suddenly the Sonnet Solo wasn't detected any more.
Apparently the default for my NUC8i7 was "Not basic"
Hi all, I've been reading these pages for some time while building my little NUC-based datacenter. The information regarding 10 Gb Thunderbolt NICs has been invaluable. My 'core' switch is a cheap Mikrotik CRS309-1G-8S+IN unit which I chose for both price and the lack of a fan. The only affordable Thunderbolt SFP+ NIC I found is the Qnap QNA-T310G1S. The old driver apparently has some issues - under ESXi 6.7 the NIC was horribly slow. Even though the link was established at 10 Gb I got transfer speeds below 1 Gb. Apple's TB2 - Gb NIC was faster. The new native driver OTOH is something else entirely. Like 10 Gb 🙂 I've upgraded to vSphere 7, works just as well.
The QNA-T310G1S is a nice piece of kit (with the native driver) so I purchased a QNA-T310G1T for a NUC that's in another part of the house. It appears to work as well as its SFP+ bretheren, stable 10 Gb over a short Cat5e patch cable is no problem. The noise however... You simply cannot use the NBASE-T version in an office environment. It has an incredibly noise fan which kicks in after a few minutes and never stops, whether the card is under load or not. It's nearly as loud as my NUC10i7's when they're running at full throttle. I'm going to keep the QNA-T310G1S's but unless Qnap can find a solution the QNA-T310G1T is going away. Behavior is the same under Windows 10 as it is under ESXi 7 by the way.
I was considering the Startech TB310G but found this in the tech specs 'Due to technical limitations of the IC solution, max bandwidth is limited to about 7.5 Gbps'. Does anyone know if the same applies to other (Aquantia ) TB3 10 Gb NICs or only to the Tehuti-based ones? Come to think of it I'm not sure if the Tehuti will work with ESXi 7.
I cannot seem to get the native vib to load under 7.0. says it was skipped. not sure if I am doing something wrong, or missing a step? using QNA-T310G1T.
I have purchased the OWC adapter (owctb3adp10gbe) and downloaded the native driver by following your links but I am unable to get it to install. I Keep getting this error:
esxcli software vib install /tmp/MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM.670.0.0.8169922-16081713.zip
Error: Unknown command or namespace software vib install /tmp/MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM.670.0.0.8169922-16081713.zip
Any help would be appreciated
esxcli software vib install -d
I am having an issue with my lab setup:
4x Intel NUC8i5BEH
4x Sonnet Solo 10G (TB3 to 10GBE) using native atlantic network driver for VMware ESXi 1.0.2.0
Netgear XS708T (10GBE Swich)
ESXi 7.0b (build 16324942)
When I change the MTU on the vSphere Distributed Switch to anything higher than 1500bytes, I will run into TCP connectivity issues:
- connection between vCenter and ESXi hosts 'sometimes' is broken (usually it works for a while and then it's disconnected)
- TCP connection (e.g. Web-Sessions, SSH,...) are getting extremely slow
- When using pktcap-uw to capture the packets I see a lot of re-transmissions and duplicate packets
- frame size on the physical switch is set to 9198bytes
- VMKPING with DF-flag (do not fragement) is working fine
- this only happens if network traffic is traversing the physical network (VMs on the same ESXi hosts are not affected)
- when changing the MTU on the vDS back to 1500bytes everything is fine again
- but I need MTU >1600 for Geneve encapsulation (NSX-T)
Any ideas?
I have a problem upgrading my ESXi 7.0 hosts to 7.0 Update 1 and I think the Atlantic driver for the OWC Thunderbolt 3 to 10G Ethernet controllers may be at fault, as no other third-party VIBs are installed.
I am getting the following error from VUM when I try the update:
"The host returns esxupdate error codes: -1. Check the Update Manager log files and esxupdate log files for more details."
I created a log bundle (1.62GB compressed) on the vCSA and looked at the VUM logs. There are no entries for the host involved.
On the host, no esxupdate.log file is created.
I realize Update 1 is very new. Is anyone else running this or similar VIBs experiencing this problem?
is the vib on github the correct one net670-atlantic-v2.2.3.0-esxi1.6.x86_x64.vib or is the one from vmware MRVL-Atlantic-Driver-Bundle_1.0.2.0-1OEM teh correct one. I tried the latter first as it seems most recent however my esxi (mac pro 6, 1) didnt recognize the akitio thunder2 10 gig adaptor. Trying to use the one available on git hub now but getting the following error
esxcli software vib install -v "/vmfs/volumes/5fee2b1d-607e7cd0-c315-003ee1c9aee7/net670-atlanti
c-v2.2.3.0-esxi1.6.x86_x64.vib"
[ProfileValidationError]
File path of '/usr/lib/vmware/vmkmod/atlantic' is claimed by multiple non-overlay VIBs: {'MRVL_bootbank_atlantic_1.0.2.0-1OEM.670.0.0.8169922', 'Aquantia_bootbank_net-atlantic_v2.2.3.0-esxi1.6'}
Please refer to the log file for more details.
You need to grab the one from VMware, that is the official driver from Marvell
Thanks for the feedback. Were there any other dependents vibs? I believe I have all the requirements listed here but I am not seeing the 10gb recognized
esxcli network nic list
Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description
------ ------------ ------ ------------ ----------- ----- ------ ----------------- ---- -----------
vmnic0 0000:0b:00.0 ntg3 Up Up 1000 Full 00:3e:e1:9X:XX:XX 1500 Broadcom Corporation NetXtreme BCM57762 Gigabit Ethernet
vmnic1 0000:0c:00.0 ntg3 Up Up 1000 Full 00:3e:e1:X9:XX:XX 1500 Broadcom Corporation NetXtreme BCM57762 Gigabit Ethernet
[root@ESXI3:~] esxcli software vib list | grep MRVL
atlantic 1.0.2.0-1OEM.670.0.0.8169922 MRVL VMwareCertified 2021-01-26
No, that’s it. My guess is the NIC type is either not AQC or it’s one that’s not supported with driver.
Can you share output of lspci -v for the TB2 NIC?
crap maybe the chip set is different
Class 0604: 8086:1d10
Class 0300: 1002:679e
Class 0403: 1002:aaa0
Class 0300: 1002:679e
Class 0403: 1002:aaa0
Class 0106: 144d:1600
0000:10:00.0 Bridge PCI bridge: PLX Technology, Inc. Device 8723
Class 0604: 10b5:8723
Class 0604: 10b5:8723
Class 0604: 10b5:8723
Class 0604: 10b5:8723
Class 0604: 10b5:8723
Class 0604: 10b5:8723
0000:12:00.0 Serial bus controller USB controller: Fresco Logic FL1100 USB 3.0 Host Controller
Class 0c03: 1b73:1100
0000:19:00.0 Network controller Ethernet controller: Tehuti Networks Ltd. TN9210 10GBase-T Ethernet Adapter
Yup, not Aquantia (which is now part of Marvell). Guessing these models were using different vendor and when going to TB3, they switched to AQC. Without drivers, it will not be detected by ESXi
I am having issues with the official driver and my new QNAP QNA-T310G1S. It was on the list of supported devices by the Marvell driver. The following shows the driver installed on my NUC:
MRVL_bootbank_atlantic_1.0.2.0-1OEM.670.0.0.8169922
Name: atlantic
Version: 1.0.2.0-1OEM.670.0.0.8169922
Type: bootbank
Vendor: MRVL
Acceptance Level: VMwareCertified
Summary: Network driver for Marvell AQtion Ethernet Controllers
Description: Native atlantic network driver for VMware ESXi
ReferenceURLs:
Creation Date: 2020-03-03
Depends: vmkapi_2_5_0_0
Conflicts:
Replaces:
Provides:
Maintenance Mode Required: True
Hardware Platforms Required:
Live Install Allowed: False
Live Remove Allowed: False
Stateless Ready: True
Overlay: False
Tags: PCIID 1d6a0001, PCIID 1d6a00b1, PCIID 1d6a07b1, PCIID 1d6a08b1, PCIID 1d6a09b1, PCIID 1d6a11b1, PCIID 1d6a12b1, PCIID 1d6a80b1, PCIID 1d6a87b1, PCIID 1d6a88b1, PCIID 1d6a89b1, PCIID 1d6a91b1, PCIID 1d6a92b1, PCIID 1d6ad100, PCIID 1d6ad107, PCIID 1d6ad108, PCIID 1d6ad109, module, driver
Payloads: atlantic
When I query the host for NICs, I get the following:
[root@desx2:~] esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:00:1f.6 ne1000 Up 1000Mbps Full 1c:69:7a:63:2a:20 1700 Intel Corporation Ethernet Connection (10) I219-V
vusb0 Pseudo uether Up 1000Mbps Full 00:e0:4c:68:19:6a 1700 Realtek USB 101001000 LAN
vusb1 Pseudo uether Up 1000Mbps Full 00:e0:4c:68:1b:07 1700 Realtek USB 101001000 LAN
The two USB 1Gb adapters and the onboard adapter are found, but not the TB adapter one. There is a light on the back of the adapter so it is getting power. I am using a known good DAC to connect it to my Ubuiquity 16 XG switch.
System has been rebooted several times. I have verified the BIOS has Thunderbolt support Enabled and the security mode to be Legacy.
What am I missing???
Thank you!
Ok, I finally figured it out. After reloading the NUC temporarily with Win10 to update the firmware on both the NUC and the QNA-T310G1S, I was encouraged by being able to access the network in Windows. I reloaded the device to ESXi and installed the VMware Certified driver and the card was not functional again. I verified all of the settings (or so I thought) in the BIOS. After reviewing the photos above, I noticed I had not adjusted the Boot Priority setting for the Thunderbolt interface. I'm not sure why having this setting in there would enable ESXi to see the hardware and load the kernel module, but it did. I'll be posting a full rundown of the troubleshooting commands and screenshots from the NUC 10th Gen BIOS on my Blog. But I wanted to share here as well since the solution was staring at me in the face--just didn't make the connection at first.
Thanks!
Yesterday I upgraded the firmware of one of my Qnap QNA-T310GS from 3.1.109 to 3.1.121 on a NUC 10 ESXi 7 U1 host. The NIC was still detected, showed as connected on the VMware console but didn't actually have a connection. I downgraded the firmware again and everything went back to normal.
Beware with new firmware on AQC100/AQC107 adapters.
Tons of good info here but a bit unsure now. I have an Intel NUC NUC8i7HVK. I really want to put a 10gb adapter in there so I need a TB3 to ??.
To be clear, are there any of these adapters that work with this NUC and ESXi 7.0U2?
HI William,
Do you if something are compatible with ESX i 8.0U1 ?
It should be a super easy way to confirm is to simply install ESXi 8.0u1 on another USB key, then you can test it out yourself to ensure it works w/o messing with your existing setup.
Confirmed working on ESXi 8.0u2 for me. Intel NUC11PAHi5, Sabrent Thunderbolt 3 to 10 Gbps RJ-45 Ethernet Adapter (TH-S3EA)
Amazing Stuff here... on an older nuc with the QNAP eth adapter. Working a treat. Love the site! Keep it up.
Awesome. That was really helpful getting my TB3 to PCIe running.
That BIOS settings did the Trick to recognize the Intel X550 Adapter with ESXi 7.0 U3.
Sonnet TB3 10G Adapter working flawless in ESXI 6.7.
But won't load in ESXi 8.0 on the same machine?
Ispci lists the adapter, but vmkernel.log says FW kickoff failed.
I already tried different atlantic drivers...could it be a Thunderbolt/PCie issue?
Does anyone know a solution?