The vast majority of VMware Homelabs is still Intel-based today but I have been seeing a slow rise of AMD-based kits being adopted, especially with AMD's desktop line of CPUs known as Ryzen. One of the considerations on whether you could use an AMD processor was whether you were planning to deploy NSX-T and in earlier releases, only Intel was supported as the NSX-T Edge required support for Data Plane Development Kit (DPDK) and this was only supported with Intel-based processors.
With the latest NSX-T 3.0 release, AMD-based processors are now supported and per the release notes, the following CPUs can be used:
- AMD EPYC 7xx1 Series (Naples)
- AMD EPYC 3000 Embedded Family and newer
- AMD EPYC 7xx2 Series (Rome)
You will notice that only AMD's server line of CPUs known as EPYC are currently supported, which makes sense for running Production workloads. If you attempt to deploy an NSX-T Edge Node running on a non-EPYC platform, you will get an error message stating the CPU is not supported and I figured this was probably due to the lack of DPDK support in the consumer CPUs.
Yesterday, in our internal "Homelab" Slack channel, I came across an interesting tidbit from Andrea Spagnolo, a Sr. Field Engineer in our Cloud Native Business Unit who shared a pretty neat trick on how to get latest NSX-T 3.0 release to work with a Ryzen-based CPU.
Disclaimer: This is not officially supported by VMware. The behaviors described here can change in the future
First off, I want to thank Andrea for sharing but also credit to Beniamino Guarnaschelli and his blog post here which actually gave Andrea the idea to take a closer look as he was trying to get this setup in his own personal homelab.
The way NSX-T ensures you that you a supported AMD CPU is by looking for the "AMD EPYC" string in the model name which is performed within the NSX-T Edge using the following /opt/vmware/nsx-edge/bin/config.py script. To bypass this check, we just need to comment out the lines that does the actual check.
Step 1 - Deploy an NSX-T Edge (either standalone or using the NSX-T Manager workflow)
Step 2 - SSH to NSX-T Edge as root and comment out lines 181-184
Step 3 - Run the following command to start the dataplane service which will cause the VM to reboot
su - admin
start service dataplane
Step 4 - You can now configure the NSX-T Edge as a Transport Node and you will not run into any errors
Note: One additional issue that Andrea ran into was that IOMMU was NOT enabled on his motherboard which is requirement. Once this was done along with the configuration tweak above, he was able to get NSX-T Edge working on his AMD (3700x) kit.
Andrea has also confirmed this trick works with both NSX-T 2.5 as well as the latest 3.0 release in case folks wanted to try this out on an earlier version.
Hi William,
thanks for the credits and for mentioning me.
Ciao
Benja
With this clarification, please note that AMD have a very good "baby EPYC" processor series out. A Supermicro motherboard with a SoC EPYC 3251 8 cores and 16 threads is a very attractive home lab solution that is equivalent to a Xeon D but probably a couple of hundred dollars less.
Thanks for this. I just ran up against the problem and was a little bit worried.
Thank you, thank you, thank you. This did the trick. I am running a Threadripper cluster and this did the trick really nice and saved me Thousands. I had also been planning to upgrade the primary datacenter to Epycs and it is really nice to know that the Edge is fully supported on it now. I was beginning to panic. The last day has been a nightmare.
Thank you for this post.
I have a node with an E5520 processor, when deploying the Edge, an error message "The host does not support required cpu features: ['aes']."
I was able to access the /opt/vmware/nsx-edge/bin/config.py file and comment out the lines that do the actual check.
When I started the dataplane service, the edge restarted as expected and the message still persists, but when I wanted to put the edge in an edge cluster, it shows me another error indicating that installing the edge it is on pending state.
Is there a way to automated the edit process? Perhaps ssh file copy, or bash shell like script?
When you run into the same problem trying to do an _upgrade_ of your NSX-T Edges, you need to make the same changes in the file that is staged to /os_bak/opt/vmware/nsx-edge/bin/config.py
Are there any update or changes to NSX-T 3.x as to support AMD Threadripper?
In my case, once comment out the 4 lines, the EDGE is automatically reboot.
After that, the Edge still cant use.
The workaround is after reboot the Edge, [stop service dataplane] →[start service dataplane].
I have to do this Every time If I poweroff/Shutdown the Edge.
Thank you for this. So far the Edge is now working on my AMD Threadripper 1920X Cluster. Have not done a reboot or an upgrade yet.
Could it be that with NSX-T 3.2.0.1 the Manager stores which CPU the edges are using and won't update it after an offline vmotion to a Host with an Intel CPU?
I've deployed edges on Hosts with unsupported AMD Ryzen CPUs which worked well after commenting out the lines in the python skript.
Unfortunately i can't edit anything on the EdgeSwitches (New TransportZones, Uplink Profile, etc.) because the NSXT Manager is saying that the edge is running on unsupported AMD CPUs...
Since i've had an Intel ESXi i've migrated the edges to that Host to ensure that no AMD CPU is getting in my way but the nsxt manager still tells me that those edges are running on unsupported AMD CPUs when i want to fix the config issue (because i did a vmotion from AMD ESXi to Intel ESXi) which isn't true because those edges are now running on an Host with an Intel(R) Core(TM) i5-10400F.
Is there a way to disable the CPU Check within the Manager?
This is the error i'm getting from the nsxt manager, when i want to fix the Sync Error/Configuration Issue after migrating the Edge to the Intel ESXi Host:
Error: [Fabric] Edge 21b1c770-bb0f-43e3-a46e-8c2637de27e7 is not ready for hardware is not supported, error detail is NSX Edge configuration has failed. Unsupported CPU: AMD Ryzen 7 1700 Eight-Core Processor (Error code: 15000)
I don't have an AMD setup, so can't really see this in action. However, I did spin up an HOL w/NSX 3.x and I was poking around the NSX Manager and it seems to have the same set of config scripts as NSX Edge. I'm wondering ... if these are somehow managed there, have you tried logging into NSX Manager instance and update the same file and seeing if that makes a difference?
Hi Oliver, were you able to disable CPU Check within the Manager? I am running the same issue as you (using ver. 3.2.1.0.0.19801966) and I am not able to configure NSX Edge from NSX-T manager....getting error: " [Fabric] Edge ea26153d-ab17-4760-8f27-7656b44dd710 is not ready for hardware is not supported, error detail is NSX Edge configuration has failed. Unsupported CPU: AMD Ryzen 7 5800X 8-Core Processor". I have commented 4 lines in /opt/vmware/nsx-edge/bin/config.py on Edge, nothing commented on NSX Manager (I haven't found similar script there). Thanks a lot in advance for your reply.