If you are using an AMD Ryzen (Consumer) CPU for your VMware Cloud Foundation (VCF) lab environment, you have most likely have applied one of the required workarounds for either installing an NSX Edge (HERE) or upgrading an NSX Edge (HERE).
For those planning to install or upgrade to the latest VCF 9.0.2 patch release, which was just released today, you may come across this issue as shown in the screenshot below when attempting an NSX Edge upgrade.
While I have yet to run through the VCF 9.0.2 upgrade myself, a similar configuration file tweak from within the NSX Edge VM will be required. A colleague recently shared a more optimal trick for bypassing the non-supported AMD processor check, which would be simpler to implement for lab environments.
Disclaimer: This is not officially supported by Broadcom, please use at your own risk.
Instead of having to manually modify the configuration scripts that performs the AMD processor check within the NSX Edges, which will revert upon successful a upgrade, we can simply trick the NSX Edge VM to think it is running on an AMD EPYC processor rather than AMD Ryzen.
The solution is similar to what is performed by Enhanced vMotion Compatibility (EVC), where we are exposing specific CPUIDs to the VM. In this case, we are using updating the cpuid.brandstring to include the "EPYC" string as part of the CPU processor string. To further simplify this workaround, instead of applying this on a per-NSX Edge basis, we can apply this globally on an ESX host, so we do not have to deal with the timings of when the NSX Edge VM is deployed and/or powered on.
Step 1 - Identify the original AMD processor string using the vSphere UI or ESX Host Client. In the example, the system is running on an AMD Ryzen 9 9955HX
Step 2 - SSH to all ESX hosts and run the following command which will include the original AMD processor string with the appended "EPYC":
echo 'cpuid.brandstring = "AMD EPYC Ryzen 9 9955HX"' >> /etc/vmware/config
A reboot of the ESX host is not required but if you had attempted an installation or/upgrade of the NSX Edge VMs, you will need to make sure the VM has gone through full power cycle or simply re-attempt the operation which should yield in a new NSX Edge deployment.
For those automating their deployment of ESX hosts using kickstart, I highly recommend you include this in your configuration, so you do not have to think about it the next time you need to re-deploy.
Thank you and your team for this workaround, works great. I have the Minisforum A2 7940HX.
William, is there more we have to do to get Edge devices to run on AMD? It installs fine after the tweak above but I keep encountering CPU locks. I had it working with east west, north south, dhcp working. But when I checked a day later they were both locked up.
Thanks for all your continued support as well.
Did you apply monitor_control.disable_apichv ="TRUE" to /etc/vmware/config already?
Are you using NVMe Tiering at all? It would be helpful if you can provide a link to your generated vc-support bundle (please include ESX hosts logs)
William, I am using Mem Tiering, after I your saw your message I turned off mem Tiering on these two edge nodes. I think the damage was done already as they had to be powered off due to lack of response and are having nothing but CPU soft locks on there console screen. Odd, only VM's having any issues. I'm running Minisforum A2 7940 by the way.
Thought this was fixed. ""Did you apply monitor_control.disable_apichv ="TRUE" to /etc/vmware/config already?"" I'll do so now. Thought all we had to do was what you mentioned above. Sorry for missing this one. Last night I removed Edge nodes from NSX, reinstalled one, then added the advanced parm to disable Mem Tiering to just this VM. Woke up this AM and on the console it's asking for root password for maintenenace, ugg. Odd, cause all the other VM's are running fine and VSAN not complaining about any issues.
Let me apply both of these fixes, maybe I'll turn off mem tiering for one host and pin one Edge node to that host and also the fixes mentioned above.
Previously had done: cpuid.brandstring = "AMD EPYC Ryzen 9 7940HX"
today did "monitor_control.disable_apichv ="TRUE"" and turned OFF mem tiering on all 3 host. After a redeploy seems as though EDGE node is fine.
As you can imagine memory is tight as I opted for the 96GB version of 3 Minisforum units. Since I added the "disable_apichv" to all 3 host I may turn on mem tiering to all 3 host but disable mem tiering on just the Edge node "sched.mem.enableTiering = false" in the advanced config of the VM and see what happens.
Thanks for all your assistance!
Just an FYI, wanted to thank you for pointing out I missed the tweak above. I turned on Mem Tiering for 3 host, turned it off as a safety feature on the 2 Edge nodes. No soft/hard CPU locks at all for 3 days.
I added the tweak to my list as in the lab your often rebuilding to try different deployments
Thank you!