WilliamLam.com

Configure NSX-T Edge to run on AMD Ryzen CPU

by // 13 Comments

The vast majority of VMware Homelabs is still Intel-based today but I have been seeing a slow rise of AMD-based kits being adopted, especially with AMD's desktop line of CPUs known as Ryzen. One of the considerations on whether you could use an AMD processor was whether you were planning to deploy NSX-T and in earlier releases, only Intel was supported as the NSX-T Edge required support for Data Plane Development Kit (DPDK) and this was only supported with Intel-based processors.

With the latest NSX-T 3.0 release, AMD-based processors are now supported and per the release notes, the following CPUs can be used:

  • AMD EPYC 7xx1 Series (Naples)
  • AMD EPYC 3000 Embedded Family and newer
  • AMD EPYC 7xx2 Series (Rome)

You will notice that only AMD's server line of CPUs known as EPYC are currently supported, which makes sense for running Production workloads. If you attempt to deploy an NSX-T Edge Node running on a non-EPYC platform, you will get an error message stating the CPU is not supported and I figured this was probably due to the lack of DPDK support in the consumer CPUs.

Yesterday, in our internal "Homelab" Slack channel, I came across an interesting tidbit from Andrea Spagnolo, a Sr. Field Engineer in our Cloud Native Business Unit who shared a pretty neat trick on how to get latest NSX-T 3.0 release to work with a Ryzen-based CPU.

Disclaimer: This is not officially supported by VMware. The behaviors described here can change in the future

First off, I want to thank Andrea for sharing but also credit to Beniamino Guarnaschelli and his blog post here which actually gave Andrea the idea to take a closer look as he was trying to get this setup in his own personal homelab.

[Read more...]

Changing the default size of the ESX-OSData volume in ESXi 7.0

by // 29 Comments

In ESXi 7.0, a new partition scheme was introduced which also brings along a new set of storage requirements. These changes are explained in the official documentation here and the following VMware KB 77009 also contains some additional info which can be helpful. Storage changes are not easy but this was necessary to not only better support some of the current capabilities but more importantly, it setups the foundation for future ESXi capabilities.

The biggest change to the partition layout is the consolidation of VMware Tools Locker, Core Dump and Scratch partitions into a new ESX-OSData volume (based on VMFS-L). This new volume can vary in size (up to 138GB) depending on a number of factors including the current ESXi boot media (USB SD-Card, Local Disk) but also the size of the device itself, which is explained in the official documentation.

From some of the comments on Twitter, Reddit and the direct inquiries that I have received, this new behavior seems to be most impactful to smaller homelabs where a fresh install of ESXi 7.0 has been performed. Folks have shared that their ESX-OSData volume has taken up 120GB which can be quite significant if you have a smaller disk which can be quite common. I normally install ESXi on a USB device and I also use vSAN, which has a different behavior and I have also not upgraded my physical ESXi host (E200-8D) to 7.0 yet.

I performed a fresh installation of ESXi 7.0 (running as Nested ESXi VM) that was configured with 1TB of storage and here is what the filesystem layout now looks:


We can see that the ESX-OSData volume takes up ~119.75GB, which is not too bad for 1TB volume but I can understand this may not be ideal if you have something smaller such as 250GB to 512GB disk. Due to the size of the local device, the boot options mentioned in the KB would not be helpful and I was curious myself if this ESX-OSData volume size could be configurable. In doing some research it looks like the size of the ESX-OSData can be specified using the following ESXi boot option (SHIFT+O during the initial boot) called autoPartitionOSDataSize

UPDATE (12/17/20) - Official support for specifying the size of ESX-OSData has been added to the release of ESXi 7.0 Update 1c with a new ESXi kernel boot option called systemMediaSize which takes one of four values:

  • min = 25GB
  • small = 55GB
  • default = 138GB (default behavior)
  • max = Consumes all available space

If you do not require or have 138GB for the ESX-OSData, you can override the default behavior by appending this option with the specified value (e.g. systemMediaSize=min). It is worth noting that by using this setting, the smallest ESX-OSData volume you can configure is 25GB. For homelabs or environment which require less than this, you would have to use the unsupported autoPartitionOSDataSize parameter , which is not officially supported as mentioned below.

Disclaimer: This may not be officially supported by VMware as it deviates from the system defaults and can have other unintended behaviors. Use at your own risk.

[Read more...]

Quick Tip - Allow unsupported CPUs when upgrading to ESXi 7.0

by // 69 Comments

As outlined in the vSphere 7.0 release notes (which everyone should carefully read through before upgrading), the following CPU processors are no longer supported:

  • Intel Family 6, Model = 2C (Westmere-EP)
  • Intel Family 6, Model = 2F (Westmere-EX)

To help put things into perspective, these processors were released about 10 years ago! So this should not come as a surprise that VMware has decided remove support for these processors which probably also implies the underlying hardware platforms are probably quite dated as well. In any case, this certainly has affected some folks and from what I have seen, it has mostly been personal homelab or smaller vSphere environments.

One of my readers had reached out to me the other day to share an interesting tidbit which might help some folks prolong their aging hardware for another vSphere release. I have not personally tested this trick and I do not recommend it as you can have other issues longer term or hit a similiar or worse situation upon the next patch or upgrade.

Disclaimer: This is not officially supported by VMware and you run the risk of having more issues in the future.

Per the reader, it looks like you can append the following ESXi boot option which will allow you to bypass the unsupported CPU during the installation/upgrade. To do so, just use SHIFT+O (see VMware documentation for more details) and append the following:

allowLegacyCPU=true

There have also been other interesting and crazy workarounds that attempt to workaround this problem. Although some of these tricks may work, folks should really think long term on what other issues can face by deferring hardware upgrade. I have always looked at homelab as not only a way to learn but to grow yourself as an individual.

Note: The boot option above is only temporarily and you will need to pass in this option upon each restart. It looks like this setting is also not configurable via ESXCLI which I initially had thought, so if you are installing this on a USB device, the best option is to edit the boot.cfg and simply append the parameter to kernelopt line so it'll automatically be included for you without having to manually typing this. If this is install on disk, then you will need to edit both /bootbank/boot.cfg and /altbootbank/boot.cfg for the settings to passed in automatically.

This is ultimately an investment you are making into yourself, so do not cut yourself short and consider looking at a newer platform, especially something like an Intel NUC which is fairly affordable both in cost as well as power, cooling and form factor.