How much Virtual Machine memory is using NVMe Tiering?

I know many in the VMware Community have upgraded to vSphere 8.0 Update 3 to play with the exciting new NVMe Tiering feature, especially for testing various workloads including deploying VMware Cloud Foundation (VCF) for homelab or testing purposes.

A question that has come up a few times now is how do I check the amount of VM memory that has been offloaded to Tier 1 memory (NVMe Tiering) versus using Tier 0 memory (DRAM)?

Today, you can get this information on a specific ESXi host by using the the memstats utility which now has a new report type called vmtier-stats.

Note: In the future, when NVMe Tiering goes out of Tech Preview into generally available (GA), there will be a centralized method for getting this information and going to individual ESXi hosts will not be required.

There are a number of different metrics provided by the vmtier-stats report, but the ones that will be of interests are the following:

One thing you may notice is that memstats uses the VM Cartel ID (e.g. vm.XXXXX) to identify a particular VM, which is not very user friendly. With that said, we can retrieve the VM Cartel ID to VM name by using ESXCLI and then saving that output to a temporarily file, so we can augment the memstats output for ease of consumption.

Step 1 - Run the following command which will output VM Cartel ID to VM name mapping for all current running VMs to a file stored in /tmp/vms

esxcli --formatter csv vm process list | awk -F ',' '{print $2,$5}' > /tmp/vms

Step 2 - Run the following command to retrieve the vmtier-stats and using our VM name mapping file (/tmp/vms), we can replace the VM Cartel ID with the friendly VM name in the output:

memstats -r vmtier-stats -u mb -s name:memSize:active:tier0Consumed:tier1Consumed | sed "$(while read -r name id; do echo "s/vm.$id/$name/g"; done < /tmp/vms)"

From the screenshot above, I have 7 VMs running (VMware Cloud Foundation) on a single ESXi with 96GB of DRAM and NVMe Tiering enabled using 400% ratio. There is ~403GB of configured memory, ~25GB of active memory, ~42GB of consumed DRAM (Tier 0) and ~50GB of memory using NVMe Tiering (Tier 1). What is interesting is that while VCF requires 384GB of memory for deployment, less than half of the physical DRAM is actually being consumed, the overall end user experience is not really impacted based on my experience.