Useful NVMe Tiering reporting using vSphere 8.0 Update 3 APIs

After successfully enabling the NVMe Tiering feature, which was introduced in vSphere 8.0 Update 3, you can find some useful details about your NVMe Tiering configuration by navigating to a specific ESXi host and under Configure->Hardware and under the Memory section as shown in the screenshot below.

There is quite a bit of information that we can see, so lets break down the individual items that are useful from an NVMe Tiering point of view and the specific vSphere APIs that can be used to retrieve this information.

Memory Tiering Enabled

The Memory Tiering field specifies whether memory tiering is enabled on the ESXi host which can have three possible values, No Tiering, Hardware Memory Tiering via Intel Optane or Software Memory Tiering via NVMe Tiering. We can retrieve this field by looking at the memoryTieringType property in the vSphere API, which has three enumerated values.

Here is a quick PowerCLI snippet to retrieve this field for a specific ESXi host:

(Get-VMHost "esxi-01.williamlam.com").ExtensionData.Hardware.MemoryTieringType

Tier 0 Memory

The Tier 0 field represents the total physical DRAM memory that is available on the ESXi host. We can retrieve this field by looking at the memoryTierInfo property in the vSphere API, which returns an array of results that contains both Tier 0 and Tier 1 values.

Here is a quick PowerCLI snippet to retrieve this field for a specific ESXi host:

((Get-VMHost "esxi-01.williamlam.com").ExtensionData.Hardware.MemoryTierInfo | where {$_.Type -eq "DRAM"}).Size

Tier 1 Memory

The Tier 1 field represents the total memory provided by NVMe Tiering that is available on the ESXi host. We can retrieve this field by looking at the memoryTierInfo property in the vSphere API, which returns an array of results that contains both Tier 0 and Tier 1 values.

Note: You can ignore the "Unmappable" terminology, this is just another way to refer to the non-DRAM memory 🙂

Here is a quick PowerCLI snippet to retrieve this field for a specific ESXi host:

((Get-VMHost "esxi-01.williamlam.com").ExtensionData.Hardware.MemoryTierInfo | where {$_.Type -eq "NVMe"}).Size

The Total field represents the total amount memory that is available to ESXi when combining both DRAM and NVMe Tiering memory, which you can aggregate using both Tier 0 and Tier 1 sizes (bytes).

NVMe Tiering Device

To understand which NVMe device is configured for NVMe Tiering, we need to navigate to Configure->Storage->Storage Devices to view the list of devices and under the Datastore column, the value you will be looking for is "Consumed for Memory Tiering" as shown in the screenshot below. We can retrieve this field by looking at the usedByMemoryTiering property when enumerating all storage devices.

Here is a quick PowerCLI snippet to retrieve this field for a specific ESXi host:

$storageSystem = Get-View (Get-vmhost "esxi-01.williamlam.com").ExtensionData.ConfigManager.StorageSystem
($storageSystem.StorageDeviceInfo.ScsiLun | where {$_.UsedByMemoryTiering -eq $true}).CanonicalName

NVMe Tiering Ratio

The amount of DRAM to NVMe ratio is 25% by default and is configured with the following ESXi Advanced Setting called Mem.TierNvmePct. We can retrieve this field by using either vSphere API (OptionaManager) or via ESXCLI.

Here is a quick PowerCLI snippet to retrieve this field for a specific ESXi host:

(Get-vmhost "esxi-01.williamlam.com" | Get-AdvancedSetting -Name Mem.TierNvmePct).Value

Report Summary

I have pulled in all the concepts from above and created the following PowerCLI script called get-nvme-tiering-info.ps1 which provides a nice summary for all ESXi hosts under a specific Sphere Cluster (you can also modify the script to have it query a specific ESXi host), which can be useful for to quickly get insights across hosts that may or may not have NVMe Tiering configured.

Here is a screenshot of what the output would look like: