Historically, enabling NVMe Tiering prior to VMware Cloud Foundation (VCF) 9.1 was not very user friendly, as it required users to remember three distinct ESXCLI commands.
# Enable or Disable NVMe Tiering
esxcli system settings kernel set -s MemoryTiering -v TRUE
# Configure the Tiering %
esxcli system settings advanced set -o /Mem/TierNvmePct -i 100
# Configure NVMe device for with NVMe Tiering
esxcli system tierdevice create -d /vmfs/devices/disks/${NVME_TIERING_DEVICE}
By now, most users have probably made a mental note of these commands, with some even incorporating them into their automated ESX Kickstart deployments (example), so it is something you do not have to remember.
VCF 9.1 introduces a new method for enabling and configuring NVMe Tiering that is significantly simpler and no longer requires a system reboot. As a result of these improvements, the ESXCLI commands used in previous releases will no longer properly enable NVMe Tiering in VCF 9.1 and later.
I have seen an increasing number of users, both internally and externally, report that they have enabled NVMe Tiering, only to discover that it was never properly activated due to the use of the legacy ESXCLI commands.
Users will typically share a screenshot from vCenter Server similar to the one above, and there are two dead giveaways that NVMe Tiering was not properly enabled.
- The first is the traditional memory capacity view, which does not reflect the combined memory capacity based on the configured NVMe Tiering ratio.
- The second is the new Memory Tiering widget in the vSphere UI, where the Tier 1 capacity, representing the NVMe Tiering device capacity, shows a value of 0.