By default, VMware Cloud Foundation (VCF) 9.0 will use the optimized Enhanced Data Path (EDP) Standard switch mode when configuring NSX, which will require you to have an EDP capable network adaptor. One way to check whether your network adaptor supports EDP is by using the Broadcom Compatibility Guide (BCG) and ensuring "Enhanced Data Path - Interrupt mode" feature
Alternatively, if you already have ESXi running, you can quickly check by running the following command: esxcfg-nics -e
If you are automating the installation of ESXi using a scripted installation (aka Kickstart) and the KS.CFG is located on a CD-ROM device and you happen to be using a Nested ESXi VM and see the following message:
cannot find kickstart file on cd-rom with path -- /KS.CFG
You are probably using an IDE-based controller for the CD-ROM device, which the underlying IDE driver for ESXi has been removed as part of VMware Cloud Foundation (VCF) 9.0.
The fix is to simply remove the IDE controller and use a SATA controller and things will work as expected!
The NVMe Tiering feature was first previewed with the release of vSphere 8.0 Update 3 and the feature is now officially supported with the release of VMware Cloud Foundation (VCF) 9.0! ?
Since the original tech preview, all major limitations and unsupported workload profiles that would prevent an organization from deploying NVMe Tiering in production has been resolved as part of the VCF 9.0 release.
Although Nested Virtualization is NOT officially supported, many users rely on Nested ESXi for testing, development, and learning purposes. With that said, if you have NVMe Tiering enabled on an ESXi 9.0 host and you attempt to power on a Nested ESXi VM or VM that is configured with Virtual Hardware-Assisted Virtualization (VHV), it will fail to power on with the following error message as shown in the screenshot below.
While it was possible to use Nested ESXi with the tech preview of NVMe Tiering, it was just not in scope as part of productizing NVMe Tiering for VCF 9.0.
All hope is not lost, while Nested Virtualization is still NOT officially supported by Broadcom, it is a very useful feature not just for our users but also for internal development purposes and after speaking with the tech lead for NVMe Tiering, this is something that will be addressed in a future update of VCF and this is just a short term limitation for now if you intend to run Nested ESXi workloads.