Dynamically generate custom vSAN ESA HCL JSON for VMware Cloud Foundation (VCF) 5.1

I recently shared on how you can deploy the latest VMware Cloud Foundation (VCF) 5.1 release with vSAN Express Storage Architecture (ESA) using Nested ESXi and leveraging a custom vSAN ESA HCL JSON file, which I had created to workaround the required vSAN ESA pre-check during the VCF Bringup process.

I will admit, I actually did not the create the custom vSAN ESA HCL JSON file manually, it was auto-generated using automation 😉

My initial goal was to make it easier for anyone to play with VCF 5.1 using both Nested ESXi as well any physical system that is capable but may is not on the official VMware vSAN ESA HCL. After a lot of trial and error testing with Nested ESXi, I finally had a functional PowerCLI script that can then be used to run against a standalone ESXi host to dynamically generate a compatible custom vSAN ESA HCL JSON file that can then be used with both VCF 5.1 with vSAN ESA or even a standalone vCenter Server deployment for vSAN ESA enablement.

For the best user experience, your physical ESXi host and/or Nested ESXi VM should only be configured with NVMe-based storage and storage controllers. If you have non-NVME devices, you should either disconnect and/or disable it within the system BIOS before attempting to run the VCF Bringup process, as these devices will be checked against the provided vSAN ESA HCL.

After downloading the PowerCLI create_custom_vsan_esa_hcl_json.ps1 script, you simply connect to a vanilla installation of ESXi using the Connect-VIServer cmdlet and if the script ran successfully, it should generate the vSAN HCL JSON file as shown in the screenshot below.

At this point, you now begin the VCF Bringup and provide this custom vSAN HCL JSON file.

Step 1 - Transfer the custom vSAN HCL JSON file to the Cloud Builder appliance under /opt/vmware/bringup/tmp/custom_vsan_esa_hcl_XXX.json

Note: For the custom vSAN HCL file to be used the jsonUpdatedTime value MUST match the latest value found in the online version of the vSAN HCL (all.json) from VMware. As of this blog post, the value should be "November 14, 2023, 11:44 PM PST" and may need to be updated if a newer vSAN HCL has been published since Cloud Builder will perform a query, if it has internet connectivity. If not, then Cloud Builder will look at the timestamp epoch value and will only use the custom vSAN HCL file if it is <=90 days compared to the current time on the Cloud Builder appliance, so if that date is greater than 90 days, simply update that with a newer value which you can run this on any Linux system: date +%s

Step 2 - Switch to root user on the Cloud Builder appliance by running the following command:

su -

Step 3 - Update the file ownership and permission of the custom vSAN HCL file so that the VCF Bringup service can access the file by running the following commands:

chmod 644 /opt/vmware/bringup/tmp/custom_vsan_esa_hcl_XXX.json
chown vcf_bringup:vcf /opt/vmware/bringup/tmp/custom_vsan_esa_hcl_XXX.json

Step 4 - Update your VCF Deployment Workbook with the full path to the custom vSAN HCL file. In my setup, I am using the VCF Deployment JSON file and you will need to add the hclFile parameter with the full path to the custom vSAN HCL file as shown in the snippet below:

Step 5 - Finally, begin the VCF Bringup process, which I have initiated using the Cloud Builder API with my VCF Deployment JSON file.

If everything was configured correctly, the VCF Bringup process should now be able to successfully validate the Nested ESXi virtual hardware using the custom vSAN HCL file which you can see using both in the Cloud Builder UI or by watching the Cloud Builder log file (/var/log/vmware/vcf/bringup/vcf-bringup-debug.log):

2023-11-17T02:05:48.260+0000 [bringup,733b207aa563f28a,b140] INFO [c.v.e.s.c.c.v.vsphere.VsphereClient,pool-3-thread-4] Successfully logged in to https://vcf-m01-esx02.primp-industries.local:443/sdk
2023-11-17T02:05:49.954+0000 [bringup,733b207aa563f28a,131f] INFO [c.v.e.s.b.v.a.i.DownloadVsanHclJson,pool-3-thread-2] Retrieving the VSAN HCL file validity from isHCLFileValid true
2023-11-17T02:05:49.954+0000 [bringup,733b207aa563f28a,131f] INFO [c.v.e.s.b.v.a.i.DownloadVsanHclJson,pool-3-thread-2] Copying vSAN HCL file from User provided local path to cloud builder tmp path
2023-11-17T02:05:49.955+0000 [bringup,733b207aa563f28a,131f] INFO [c.v.e.s.b.v.a.i.DownloadVsanHclJson,pool-3-thread-2] Setting the proxy fields {"hclFilePath":"/opt/vmware/bringup/tmp/custom_vsan_esa_hcl_XXX.json"}
2023-11-17T02:05:56.123+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.v.vsphere.VsphereClient,pool-3-thread-15] Successfully logged in to https://vcf-m01-esx02.primp-industries.local:443/sdk
2023-11-17T02:05:56.124+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.vmware.vsan.VsanClient,pool-3-thread-15] Successfully login to https://vcf-m01-esx02.primp-industries.local:443/vsan
2023-11-17T02:05:56.164+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.v.vsan.VsanManagerBase,pool-3-thread-15] Checking if vSAN is enabled on the vSAN system of vcf-m01-esx02.primp-industries.local.
2023-11-17T02:06:18.311+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-15] Task: (MOR:haTask--vim.host.VsanSystem.update-3521218305) (Name:update) Entity: (MOR:ha-host) (Name:ha-host) is complete
2023-11-17T02:06:18.312+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.v.vsan.VsanManagerBase,pool-3-thread-15] Uploading VSAN HCL. Attempt number 1
2023-11-17T02:06:21.011+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.v.vsan.VsanManagerBase,pool-3-thread-15] VSAN HCL was successfully uploaded.
2023-11-17T02:06:38.090+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.v.vsphere.VcManagerBase,pool-3-thread-15] Task: (MOR:haTask--vim.host.VsanSystem.update-3521218352) (Name:update) Entity: (MOR:ha-host) (Name:ha-host) is complete
2023-11-17T02:06:38.579+0000 [bringup,733b207aa563f28a,87cb] INFO [c.v.e.s.c.c.v.vsan.VsanManagerBase,pool-3-thread-15] Successfully verified the HCL compatibility.