A fellow colleague was having some problems with the vCenter Server Appliance (VCSA) CLI Installer when using the vSAN ESA (Express Storage Architecture) bootstrap method, which automatically the configures vSAN ESA on a physical ESXi host before deploying the VCSA on top of the vSAN ESA Datastore.
Whether he manually downloaded the vSAN JSON HCL or generated his own custom vSAN ESA HCL, the VCSA Installer would fail with the following error:
Expecting value: line 1 column 1 (char 0)
As of VMware Cloud Foundation 5.1.0, you can now use the new vSAN Express Storage Architecture (ESA) when deploying a VCF Workload Domain where as in earlier releases, only the vSAN Original Storage Architecture (OSA) was supported, which is typically what I deploy for testing purposes.
While working on some enhancements to my VCF Automated Lab Deployment Script, I needed to go through a VCF Workload Domain deployment using vSAN ESA but I kept running into the following error within SDDC Manager: Perform vSAN ESA Auto Disk Claim
Based on the error message, I had assumed there was an issue with claiming the SSD on the Nested ESXi VM for vSAN ESA but when I logged into the VCF Workload Domain vCenter Server, I was able to claim the devices without any issues. Looking at the SDDC Manager logs /var/log/vmware/vcf/domainmanager/domainmanager.log also did not yield anything useful:
2024-05-27T12:43:44.545+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-16 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.598+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-19 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.650+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-20 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.701+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-21 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.702+0000 DEBUG [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.e.s.c.c.v.vsphere.VsphereClient,dm-exec-14] Destroying 2 open views 2024-05-27T12:43:44.722+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Failed to verify storage pool disks for cluster wld-w01-cl01 2024-05-27T12:43:44.723+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Found host(s) in clust er wld-w01-cl01 without contributing to storage pool. 2024-05-27T12:43:44.723+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-14] [A337BL] VSPHERE_AUTO_DISK_CLAIM_FAILED Failed to enable auto claim on cluster wld-w01-cl01 com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to enable auto claim on cluster wld-w01-cl01 at com.vmware.vcf.common.fsm.plugins.action.impl.ClaimDisksForVsanEsaClusterAction.postValidate(ClaimDisksForVsanEsaClusterAction.java:119) at com.vmware.vcf.common.fsm.plugins.action.impl.ClaimDisksForVsanEsaClusterAction.postValidate(ClaimDisksForVsanEsaClusterAction.java:23) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.lambda$static$1(FsmActionState.java:23) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144) at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400) at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:561) at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124) at jdk.internal.reflect.GeneratedMethodAccessor397.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:85) at com.google.common.eventbus.Subscriber.lambda$dispatchEvent$0(Subscriber.java:71) at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:59) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Found host(s) in cluster wld-w01-cl01 without contributing to storage pool. at com.vmware.vcf.common.fsm.plugins.action.impl.ClaimDisksForVsanEsaClusterAction.postValidate(ClaimDisksForVsanEsaClusterAction.java:113) ... 17 common frames omitted
After a bit of digging, I was wondering if this was related to a similiar issue that is seen during the VCF Bringup of the VCF Management Domain using vSAN ESA where a custom vSAN HCL JSON was required?
I recently shared on how you can deploy the latest VMware Cloud Foundation (VCF) 5.1 release with vSAN Express Storage Architecture (ESA) using Nested ESXi and leveraging a custom vSAN ESA HCL JSON file, which I had created to workaround the required vSAN ESA pre-check during the VCF Bringup process.
I will admit, I actually did not the create the custom vSAN ESA HCL JSON file manually, it was auto-generated using automation 😉
My initial goal was to make it easier for anyone to play with VCF 5.1 using both Nested ESXi as well any physical system that is capable but may is not on the official VMware vSAN ESA HCL. After a lot of trial and error testing with Nested ESXi, I finally had a functional PowerCLI script that can then be used to run against a standalone ESXi host to dynamically generate a compatible custom vSAN ESA HCL JSON file that can then be used with both VCF 5.1 with vSAN ESA or even a standalone vCenter Server deployment for vSAN ESA enablement.