WilliamLam.com

Using Nested ESXi for VMware Cloud Foundation (VCF) Workload Domain fails with vSAN ESA Auto Disk Claim 

by // 2 Comments

As of VMware Cloud Foundation 5.1.0, you can now use the new vSAN Express Storage Architecture (ESA) when deploying a VCF Workload Domain where as in earlier releases, only the vSAN Original Storage Architecture (OSA) was supported, which is typically what I deploy for testing purposes.

While working on some enhancements to my VCF Automated Lab Deployment Script, I needed to go through a VCF Workload Domain deployment using vSAN ESA but I kept running into the following error within SDDC Manager: Perform vSAN ESA Auto Disk Claim


Based on the error message, I had assumed there was an issue with claiming the SSD on the Nested ESXi VM for vSAN ESA but when I logged into the VCF Workload Domain vCenter Server, I was able to claim the devices without any issues. Looking at the SDDC Manager logs /var/log/vmware/vcf/domainmanager/domainmanager.log also did not yield anything useful:

2024-05-27T12:43:44.545+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-16 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.598+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-19 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.650+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-20 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.701+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Host host-21 does not contribute to the vSAN ESA storage. 2024-05-27T12:43:44.702+0000 DEBUG [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.e.s.c.c.v.vsphere.VsphereClient,dm-exec-14] Destroying 2 open views 2024-05-27T12:43:44.722+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] Failed to verify storage pool disks for cluster wld-w01-cl01 2024-05-27T12:43:44.723+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.v.c.f.p.a.i.ClaimDisksForVsanEsaClusterAction,dm-exec-14] com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Found host(s) in clust er wld-w01-cl01 without contributing to storage pool. 2024-05-27T12:43:44.723+0000 ERROR [vcf_dm,66548000bff276b81889b17ab22d34ed,2f33] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-14] [A337BL] VSPHERE_AUTO_DISK_CLAIM_FAILED Failed to enable auto claim on cluster wld-w01-cl01 com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to enable auto claim on cluster wld-w01-cl01 at com.vmware.vcf.common.fsm.plugins.action.impl.ClaimDisksForVsanEsaClusterAction.postValidate(ClaimDisksForVsanEsaClusterAction.java:119) at com.vmware.vcf.common.fsm.plugins.action.impl.ClaimDisksForVsanEsaClusterAction.postValidate(ClaimDisksForVsanEsaClusterAction.java:23) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.lambda$static$1(FsmActionState.java:23) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144) at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400) at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:561) at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124) at jdk.internal.reflect.GeneratedMethodAccessor397.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:85) at com.google.common.eventbus.Subscriber.lambda$dispatchEvent$0(Subscriber.java:71) at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:59) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Found host(s) in cluster wld-w01-cl01 without contributing to storage pool. at com.vmware.vcf.common.fsm.plugins.action.impl.ClaimDisksForVsanEsaClusterAction.postValidate(ClaimDisksForVsanEsaClusterAction.java:113) ... 17 common frames omitted

After a bit of digging, I was wondering if this was related to a similiar issue that is seen during the VCF Bringup of the VCF Management Domain using vSAN ESA where a custom vSAN HCL JSON was required?

[Read more...]

Customizing SMBIOS strings (hardware manufacturer and vendor) for Nested ESXi 

by // 11 Comments

Nested ESXi continues to be an invaluable resource that I use on almost a daily basis from solution prototyping, customer reproduction to automated lab deployments supporting both VMware Cloud Foundation (VCF) and VMware vSphere Foundation (VVF), the use cases are simply endless!

While you can do almost anything with Nested ESXi, mimicking or simulating specific hardware strings such as the manufacturer or vendor is not really possible or at least very easily. I was recently reminded of this topic again as this has been something I wanted to look into but just not had the time. In fact, some of this was inspired by a conversation I had with fellow colleague Luke Huckaba who found a clever trick playing with the default Virtual Machine boot ROMs which are shipped with both ESXi and our VMware Desktop Hypervisors (Workstation/Fusion).

[Read more...]

Creating an offline VMware Cloud Foundation (VCF) Depot for multiple VCF environments

by // 10 Comments

By default, SDDC Manager uses the online VMware Depot for downloading all software updates (security, patch, upgrade) for life-cycling a single VMware Cloud Foundation (VCF) instance. SDDC Manager can also support air-gapped or disconnected environments by leveraging the VCF Offline Bundle Transfer Utility (OBTU), which my colleague Eric Gray recently published a detailed blog post demo'ing OBTU.

While a solution exists for managing an air-gapped VCF environment, the process can be time consuming if you have to support multiple VCF environment as you need to transfer all VCF bundles from where the OBTU is running to each individual SDDC Manager. Ideally, you vi  just host your own offline VCF Depot that all SDDC Managers can simply point to, similar to what is used by vCenter Server with the their Update Manager Download Service (UMDS).

Luckily, an offline depot is not a foreign concept for VCF and while researching this topic, I was even pointed to this old 2020 blog post in creating your own local VCF 3.x depot, but the process was out of date and more importantly, OBTU was not available at the time.

By leveraging a combination of OBTU and the 2020 blog post, I was able to come up with an updated process to create an offline VCF depot that could be used by multiple VCF environments. I was able to successfully validate my offline VCF depot by applying the latest VCF 5.1.1 update to my existing VCF 5.x environment!

[Read more...]