Since the introduction of the License Later feature with VMware Cloud Foundation (VCF) 5.1.1, I typically deploy my VCF lab without needing any license keys and simply take advantage of the built-in 60 day evaluation period.
I was doing some testing recently with the latest VCF 5.2 release, which had just GA'ed last week and I wanted to use my new VCF licenses which included vCenter Server, ESXi, vSAN and NSX and right before the deployment was about to finish, I ran into the following error Update SDDC Manager with Licensing Information with Cloud Builder:
I thought the error was strange since my license keys had passed the Cloud Builder pre-check, so I had assumed the issue must be something else but after looking at the log file /var/log/operationsmanager.log on SDDC Manager:
Unable to decode the license key XXXX-XXXXX-XXXXX-XXXXX-YYYYY
com.vmware.licensecheck.SerialNumException: Invalid serial number: no dormant license file corresponding to the serial number
....
"errorCode":"LICENSE_KEY_NOT_RECOGNISED","arguments":[],"message":"The SDDC Manager is unable to recognize the license key. Try applying the license key to the product.","remediationMessage":"Try applying the license key to the product."
I quickly saw what the problem was which is that SDDC Manager is missing certain dormant license file (DLF) to decode several of the VCF license keys including the new vSAN TiB license which is only supported with vSphere 8.0 Update 3 and later.
In speaking with VCF Engineering, it turns out my assessment was correct and that VCF 5.2 does not currently understand the newer VCF license keys and users should just use the "License Later" feature by default. I have asked for our VCF 5.2 release notes to be updated to include this known issue so that others will not run into this problem.
With that said, since this error happens at the VERY end of the VCF deployment where everything has been deployed, I was really hoping I could find a quick workaround so that I could still use my VCF environment. It turns out an easy solution would be to just copy the required DLF from the vCenter Server Appliance (VCSA), since it officially supports the new license keys and that would allow SDDC Manager to finish the workflow and complete the deployment.
Below are the source DLF for both the new vSAN TiB license as well as the ESXi and vCenter Server license keys, which I also had problems decoding:
vSAN
- VCSA Source: /usr/lib/vmware-vpx/vsan-health/licenses/license-vsan-80-e16-c7-202201
- SDDC Manager Destination: /opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201
ESXi
- /usr/lib/vmware-vpx/licenses/site/license-esx-80-e27-c5-202201
- /opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201
vCenter Server
- /usr/lib/vmware-vpx/licenses/site/license-vc-80-e5-202201
- /opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201
The easiest way to transfer the required DLFs from above is the following:
Step 1 - SSH to SDDC Manager using the vcf account
Step 2 - SCP the three DLF files from your deployed VCSA to SDDC Manager which you can do by running the following commands:
scp root[at]VCSA_FQDN:/usr/lib/vmware-vpx/vsan-health/licenses/license-vsan-80-e16-c7-202201 /opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201 scp root[at]VCSA_FQDN:/usr/lib/vmware-vpx/licenses/site/license-esx-80-e27-c5-202201 /opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201 scp root[at]VCSA_FQDN:/usr/lib/vmware-vpx/licenses/site/license-vc-80-e5-202201 /opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201
Step 3 - Next, we need to update the ownership and permissions on the three files by running the following on SDDC Manager:
chown vcf_operationsmanager:vcf /opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201 chown vcf_operationsmanager:vcf /opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201 chown vcf_operationsmanager:vcf /opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201 chmod 740 /opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201 chmod 740/opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201 chmod 740 /opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201
Step 4 - Finally, we can head back over to the Cloud Builder UI and retry the operation and it should now succeed as SDDC Manager can now decode the license keys!
Jason Orchard says
hi William
any idea how to overcome this?
vcf@sddc-manager [ ~ ]$ chown vcf_operationsmanager:vcf /opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201
chown: cannot access '/opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201': Permission denied
vcf@sddc-manager [ ~ ]$ chown vcf_operationsmanager:vcf /opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201
chown: cannot access '/opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201': Permission denied
vcf@sddc-manager [ ~ ]$ chown vcf_operationsmanager:vcf /opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201
chown: cannot access '/opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201': Permission denied
vcf@sddc-manager [ ~ ]$ chmod 740 /opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201
chmod: cannot access '/opt/vmware/vcf/operationsmanager/dlfs/vsan/license-vsan-80-e16-c7-202201': Permission denied
vcf@sddc-manager [ ~ ]$ chmod 740/opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201
chmod: missing operand after ‘740/opt/vmware/vcf/operationsmanager/dlfs/esxi/license-esx-80-e27-c5-202201’
Try 'chmod --help' for more information.
vcf@sddc-manager [ ~ ]$ chmod 740 /opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201
chmod: cannot access '/opt/vmware/vcf/operationsmanager/dlfs/vcenter/license-vc-80-e5-202201': Permission denied
vcf@sddc-manager [ ~ ]$
vcf@sddc-manager [ ~ ]$ sudo su
[sudo] password for vcf
Sorry, user vcf is not allowed to execute '/usr/bin/su' as root on sddc-manager.vcf.sddc.lab.
vcf@sddc-manager [ ~ ]$
William Lam says
You need to “su” not use sudo
Sharath says
Hello William,
Have you stretched the VSAN cluster in your 5.2 lab?
Did you face any license key issues?
I was able to use the keys in parameter sheet and AZ1 deployed just fine with the provided license keys. But running into error while stretching the vsan cluster. It is not able validate the same keys which I used for AZ1.
We saw this kind of error in older version of VCF.
Description Validate vSAN Licenses
Progress Messages Current applied vSAN license doesn't support the required cpu count 8.
Error
Message: Current applied vSAN license doesn't support the required cpu count 8.
Remediation Message: Please apply proper vSAN license with neccessary cpu count.