WilliamLam.com

Quick Tip - vSphere with Tanzu fails to sync Content Library with 500 Internal Server Error

by // Leave a Comment

While setting up a new vSphere with Tanzu environment (which can run with just 32GB of memory), I ran into a really strange issue where my vSphere Content Library templates were not being picked up by the VM Service. I was going insane as I have configured this a number of times and I have never ran into this particulare issue before. I thought maybe it was a configuration problem but the enablement of vSphere with Tanzu was 100% successful and everything was showing green.

While looking at the vmware-system-vmop-controller-manager container log, I noticed that the VM Service can see the template but it just fails to extract and process it and throws a 500 Internal Server Error message:

E0917 12:08:23.060929 1 content_library_provider.go:275] vsphere/contentlibrary "msg"="error extracting the OVF envelope from the library item" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "contentLibraryUUID"="a298369b-7239-4b1c-988f-d361e5a001d6" "itemName"="ubuntu-22.04-custom-image"
E0917 12:08:23.060984 1 content_library_provider.go:275] vsphere/contentlibrary "msg"="error extracting the OVF envelope from the library item" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "contentLibraryUUID"="1bfd8950-c846-46cf-bd50-55150a16bab3" "itemName"="photon-ova"
E0917 12:08:23.060998 1 contentsource_controller.go:203] controllers/ContentSource "msg"="failed to get VirtualMachineImage from content library" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "clProviderName"="a298369b-7239-4b1c-988f-d361e5a001d6" "clProviderUUID"="a298369b-7239-4b1c-988f-d361e5a001d6"
E0917 12:08:23.061011 1 contentsource_controller.go:203] controllers/ContentSource "msg"="failed to get VirtualMachineImage from content library" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "clProviderName"="1bfd8950-c846-46cf-bd50-55150a16bab3" "clProviderUUID"="1bfd8950-c846-46cf-bd50-55150a16bab3"
E0917 12:08:23.061032 1 contentsource_controller.go:401] controllers/ContentSource "msg"="Error in syncing image from the content provider" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="1bfd8950-c846-46cf-bd50-55150a16bab3"
E0917 12:08:23.061079 1 controller.go:317] controller/contentsource "msg"="Reconciler error" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="1bfd8950-c846-46cf-bd50-55150a16bab3" "namespace"="" "reconciler group"="vmoperator.vmware.com" "reconciler kind"="ContentSource"
E0917 12:08:23.061123 1 contentsource_controller.go:401] controllers/ContentSource "msg"="Error in syncing image from the content provider" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="a298369b-7239-4b1c-988f-d361e5a001d6"
E0917 12:08:23.061165 1 controller.go:317] controller/contentsource "msg"="Reconciler error" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="a298369b-7239-4b1c-988f-d361e5a001d6" "namespace"="" "reconciler group"="vmoperator.vmware.com" "reconciler kind"="ContentSource"

I was running out of ideas and things to try and I decided to look at the content library configuration to see if I had missed something.

[Read more...]

Quick Tip - Dynamic OVF input properties using DeploymentOptions

by // Leave a Comment

I first talked about OVF DeploymentOptions back in 2013, which enables an OVF/OVA author to define a set of deployment profiles (e.g. small, medium, large) which then automatically translate to a pre-defined set of compute, network and storage configurations when deploying an OVF/OVA. There are a number VMware Appliances that takes advantage of this OVF capability, the most well known is the vCenter Server Appliance (VCSA) when it prompts you to select the size of the VCSA that you wish to deploy.


Now although the primary driver for DeploymentOptions is for having out of the box resource configurations when deploying an OVF/OVA, it can also be used to control which OVF properties are shown to end users for input based on the selected deployment option.

I recently had a need for this capability and it was only after taking another look at the OVF specification, did I realize this was possible through the use of DeploymentOptions. Below is a quick example on how you can control specific OVF properties. Imagine, we have three deployment options: Development, Stage and Production which maps to the following DeploymentOption IDs: dev, stage and prod

[Read more...]

Why does Deploy OVF Template operation show vpxd-extension-[uuid]?

by // Leave a Comment

A question that I had received awhile ago from a customer was how to identify the specific user(s) who have deployed an OVF/OVA? Customers can easily do this by leveraging vCenter Serve Events, which are extremely rich with information that can help answer this and many other questions you might have in your vSphere enviornment.


The first challenge that you will find is that an OVF/OVA import operation is mapped to a generic vCenter TaskEvent, which will require some additional filtering. Secondly, depending on the method that was used to deploy the OVF/OVA such as the vSphere UI or using Automation tools like OVFTool or PowerCLI, you will also slightly diffrent behaviors in terms of the vCenter Server Events that are emitted.

If you deploy an OVF/OVA using the vSphere UI, you may have noticed there are actually two vCenter tasks which are displayed and running simultaneously as shown in the screenshot above. The first task is called "Import OVF package" and you will see that this task is associated with the actual user who initiated the import. The second task called "Deploy OVF template" is then associated with a vCenter system account that handles the actual deployment which will show up with a vpxd-extension-[uuid] username. This occurs because the user who is performing the import is not interacting directly with with vCenter Server, but rather through the vSphere UI which uses a system account to then communicate the operation to vCenter Server.

For this reason, when an OVF/OVA is imported through the vSphere UI, you will need to look at the TaskEvent and filter for the initial import operation. If an OVF/OVA is imported using the vSphere API using something like OVFTool, PowerCLI or any other vSphere SDK, then you will only see the Deploy OVF Template operation and the user associated with that operation is the person who initiated the import.

Using PowerCLI and the Get-VIEvent cmdlet, we can easily filter out these two types of TaskEvents.

Get-VIEvent | where {$_.GetType().Name -eq "TaskEvent" -and $_.FullFormattedMessage -eq "Task: Import OVF package"}

Info                 : VMware.Vim.TaskInfo
Key                  : 2036862
ChainId              : 2036862
CreatedTime          : 4/23/2021 9:30:36 AM
UserName             : vsphere.local\william
Datacenter           : VMware.Vim.DatacenterEventArgument
ComputeResource      : VMware.Vim.ComputeResourceEventArgument
Host                 :
Vm                   :
Ds                   :
Net                  :
Dvs                  :
FullFormattedMessage : Task: Import OVF package
ChangeTag            :

Get-VIEvent | where {$_.GetType().Name -eq "TaskEvent" -and $_.FullFormattedMessage -eq "Task: Import OVF package"}

Info                 : VMware.Vim.TaskInfo
Key                  : 2036869
ChainId              : 2036869
CreatedTime          : 4/23/2021 9:31:01 AM
UserName             : VSPHERE.LOCAL\vpxd-extension-767f8016-870d-4a98-a457-8247454fa759
Datacenter           : VMware.Vim.DatacenterEventArgument
ComputeResource      : VMware.Vim.ComputeResourceEventArgument
Host                 :
Vm                   :
Ds                   :
Net                  :
Dvs                  :
FullFormattedMessage : Task: Deploy OVF template
ChangeTag            :