WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple
You are here: Home / VMware Tanzu / Quick Tip - vSphere with Tanzu fails to sync Content Library with 500 Internal Server Error

Quick Tip - vSphere with Tanzu fails to sync Content Library with 500 Internal Server Error

09.19.2022 by William Lam // Leave a Comment

While setting up a new vSphere with Tanzu environment (which can run with just 32GB of memory), I ran into a really strange issue where my vSphere Content Library templates were not being picked up by the VM Service. I was going insane as I have configured this a number of times and I have never ran into this particulare issue before. I thought maybe it was a configuration problem but the enablement of vSphere with Tanzu was 100% successful and everything was showing green.

While looking at the vmware-system-vmop-controller-manager container log, I noticed that the VM Service can see the template but it just fails to extract and process it and throws a 500 Internal Server Error message:

E0917 12:08:23.060929 1 content_library_provider.go:275] vsphere/contentlibrary "msg"="error extracting the OVF envelope from the library item" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "contentLibraryUUID"="a298369b-7239-4b1c-988f-d361e5a001d6" "itemName"="ubuntu-22.04-custom-image"
E0917 12:08:23.060984 1 content_library_provider.go:275] vsphere/contentlibrary "msg"="error extracting the OVF envelope from the library item" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "contentLibraryUUID"="1bfd8950-c846-46cf-bd50-55150a16bab3" "itemName"="photon-ova"
E0917 12:08:23.060998 1 contentsource_controller.go:203] controllers/ContentSource "msg"="failed to get VirtualMachineImage from content library" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "clProviderName"="a298369b-7239-4b1c-988f-d361e5a001d6" "clProviderUUID"="a298369b-7239-4b1c-988f-d361e5a001d6"
E0917 12:08:23.061011 1 contentsource_controller.go:203] controllers/ContentSource "msg"="failed to get VirtualMachineImage from content library" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "clProviderName"="1bfd8950-c846-46cf-bd50-55150a16bab3" "clProviderUUID"="1bfd8950-c846-46cf-bd50-55150a16bab3"
E0917 12:08:23.061032 1 contentsource_controller.go:401] controllers/ContentSource "msg"="Error in syncing image from the content provider" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="1bfd8950-c846-46cf-bd50-55150a16bab3"
E0917 12:08:23.061079 1 controller.go:317] controller/contentsource "msg"="Reconciler error" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="1bfd8950-c846-46cf-bd50-55150a16bab3" "namespace"="" "reconciler group"="vmoperator.vmware.com" "reconciler kind"="ContentSource"
E0917 12:08:23.061123 1 contentsource_controller.go:401] controllers/ContentSource "msg"="Error in syncing image from the content provider" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="a298369b-7239-4b1c-988f-d361e5a001d6"
E0917 12:08:23.061165 1 controller.go:317] controller/contentsource "msg"="Reconciler error" "error"="POST https://vcsa.tanzu.local:443/rest/com/vmware/content/library/item/download-session: 500 Internal Server Error" "name"="a298369b-7239-4b1c-988f-d361e5a001d6" "namespace"="" "reconciler group"="vmoperator.vmware.com" "reconciler kind"="ContentSource"

I was running out of ideas and things to try and I decided to look at the content library configuration to see if I had missed something.

While creating the content library, I remember clicking really fast and I had checked the Apply Security Policy box, which I assumed was a good thing and would not cause any issues?


It turns out, this was the culprit! The new default OVF security policy was introduced back in vSphere 7.0 Update 3 and provides the following functionality:

Starting with vSphere 7.0 Update 3, you can protect the OVF items by applying default OVF security policy to a content library. The OVF security policy enforces strict validation on OVF items when you deploy or update the item, import items, or synchronize OVF and OVA templates. To make sure that the OVF and OVA templates are signed by a trusted certificate, you can add the OVF signing certificate from a trusted CA.

Under normal circumstances, this would not be an issue, especially if you are using VMware's vSphere with Tanzu content library. However, because the content library I had created contained a custom OVF template and it was not signed, it failed the strict validation which resulted in the VM Service not being able to discover and process my OVF template.

Long story short, be careful when enabling default options, even if you think they it can be helpful and make sure you really understand how feature works being doing so. I realized that this was new feature in vSphere 7.0 Update 3 and since I had not come across it before, I took the conservative approach of simply enabling it and wasted several hours of debugging. I think a nice enhancement in the logs would be to state that OVF item does not meet the strict security policy defined for content library, I think that would have really helped and directed me to right place to look.

More from my site

  • How to download offline copy of the Tanzu Kubernetes releases (TKr) Content Library?
  • Quick Tip - Correctly naming TKR's in Local Content Library for vSphere with Tanzu in vSphere 8
  • Heads Up - Verify the SSL certificate trust for your vSphere with Tanzu Content Library
  • Hidden OVF 2.0 capablity found in the vSphere Content Library
  • Quickly deploying vSphere IaaS (formerly vSphere with Tanzu) Control Plane Services via YAMLs

Categories // VMware Tanzu, vSphere 7.0, vSphere 8.0 Tags // content library, ovf, vSphere Kubernetes Service

Thanks for the comment!Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...