WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

Quick Tip - Setting up Kubernetes using Containerd on Photon OS

07.28.2021 by William Lam // 1 Comment

As part of the VMware Event Broker Appliance (VEBA) project, I was recently evaluating a newer version of Kubernetes (v1.21.3) and also switching the container runtime from Docker to Containerd. I figured this probably should not be that difficult, especially since we are already use Containerd within Tanzu Kubernetes Grid (TKG) which is our commercial Kubernetes (k8s) offering and that base OS is VMware Photon OS. How hard could this be, right!? (famous last words) 😂

We use kubeadm to setup K8s and read in a very basic configuration file and after following the official K8s instructions for prepping the environment to use containerd, I was surprised when I ran into the following error:

Unfortunately, an error has occurred:
timed out waiting for the condition

This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.

Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock logs CONTAINERID'

error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

Unfortunately, this lead me down a huge rat hole of troubleshooting and trying various configurations and suggestions from the Internet. Ultimately, none of the suggested solutions solved my problem. After exhausting all my options and spending more time that I would like to admit, I decided to ask in the Kubernetes Slack community to see if anyone might have an idea. There were not any specific suggestions that helped me understand the issue further but there was a question about how Containerd came to be on the system and that gave me one more thing to try.

Both Photon OS 3.0 and 4.0 ships with Containerd and after installing the desired kubeadm, kubectl and kubelet, I had wrongfully assumed that the version of Containerd would simply work.

[Read more...]

Categories // Kubernetes Tags // Kubernetes, Photon OS

Heads Up - Verify the SSL certificate trust for your vSphere with Tanzu Content Library

07.28.2021 by William Lam // 1 Comment

I just learned that SSL certificate for VMware's vSphere with Tanzu Content Library (https://wp-content.vmware.com/v2/latest/lib.json) had just been updated a few days ago and this will have an impact for anyone who had subscribed to the Content Library prior to the certificate update.

Since I had setup the subscribed Content Library several months back, all the OVAs at the time were already sync'ed and there are no immediate errors when the "Fetch Content Library" task is performed (which will be fixed in a future release). However, I was seeing some strange issues with deploying specific versions of Kubernetes and I did not think much of it and deploying another version was fine, so I figured maybe it was just my setup. I also had another lab, so I ended up using that environment most recently.

It is only until you click on a specific Content Library Item and perform a manual sync will you see the following error, which indicates you are affected:

A general system error occurred: HTTP request error: cannot authenticate SSL certificate for host wp-content.vmware.com.


The quickest way to check whether you are affected is by looking at the configured SSL Thumbprint of your subscribed Content Library and comparing that to the vSphere with Tanzu Content Library endpoint.


Unfortunately, the configured SSL Thumbprint for the subscribed Content Library is not visible in the vSphere UI, but this information is available in the vSphere Content Library API.

I just put together this quick PowerCLI snippet which will retrieve the SSL Thumbprint for your subscribed Content Library and compare to the current thumbprint. If it does not match, you will get an error message printing out the current SSL Thumbprint.

Connect-CisServer -Server vcsa.primp-industries.local -User *protected email* -Password VMware1!

$SubscribedCLName = "TKG-Content-Library"
$TKGCLThumbprint = "01:8D:FD:13:A6:9E:CA:AC:CB:7C:67:18:C1:47:11:8C:64:91:5D:C9"

$contentLibraryService = Get-CisService com.vmware.content.library
$LibraryIDs = $contentLibraryService.list()

foreach($libraryID in $LibraryIDs) {
        $library = $contentLibraryService.get($libraryID)
        if($library.name -eq $SubscribedCLName) {
            $ContentLibrary = $Library
            break
        }
}

if($ContentLibrary.subscription_info.ssl_thumbprint -ne $TKGCLThumbprint.toLower()) {
    Write-Error "SSL Thumbprint $($ContentLibrary.subscription_info.ssl_thumbprint) for $SubscribedCLName does not currently match!`n"
}

The fix is straight forward, simply edit the settings of your subscribed Content Library, do not make any changes and then click on OK. Since the configured SSL Thumbprint no longer matches the hosted Content Library, you will be prompted with an action to confirm the new thumbprint and then you can save the settings.


As of right now, the SSL Certificate for the hosted vSphere with Tanzu Content Library is valid until July 7, 2022 and I expect that VMware will replace the TLS certificate prior to that date and this operation will need to be performed again. Since this issue was initially reported internally, I have also asked to see if an official VMware KB to be published.

Categories // VMware Tanzu Tags // content library, vSphere Kubernetes Service

Quick Tip - Dynamic OVF input properties using DeploymentOptions

07.23.2021 by William Lam // Leave a Comment

I first talked about OVF DeploymentOptions back in 2013, which enables an OVF/OVA author to define a set of deployment profiles (e.g. small, medium, large) which then automatically translate to a pre-defined set of compute, network and storage configurations when deploying an OVF/OVA. There are a number VMware Appliances that takes advantage of this OVF capability, the most well known is the vCenter Server Appliance (VCSA) when it prompts you to select the size of the VCSA that you wish to deploy.


Now although the primary driver for DeploymentOptions is for having out of the box resource configurations when deploying an OVF/OVA, it can also be used to control which OVF properties are shown to end users for input based on the selected deployment option.

I recently had a need for this capability and it was only after taking another look at the OVF specification, did I realize this was possible through the use of DeploymentOptions. Below is a quick example on how you can control specific OVF properties. Imagine, we have three deployment options: Development, Stage and Production which maps to the following DeploymentOption IDs: dev, stage and prod

[Read more...]

Categories // Automation, OVFTool Tags // DeploymentOptionSection, ova, ovf

  • « Previous Page
  • 1
  • …
  • 135
  • 136
  • 137
  • 138
  • 139
  • …
  • 561
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • VMware Flings is now available in Free Downloads of Broadcom Support Portal (BSP) 05/19/2025
  • VMUG Connect 2025 - Minimal VMware Cloud Foundation (VCF) 5.x in a Box  05/15/2025
  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...