WilliamLam.com

Enhancements to VMware Tools 12 for Container Application Discovery in vSphere 

by // 2 Comments

VMware Tools 12 was just released and it adds a number of new features including support for Windows 11 and Windows Server 2022, Salt Stack Minion deployment and the use of OpenSSL 3.0 library to just name a few.

One additional feature that is quite interesting is the enhancement to the Application Discovery feature that was shipped with VMware Tools 11 which provides organizations with additional visibility of the running processes within a VM.

With VMware Tools 12, we now have a more granular method for discovering container-based processes (Docker or Containerd) running within a Linux VM, which is pretty cool if you ask me!


Similiar to the Application Discovery feature, a new VM guestinfo variable has been introduced called guestinfo.vmtools.containerinfo that will be populated with the list of running containers. By default, the polling interval is every 6 hours with a default of listing the first 100 containers, these and other settings can be adjusted which you can find in the official VMware documentation.

Simliar to the Application Discovery feature, I have also updated my PowerCLI function Get-VMApplicationInfo.ps1 to include this additional functionality for users that would like to extract this information and I have created a new function called Get-VMContainerInfo, which you see how it functions in the screenshot above. In addition to console output, you can also save the information in both CSV and JSON format.

Quick Tip - Setting up Kubernetes using Containerd on Photon OS

by // 1 Comment

As part of the VMware Event Broker Appliance (VEBA) project, I was recently evaluating a newer version of Kubernetes (v1.21.3) and also switching the container runtime from Docker to Containerd. I figured this probably should not be that difficult, especially since we are already use Containerd within Tanzu Kubernetes Grid (TKG) which is our commercial Kubernetes (k8s) offering and that base OS is VMware Photon OS. How hard could this be, right!? (famous last words) 😂

We use kubeadm to setup K8s and read in a very basic configuration file and after following the official K8s instructions for prepping the environment to use containerd, I was surprised when I ran into the following error:

Unfortunately, an error has occurred:
timed out waiting for the condition

This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.

Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock logs CONTAINERID'

error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

Unfortunately, this lead me down a huge rat hole of troubleshooting and trying various configurations and suggestions from the Internet. Ultimately, none of the suggested solutions solved my problem. After exhausting all my options and spending more time that I would like to admit, I decided to ask in the Kubernetes Slack community to see if anyone might have an idea. There were not any specific suggestions that helped me understand the issue further but there was a question about how Containerd came to be on the system and that gave me one more thing to try.

Both Photon OS 3.0 and 4.0 ships with Containerd and after installing the desired kubeadm, kubectl and kubelet, I had wrongfully assumed that the version of Containerd would simply work.

[Read more...]

Packer reference for VMware Harbor Virtual Appliance

by // 2 Comments

I recently had a need to setup a container registry for a project that I was working on and Harbor was of course my default choice. Although Harbor is pretty easy to setup, I did not want to manually go through the installation each time I needed Harbor and I figured it was time to build my own Harbor Virtual Appliance (OVA), just like I have shown in the past with these reference implementations here and here.

For those interested, you can find the reference implementation for building a Harbor Virtual Appliance at https://github.com/lamw/harbor-appliance

Note: For internal VMware Employees, if you prefer not to build the appliance yourself, drop me an email or DM and I can provide you with the link to the Harbor Appliance OVA.

When deploying the Harbor Appliance, you will find the basic OVF properties that I have encoded including networking, credentials, debugging and advanced settings. Hopefully should be pretty straight forward for anyone who has deployed an OVA before to vSphere.

[Read more...]