WilliamLam.com

VCF 9.1.1 - New UI Support for Deploying VCF Management Services (VCFMS) & VCF Automation (VCFA) on Non-Management Networks

by // 2 Comments

When using the VCF 9.1.0 Installer to deploy VCF Management Services (VCFMS) and/or VCF Automation (VCFA), users can choose to deploy these components on either the default VM Management Network or an alternative network.


However, deploying VCFMS to a non-management network as part of an upgrade workflow, or VCFA during Day-N operations, requires using the VCF APIs (as explained here and here, respectively), as these workflows are not supported through the UI.

I was pleasantly surprised to recently learn from a colleague that with the latest VCF 9.1.1 release, the UI workflows have been enhanced! Instead of requiring the SDDC Manager or Fleet LCM API, you can now deploy these components to a non-management network directly from the UI.

Here is a screenshot when upgrading your environment to VCF 9.1.1 and when deploying VCFMS, you will now have an option to select a different network.


Here is a screenshot after upgrading you environment to VCF 9.1.1 and you have no deployed VCFA and as part of the Day-N operation, you will now have an option to select a different network.

Quick Tip: Reducing High CPU Utilization in VCF Automation (VCFA) on AMD Zen4/Zen5 CPUs

by // 5 Comments

A couple of weeks ago, I received an interesting email from a reader who had observed high CPU utilization after upgrading VCF Automation (VCFA) to the latest VCF 9.1 release. They were running VCF 9.1 on the popular Minisforum MS-A2 platform, which uses AMD Zen4/Zen5 (Ryzen) consumer processors which are not supported on the Broadcom Compatibility Guide (BCG). As I have covered previously (here and here), these processors have been known to exhibit slower entropy generation that can lead to higher than expected CPU utilization without the appropriate optimizations.

With the help of Claude to analyze the issue, the reader arrived at the following conclusion:

Five JVM-based services were dying repeatedly during startup because FIPS_MODE=strict was slowing down TLS handshakes (both database and internal service-to-service HTTPS) enough to occasionally exceed connection timeouts. Every restart meant a full JVM boot: Spring context initialization, Liquibase migrations, Hibernate, index rebuilding, which is expensive. The crash loop itself became the CPU load. Setting FIPS_MODE=disabled on the affected deployments stopped the crash loop entirely, and CPU usage returned to near-idle.

Interestingly, around the same time, I also heard from an internal colleague who had independently observed that disabling FIPS for specific VCFA services significantly reduced CPU utilization. Since FIPS is enabled by default for VCF 9.1 deployments, and there are known performance implications with FIPS and OpenSSL 3.x, these effects can be amplified on AMD Zen4/Zen5 (Ryzen) consumer processors due to their slower entropy generation, resulting in higher than expected CPU utilization.

Fortunately, both individuals had identified workarounds for the issue, but their recommendations did not completely overlap. Before sharing the guidance more broadly, I wanted to spend some time validating the solution to identify the minimal set of changes required to consistently achieve the same outcome.

[Read more...]

Quick Tip: How to Identify Which Kubernetes Cluster Owns a vSphere Container Volume (PV)

by // Leave a Comment

I was recently cleaning up some orphaned vSphere Container Volumes and while I was able to compare the Persistent Volumes (PV) IDs against the ones currently in use by a couple of Kubernetes clusters, I wanted to be 100% sure before removing them, especially since I planned bulk delete using PowerCLI’s Remove-CnsVolume cmdlet.

You can view all provisioned vSphere Container Volumes in the vSphere UI by navigating to vSphere Cluster->Monitor->Cloud Native Storage->Container Volumes. In addition to the volume details, you also get access to all of the associated metadata and labels, very similar to what you would see using kubectl, which is quite nice!


While reviewing the PV IDs in the UI, I noticed an attribute named Kubernetes Cluster, which appeared to identify the Kubernetes (K8s) cluster that owned the volume. How do I find this Kubernetes Cluster ID?

[Read more...]