Automating Workload Management on vSphere with Tanzu

As promised, here is the complimentary solution to my existing Automated vSphere with Tanzu Lab Deployment Script, which will automatically deploy and configure the required infrastructure (vCenter Server Appliance, ESXi, vSAN and HAProxy VMs) so that you can quickly jump to enabling Workload Management on your vSphere Cluster.

FYI: Ben Corrie, one of the Engineers on the vSphere with Tanzu team recently published a vSphere with Tanzu 4-Part Deep Dive video series where he walks you through in deploying everything from scratch along with the concepts that should help you better understand how vSphere with Tanzu works. He is actually doing this in his own personal homelab and thought this might be useful to share with others. Kudos Ben and highly recommend folks check out his video if you new to vSphere with Tanzu and Kubernetes.

Enabling Workload Management is a manual step after the automated deployment script and as you know, I prefer to automate as much as I can. I have updated my existing PowerCLI Workload Management Module to now also support the new vSphere with Tanzu capability using HAProxy for networking instead of NSX-T. The module can be downloaded from PowerShell Gallery by simply running

Install-Module VMware.WorkloadManagement

Note: I should also mention that as of PowerCLI 12.1, there are now official PowerCLI cmdlets for configuring Workload Management which did not exists when vSphere 7.0 initially GA'ed and reason for this module. I have not personally taken a look at the cmdlets, but expect simliar set of user input will be required to enable.

Given the difference of input between an NSX-T and HAProxy based vSphere with Tanzu Cluster, I decided to create a new function called New-WorkloadManagement2 (very creative) and it accepts the following parameters:

• ClusterName - Name of vSphere Cluster to enable Workload Management
• TanzuvCenterServer - Hostname/IP of the new Tanzu vCenter Server that was deployed
• TanzuvCenterServerUsername - Username to connect to new Tanzu vCenter Server
• TanzuvCenterServerPassword - Password to connect to new Tanzu vCenter Server
• TanzuContentLibrary - Name of the Tanzu Kubernetes Grid subscribed Content Library
• ControlPlaneSize - Size of Control Plane VMs (TINY, SMALL, MEDIUM, LARGE)
• HAProxyVMName - The display name of the HAProxy VM
• HAProxyRootPassword - Root password for HAProxy VM
• HAProxyUsername - HAProxy Control Plane Username (default: wcp)
• HAProxyPassword - HAProxy Control Plane Password
• HAProxyIPAddress - HAProxy Control Plane IP Address
• HAProxyPort - HAProxy Control Plane port (default: 5556)
• MgmtNetwork - Supervisor Management Network for Control Plane VMs (default: DVPG-Supervisor-Management-Network)
• MgmtNetworkStartIP - Starting IP Address for Control Plane VMs (5 consecutive free addresses)
• MgmtNetworkSubnet - Netmask for Management Network
• MgmtNetworkGateway - Gateway for Management Network
• MgmtNetworkDNS - DNS Server(s) to use for Management Network
• MgmtNetworkDNSDomain - DNS Domain(s)
• MgmtNetworkNTP - NTP Server(s)
• WorkloadNetworkLabel - Workload Network label defined in vSphere with Tanzu (default: network-1)
• WorkloadNetwork - Workload Network (default: DVPG-Workload-Network)
• WorkloadNetworkStartIP - Starting IP Address for Workload VMs
• WorkloadNetworkIPCount -Number of IP Addresses to allocate from starting from WorkloadNetworkStartIP
• WorkloadNetworkSubnet - Subnet for Workload Network
• WorkloadNetworkGateway - Gateway for Workload Network
• WorkloadNetworkDNS - DNS Server(s) to use for Workloads
• WorkloadNetworkServiceCIDR - K8S Service CIDR (default: 10.96.0.0/24)
• StoragePolicyName - Name of VM Storage Policy to use for Control Plane VMs, Ephemeral Disks & Image Cache
• HAProxyVMvCenterServer- Hostname/IP of the vCenter Server managing HAProxy VM to automatically retrieve CA certificate
• HAProxyVMvCenterUsername - Username to connect to vCenter Server managing HAProxy VM to automatically retrieve CA certificate
• HAProxyVMvCenterPassword - Password to connect to vCenter Server managing HAProxy VM to automatically retrieve CA certificate
• LoadBalancerLabel - Load Balancer label defined in vSphere with Tanzu (default: tanzu-haproy-1)
• LoadBalancerStartIP - Starting IP Address for HAProxy Load Balancer
• LoadBalancerIPCount - Number of IP Addresses to allocate from starting from LoadBalancerStartIP
• LoginBanner - Login message to show during kubectl login

At first glance, this may seem like a large amount of input and it is but that is what is needed when using the vSphere UI to enable Workload Management. I have streamlined some of the parameters and used defaults (assuming you used my deployment script). If not, you can always override the default values if you deployed vSphere with Tanzu using HAProxy the manual method as this ultimately just uses the vCenter Server REST API for Workload Management.

Here is an example of using the New-WorkloadManagement2 function:

$vSphereWithTanzuParams = @{
    ClusterName = "Workload-Cluster";
    TanzuvCenterServer = "tanzu-vcsa-1.cpbu.corp";
    TanzuvCenterServerUsername = "*protected email*";
    TanzuvCenterServerPassword = "VMware1!";
    TanzuContentLibrary = "TKG-Content-Library";
    ControlPlaneSize = "TINY";
    MgmtNetworkStartIP = "172.17.31.120";
    MgmtNetworkSubnet = "255.255.255.0";
    MgmtNetworkGateway = "172.17.31.1";
    MgmtNetworkDNS = @("172.17.31.5");
    MgmtNetworkDNSDomain = "cpbu.corp";
    MgmtNetworkNTP = @("5.199.135.170");
    WorkloadNetworkStartIP = "172.17.36.130";
    WorkloadNetworkIPCount = 20;
    WorkloadNetworkSubnet = "255.255.255.0";
    WorkloadNetworkGateway = "172.17.36.1";
    WorkloadNetworkDNS = @("172.17.31.5");
    WorkloadNetworkServiceCIDR = "10.96.0.0/24";
    StoragePolicyName = "tanzu-gold-storage-policy";
    HAProxyVMvCenterServer = "mgmt-vcsa-01.cpbu.corp";
    HAProxyVMvCenterUsername = "*protected email*";
    HAProxyVMvCenterPassword = "VMware1!";
    HAProxyVMName = "tanzu-haproxy-1";
    HAProxyIPAddress = "172.17.31.116";
    HAProxyRootPassword = "VMware1!";
    HAProxyUsername = "wcp";
    HAProxyPassword = "VMware1!";
    LoadBalancerStartIP = "172.17.36.2";
    LoadBalancerIPCount = 125
}
New-WorkloadManagement2 @vSphereWithTanzuParams

The function will first connect to the vCenter Server which was used to deploy the HAProxy VM and then use the Guest Operations API (Invoke-VMScript) to automatically extract the HAProxy certificate which is one of the required parameters. It will then disconnect from that vCenter Server and connect to the new deployed vCenter Server from the automation script to provide the rest of the required paraemters. If the enablement request was accepted, you should see a message that asks you to use the vSphere UI to monitor the progress and when you navigate to the Workload Management section of the vSphere UI, you should see the enablement process begin. If all information was provided correctly, this can take up to 30-40min to complete.