WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / Automation / Automating Workload Management on vSphere with Tanzu

Automating Workload Management on vSphere with Tanzu

10.20.2020 by William Lam // 7 Comments

As promised, here is the complimentary solution to my existing Automated vSphere with Tanzu Lab Deployment Script, which will automatically deploy and configure the required infrastructure (vCenter Server Appliance, ESXi, vSAN and HAProxy VMs) so that you can quickly jump to enabling Workload Management on your vSphere Cluster.

FYI: Ben Corrie, one of the Engineers on the vSphere with Tanzu team recently published a vSphere with Tanzu 4-Part Deep Dive video series where he walks you through in deploying everything from scratch along with the concepts that should help you better understand how vSphere with Tanzu works. He is actually doing this in his own personal homelab and thought this might be useful to share with others. Kudos Ben and highly recommend folks check out his video if you new to vSphere with Tanzu and Kubernetes.


Enabling Workload Management is a manual step after the automated deployment script and as you know, I prefer to automate as much as I can. I have updated my existing PowerCLI Workload Management Module to now also support the new vSphere with Tanzu capability using HAProxy for networking instead of NSX-T. The module can be downloaded from PowerShell Gallery by simply running

Install-Module VMware.WorkloadManagement

Note: I should also mention that as of PowerCLI 12.1, there are now official PowerCLI cmdlets for configuring Workload Management which did not exists when vSphere 7.0 initially GA'ed and reason for this module. I have not personally taken a look at the cmdlets, but expect simliar set of user input will be required to enable.

Given the difference of input between an NSX-T and HAProxy based vSphere with Tanzu Cluster, I decided to create a new function called New-WorkloadManagement2 (very creative) and it accepts the following parameters:

  • ClusterName - Name of vSphere Cluster to enable Workload Management
  • TanzuvCenterServer - Hostname/IP of the new Tanzu vCenter Server that was deployed
  • TanzuvCenterServerUsername - Username to connect to new Tanzu vCenter Server
  • TanzuvCenterServerPassword - Password to connect to new Tanzu vCenter Server
  • TanzuContentLibrary - Name of the Tanzu Kubernetes Grid subscribed Content Library
  • ControlPlaneSize - Size of Control Plane VMs (TINY, SMALL, MEDIUM, LARGE)
  • HAProxyVMName - The display name of the HAProxy VM
  • HAProxyRootPassword - Root password for HAProxy VM
  • HAProxyUsername - HAProxy Control Plane Username (default: wcp)
  • HAProxyPassword - HAProxy Control Plane Password
  • HAProxyIPAddress - HAProxy Control Plane IP Address
  • HAProxyPort - HAProxy Control Plane port (default: 5556)
  • MgmtNetwork - Supervisor Management Network for Control Plane VMs (default: DVPG-Supervisor-Management-Network)
  • MgmtNetworkStartIP - Starting IP Address for Control Plane VMs (5 consecutive free addresses)
  • MgmtNetworkSubnet - Netmask for Management Network
  • MgmtNetworkGateway - Gateway for Management Network
  • MgmtNetworkDNS - DNS Server(s) to use for Management Network
  • MgmtNetworkDNSDomain - DNS Domain(s)
  • MgmtNetworkNTP - NTP Server(s)
  • WorkloadNetworkLabel - Workload Network label defined in vSphere with Tanzu (default: network-1)
  • WorkloadNetwork - Workload Network (default: DVPG-Workload-Network)
  • WorkloadNetworkStartIP - Starting IP Address for Workload VMs
  • WorkloadNetworkIPCount -Number of IP Addresses to allocate from starting from WorkloadNetworkStartIP
  • WorkloadNetworkSubnet - Subnet for Workload Network
  • WorkloadNetworkGateway - Gateway for Workload Network
  • WorkloadNetworkDNS - DNS Server(s) to use for Workloads
  • WorkloadNetworkServiceCIDR - K8S Service CIDR (default: 10.96.0.0/24)
  • StoragePolicyName - Name of VM Storage Policy to use for Control Plane VMs, Ephemeral Disks & Image Cache
  • HAProxyVMvCenterServer- Hostname/IP of the vCenter Server managing HAProxy VM to automatically retrieve CA certificate
  • HAProxyVMvCenterUsername - Username to connect to vCenter Server managing HAProxy VM to automatically retrieve CA certificate
  • HAProxyVMvCenterPassword - Password to connect to vCenter Server managing HAProxy VM to automatically retrieve CA certificate
  • LoadBalancerLabel - Load Balancer label defined in vSphere with Tanzu (default: tanzu-haproy-1)
  • LoadBalancerStartIP - Starting IP Address for HAProxy Load Balancer
  • LoadBalancerIPCount - Number of IP Addresses to allocate from starting from LoadBalancerStartIP
  • LoginBanner - Login message to show during kubectl login

At first glance, this may seem like a large amount of input and it is but that is what is needed when using the vSphere UI to enable Workload Management. I have streamlined some of the parameters and used defaults (assuming you used my deployment script). If not, you can always override the default values if you deployed vSphere with Tanzu using HAProxy the manual method as this ultimately just uses the vCenter Server REST API for Workload Management.

Here is an example of using the New-WorkloadManagement2 function:

$vSphereWithTanzuParams = @{
    ClusterName = "Workload-Cluster";
    TanzuvCenterServer = "tanzu-vcsa-1.cpbu.corp";
    TanzuvCenterServerUsername = "*protected email*";
    TanzuvCenterServerPassword = "VMware1!";
    TanzuContentLibrary = "TKG-Content-Library";
    ControlPlaneSize = "TINY";
    MgmtNetworkStartIP = "172.17.31.120";
    MgmtNetworkSubnet = "255.255.255.0";
    MgmtNetworkGateway = "172.17.31.1";
    MgmtNetworkDNS = @("172.17.31.5");
    MgmtNetworkDNSDomain = "cpbu.corp";
    MgmtNetworkNTP = @("5.199.135.170");
    WorkloadNetworkStartIP = "172.17.36.130";
    WorkloadNetworkIPCount = 20;
    WorkloadNetworkSubnet = "255.255.255.0";
    WorkloadNetworkGateway = "172.17.36.1";
    WorkloadNetworkDNS = @("172.17.31.5");
    WorkloadNetworkServiceCIDR = "10.96.0.0/24";
    StoragePolicyName = "tanzu-gold-storage-policy";
    HAProxyVMvCenterServer = "mgmt-vcsa-01.cpbu.corp";
    HAProxyVMvCenterUsername = "*protected email*";
    HAProxyVMvCenterPassword = "VMware1!";
    HAProxyVMName = "tanzu-haproxy-1";
    HAProxyIPAddress = "172.17.31.116";
    HAProxyRootPassword = "VMware1!";
    HAProxyUsername = "wcp";
    HAProxyPassword = "VMware1!";
    LoadBalancerStartIP = "172.17.36.2";
    LoadBalancerIPCount = 125
}
New-WorkloadManagement2 @vSphereWithTanzuParams

The function will first connect to the vCenter Server which was used to deploy the HAProxy VM and then use the Guest Operations API (Invoke-VMScript) to automatically extract the HAProxy certificate which is one of the required parameters. It will then disconnect from that vCenter Server and connect to the new deployed vCenter Server from the automation script to provide the rest of the required paraemters. If the enablement request was accepted, you should see a message that asks you to use the vSphere UI to monitor the progress and when you navigate to the Workload Management section of the vSphere UI, you should see the enablement process begin. If all information was provided correctly, this can take up to 30-40min to complete.

More from my site

  • Automated enablement of vSphere with Tanzu using vSphere Zones in vSphere 8
  • Automated Lab Deployment Script for vSphere with Tanzu using NSX Advanced Load Balancer (NSX ALB)
  • Automating HAProxy VM deployment with 3-NIC configuration using PowerCLI
  • vSphere with Tanzu using Intel Arc GPU
  • PowerCLI 13.0 on Photon OS

Categories // Automation, PowerCLI, VMware Tanzu Tags // PowerCLI, vSphere with Tanzu, Workload Management

Comments

  1. makubeck says

    10/21/2020 at 8:23 am

    HI William Lam,

    Your article helpful for setup my lab. But i have some question.
    From screenshot below link, After I tried deploy guestbook application on deployments menu at compute tab on namespce not show anything. Is this correct?

    https://ibb.co/hMjtv7M

    Reply
  2. Paul says

    10/23/2020 at 1:14 pm

    William, thank you for the very good and so often helpfull bolg! I tried to use the new cmdlet Enable-WMCluster to enable Workload Management, but since it is very new i can find some esamples. Would be VERY great if you could do an script also with the new PowerCLi commands. Thank you so much. Paul

    Reply
  3. Ivan Garcia says

    10/28/2020 at 2:53 am

    Hi there William, after all congrats for the amazing work!!
    I'd like ask you for help to be able to fix an issue while deploying my Tanzu home lab.

    The whole infrastructure is well deployed ... nested esx, networking, and Workload Management
    the problem is I can't access to the Control Plane IP Address (192.168.20.11) I've tried to re-deploy many times the whole infra and all the time getting success but I never get access to control plane IP

    The 192.168.20.x network is accessible by from the any VM in the same range in Nested Infraestructure and the parent one ... all IP's 10.24.168.50-52 (Supervisor Cluster) is reachable as well.

    But the HA Proxy Control Plane Address (192.168.20.11) isn't reachable.

    Does anyone can help me with this ?? I'm totally stuck!!

    Thanks in advance.

    Reply
  4. Paul says

    10/29/2020 at 9:59 am

    William, I tried your WorkloadManagement2 Skript, thank you very much. But I have this problem: I do not use subscribed content-library, but a local (following the procedure on the vmware site). I do this for performance reasons. If I use the GU everything goes fine. but your skript seem to accept only subscribed Content Libraries. Maybe you could fix this also to support local content libraries? would be great thank you so much!

    The error I get is this:
    ...
    Connecting to Tanzu vCenter Server to enable Workload Management ...
    TKG Content Library has not fully sync'ed, please try again later

    Reply
  5. Viky says

    11/03/2020 at 2:31 am

    Great post!
    How can I specify Workload and Magement network portgroups in this script?

    Thank you.
    Viky

    Reply
    • William Lam says

      11/03/2020 at 3:56 am

      Did you even read the blog post? Details are there 🙂

      Reply
  6. Andres Sanchez says

    11/19/2021 at 7:22 am

    Hi...Team.

    Lam congratuation for the post.!

    I have a question.....could you help me.

    implement tanzu, but it does not complete, when seeing events on the host I see error:

    "An error occurred while customizing VM SupervisorControlPlaneVM (2) . For details reference the log file in the guest OS."

    I also see warnings:
    Guest operation authentication failed for operation Start Program on Virtual machine SupervisorControlPlaneVM (2) .

    When viewing in kubernetes state:

    Error: "Resource Type Deployment, Identifier vmware-system-netop/vmware-system-netop-controller-manager is not found."

    Warning: "Node Kubernetes health status is Unknown"

    Info: "Kubernetes cluster health endpoint problem at . Details: Waiting for API Master IP assignment"
    "Customization operations of the guest OS for Master node VM with identifier vm-3166 is pending."

    followed by that I see that it enters a loop, where it eliminates the vm "SupervisorControlPlaneVM" and recreates them, and thus it remains creating and eliminating

    Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • How to disable the Efficiency Cores (E-cores) on an Intel NUC? 03/24/2023
  • Changing the default HTTP(s) Reverse Proxy Ports on ESXi 8.0 03/22/2023
  • NFS Multi-Connections in vSphere 8.0 Update 1 03/20/2023
  • Quick Tip - How to download ESXi ISO image for all releases including patch updates? 03/15/2023
  • SSD with multiple NVMe namespaces for VMware Homelab 03/14/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...