One of the last things on my to-do list after creating my Automated vSphere 7 and vSphere with Kubernetes Lab Deployment Script which is still the quickest and most reliable way to have a fully deployed and configured environment to try out vSphere with Kubernetes using Nested ESXi, was to also automate the enablement of Workload Management for a given vSphere Cluster.
There are two new vCenter Server REST APIs to be aware of as it pertains to vSphere with Kubernetes:
- namespaces = Manages the lifecycle and access control to a vSphere Namespace
- namespace-management = Despite the name, this refers to lifecycle and management of a Workload Management Cluster
I also have to mention that Vikas Shitole, who works on vCenter Server, has fantastic blog series covering various parts of the new vSphere with Kubernetes API along with Python examples if you want to dive further. Since Vikas has done a great job covering Python, I figure I will demonstrate how to consume these new vSphere with Kubernetes API using PowerCLI, which many of our customers use to automate.
I have created a new WorkloadManagement.psm1 PowerCLI module which includes following functions:
- Get-WorkloadManagement
- New-WorkloadManagement
- Remove-WorkloadManagement
Below are the two steps required to get started with the Workload Management PowerCLI Module.
Step 1 - Install the WorkloadManagement PowerCLI Module by running the following command:
Install-Module VMware.WorkloadManagement.psm1
Step 2 - A connection to the vCenter REST API endpoint using the Connect-CisServer cmdlet is required for enabling and disabling Workload Management Cluster
Connect-CisServer -Server pacific-vcsa-2.cpbu.corp -User *protected email* -Password VMware1!
A connection to vCenter Server using Connect-VIServer cmdlet is only required if you wish to retrieve information about an existing Workload Management Cluster
Connect-VIServer -Server pacific-vcsa-2.cpbu.corp -User *protected email* -Password VMware1!
Enable Workload Management
To enable Workload Management for a vSphere Cluster, you will use the New-WorkloadManagement function which accepts the exact same input as the Workload Management UI within vCenter Server. You can monitor the progress of the removal by using the vSphere UI. Below are the supported paraemters and their definitions.
ClusterName | Name of vSphere Cluster to enable Workload Management |
---|---|
ControlPlaneSize | Size of Control Plane VMs (TINY;SMALL;MEDIUM;LARGE) |
MgmtNetwork | Management Network for Control Plane VMs |
MgmtNetworkStartIP | Starting IP Address for Control Plane VMs (5 consecutive free addresses) |
MgmtNetworkSubnet | Netmask for Management Network |
MgmtNetworkGateway | Gateway for Management Network |
MgmtNetworkDNS | DNS Server(s) to use for Management Network |
MgmtNetworkDNSDomain | DNS Domain(s) |
MgmtNetworkNTP | NTP Server(s) |
WorkloadNetworkVDS | Name of vSphere 7 Distributed Virtual Switch (VDS) configured with NSX-T |
WorkloadNetworkEdgeCluster | Name of NSX-T Edge Cluster |
WorkloadNetworkDNS | DNS Server(s) to use for Workloads |
WorkloadNetworkPodCIDR | K8s POD CIDR (default: 10.244.0.0/21) |
WorkloadNetworkServiceCIDR | K8S Service CIDR (default: 10.96.0.0/24) |
WorkloadNetworkIngressCIDR | CIDR for Workload Ingress (recommend /27 or larger) |
WorkloadNetworkEgressCIDR | CIDR for Workload Egress (recommend /27 or larger) |
ControlPlaneStoragePolicy | Name of VM Storage Policy to use for Control Plane VMs |
EphemeralDiskStoragePolicy | Name of VM Storage Policy to use for Ephemeral Disk |
ImageCacheStoragePolicy | Name of VM Storage Policy to use for Image Cache |
Here is an example of using the New-WorkloadManagement function:
$workloadManagementParameters = @{
ClusterName = "Workload-Cluster";
ControlPlaneSize = "TINY";
MgmtNetwork = "DVPG-Management Network";
MgmtNetworkStartIP = "172.17.36.51";
MgmtNetworkSubnet = "255.255.255.0";
MgmtNetworkGateway = "172.17.36.253";
MgmtNetworkDNS = "172.17.31.5";
MgmtNetworkDNSDomain = "cpub.corp";
MgmtNetworkNTP = "5.199.135.170";
WorkloadNetworkVDS = "Pacific-VDS";
WorkloadNetworkEdgeCluster = "Edge-Cluster-01";
WorkloadNetworkDNS = "172.17.31.5";
WorkloadNetworkIngressCIDR = "172.17.36.64/27";
WorkloadNetworkEgressCIDR = "172.17.36.96/27";
ControlPlaneStoragePolicy = "pacific-gold-storage-policy";
EphemeralDiskStoragePolicy = "pacific-gold-storage-policy";
ImageCacheStoragePolicy = "pacific-gold-storage-policy";
}
New-WorkloadManagement @workloadManagementParameters
Note: In the snippet above, I am using "splatting" which I just learned was possible with parameter but the screenshot uses the standard function parameters, both methods work.
Retrieve Workload Management
To retrieve all Workload Management Clusters, use the Get-WorkloadManagement function.
Get-WorkloadManagement
To include additional usage information pertaining to cpu, memory and storage you can append the -Stats option.
Get-WorkloadManagement -Stats
Disable Workload Management
To disable Workload Management on existing vSphere Cluster which has it enabled, use the Remove-WorkloadManagement function and provide the name of the vSphere Cluster. You can monitor the progress of the removal by using the vSphere UI.
Remove-WorkloadManagement -ClusterName "Workload-Cluster"
Florian Grehl @virten says
Thank you, very nice! Works for me with a small modification:
The NTP needs to be a string, or the deployment will fail:
Line 86: [Parameter(Mandatory=$True)][string[]]$MgmtNetworkNTP
The problem "Address . is not a valid IP address or FQDN." can be seen in wcpsvc.log:
"master_NTP_servers": { "OPTIONAL": [ "1", "8", "5", ".", "1", "2", "0", ".", "2", "2", ".", "1", "2" ] },
Also, the hardcoded POD and Service CIDRs are for the TINY deployment only. The subnet bit is increased by one for each larger deployment.
You can get the defaults with GET /api/vcenter/namespace-management/cluster-size-info
elnemesisdivina says
I found that if you don't use and equal sign in between for ClusterName "Workload-Cluster" for the splatting it fails.
my 2 pesos. 🙂
Paul Knoll says
Hi William, thanks a lot for your skript. I am using them since a year ...
One information to you. I use New-Workloadmanagement2 for teaching and learning a lot. It works perfect up to Powershell 7.1.5. Yesterday I tried to upgrade to PS 7.2, then the skript gives a lot of error. but wiht 7.1.5 it works great.
I just wanted to inform you
Best regards
Paul