WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / Automation / How to create a kubernetes service account for vSphere with Tanzu?

How to create a kubernetes service account for vSphere with Tanzu?

11.29.2021 by William Lam // 1 Comment

Before you can interact and consume resources from a vSphere with Tanzu enabled cluster, users must first login and one way to accomplish this is by using the kubectl-vsphere plugin.

Once authenticated, a JWT (JSON Web Token), pronounced jot token, will be issued along with other values which will be appended to your local ~/.kube/config file. Users will then be able to perform kubectl operations based on the roles they have been assigned for a given vSphere Namespace. In case you did not know, these JWT tokens are only valid for 10 hours and after that, you will need to login again to retrieve a new JWT token.

We can also confirm this by decoding our JWT token found within the ~/.kube/config file and using jwt.io website. Once decoded, we can see when the token was issued using iat (Issued At) and when the token will expired using exp (Expiration Time) as shown in the screenshot below.

The default 10 hour expiry is currently not configurable which can be a challenge for anyone looking to setup unattended automation or GitOps with vSphere with Tanzu.

An alternative solution is to create a Kubernetes (k8s) service account, which by default does not contain a token expiry. Using this information and my recent Deep Dive into vSphere Namespace Roles, I was able to create a service account that can perform the same set of vSphere with Tanzu operations without having to re-login every 10 hours.

Note (06/07/22) - The "Edit" vSphere Namespace Role now includes the ability to create K8s service account and rolebinding without having to go into Supervisor Cluster Control Plane VM

Step 1 - SSH to the VCSA and then run the following script to retrieve the Supervisor Cluster Control Plane VM credentials:

/usr/lib/vmware-wcp/decryptK8Pwd.py

Step 2 - SSH to the IP Address using root username and the password provided from the previous command

Step 3 - Create a new K8s service account that is scoped to the specific vSphere Namespace. In my example, service account name is called tanzu-svc and it is scoped to primp-industries vSphere Namespace.

SA_NAME="tanzu-svc"
SV_NAMESPACE="primp-industries"

kubectl -n ${SV_NAMESPACE} create serviceaccount ${SA_NAME}

Step 4a - Create a ClusterRoleBinding for either the view or edit vSphere Namespace role for our service account.

SA_PERMISSION="edit" # view or edit

cat << EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ${SA_NAME}:${SA_PERMISSION}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ${SA_PERMISSION}
subjects:
- kind: ServiceAccount
  name: ${SA_NAME}
  namespace: ${SV_NAMESPACE}
EOF

Step 4b (Optional) - To configure our service account to have the "owner" vSphere Namespace role, we need to create an additional ClusterRoleBinding after completing Step 4a.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ${SA_NAME}:namespace-delete
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: vmware-system-nsop-namespace-delete-cluster-role
subjects:
- kind: ServiceAccount
  name: ${SA_NAME}
  namespace: ${SV_NAMESPACE}
EOF

At this point, you can now log out of the VCSA and the remainder steps can be performed using any account that can login to vSphere with Tanzu enabled Cluster.

Step 5 - Once you are logged in using the kubectl-vsphere plugin, specify the name of the service account, vSphere Namespace and the Control Plane address to automatically create the kubeconfig file for our service account.

SA_NAME="tanzu-svc"
SV_NAMESPACE="primp-industries"
SV_CONTROL_PLANE_ADDRESS="10.10.0.65"

SA_SECRET_NAME=$(kubectl get sa ${SA_NAME} -o jsonpath='{.secrets[0].name}')
SA_SECRET_VALUE=$(kubectl get secret/${SA_SECRET_NAME} -o jsonpath='{.data.token}' | base64 --decode)
SA_SERVER_URI="https://${SV_CONTROL_PLANE_ADDRESS}:6443"

cat << EOF > ${SA_NAME}-kubeconfig
apiVersion: v1
kind: Config
current-context: ${SV_NAMESPACE}
clusters:
- cluster:
    server: ${SA_SERVER_URI}
  name: ${SV_CONTROL_PLANE_ADDRESS}
contexts:
- context:
    cluster: ${SV_CONTROL_PLANE_ADDRESS}
    namespace: ${SV_NAMESPACE}
    user: ${SA_NAME}
  name: ${SV_NAMESPACE}
users:
- name: ${SA_NAME}
  user:
    token: ${SA_SECRET_VALUE}
EOF

Step 6 - Lastly, lets now verify our kubeconfig file is valid by running a basic kubectl command such as listing the available Tanzu Kubernetes Releases (TKR):

kubectl --kubeconfig tanzu-svc-kubeconfig get tkr


If the previous command was successful, you now have a valid kubeconfig that can be used for automation and/or GitOps purposes without having to re-authenticate every 10hrs.

More from my site

  • Packer reference for VMware Harbor Virtual Appliance
  • Is vSphere with Kubernetes available for evaluation? 
  • Admin account for embedded Harbor registry in vSphere with Kubernetes
  • Setup custom login banner when logging into a vSphere with Kubernetes Cluster
  • Workload Management PowerCLI Module for automating vSphere with Kubernetes

Categories // Automation, VMware Tanzu, vSphere 7.0 Tags // vSphere with Kubernetes

Comments

  1. Arvind Patil says

    08/24/2022 at 1:58 pm

    Hi William,
    This kubeconfig is for the supervisor cluster. How do I create similar kubeconfig for a workload cluster. Kubeconfig for workload cluster also expires after 10 hours.

    I tried following the same steps but I am stuck at creating a ClusterRoleBinding. I would like to create a kubeconfig that should work for all namespaces in the workload cluster. ClusterRoleBinding specs requires to specify a namespace. But at the workload cluster level I want the kubeconfig to work for multiple namespaces. How do I create such kubeconfig for a workload cluster?
    Thanks,
    Arvind

    Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • Changing the default HTTP(s) Reverse Proxy Ports on ESXi 8.0 03/22/2023
  • Quick Tip - How to download ESXi ISO image for all releases including patch updates? 03/15/2023
  • SSD with multiple NVMe namespaces for VMware Homelab 03/14/2023
  • Is my vSphere Cluster managed by vSphere Lifecycle Manager (vLCM) as a Desired Image or Baseline? 03/10/2023
  • Interesting VMware Homelab Kits for 2023 03/08/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...