How to create a kubernetes service account for vSphere with Tanzu?

Before you can interact and consume resources from a vSphere with Tanzu enabled cluster, users must first login and one way to accomplish this is by using the kubectl-vsphere plugin.

Once authenticated, a JWT (JSON Web Token), pronounced jot token, will be issued along with other values which will be appended to your local ~/.kube/config file. Users will then be able to perform kubectl operations based on the roles they have been assigned for a given vSphere Namespace. In case you did not know, these JWT tokens are only valid for 10 hours and after that, you will need to login again to retrieve a new JWT token.

We can also confirm this by decoding our JWT token found within the ~/.kube/config file and using jwt.io website. Once decoded, we can see when the token was issued using iat (Issued At) and when the token will expired using exp (Expiration Time) as shown in the screenshot below.

The default 10 hour expiry is currently not configurable which can be a challenge for anyone looking to setup unattended automation or GitOps with vSphere with Tanzu.

An alternative solution is to create a Kubernetes (k8s) service account, which by default does not contain a token expiry. Using this information and my recent Deep Dive into vSphere Namespace Roles, I was able to create a service account that can perform the same set of vSphere with Tanzu operations without having to re-login every 10 hours.

Note (06/07/22) - The "Edit" vSphere Namespace Role now includes the ability to create K8s service account and rolebinding without having to go into Supervisor Cluster Control Plane VM

Step 1 - SSH to the VCSA and then run the following script to retrieve the Supervisor Cluster Control Plane VM credentials:

/usr/lib/vmware-wcp/decryptK8Pwd.py

Step 2 - SSH to the IP Address using root username and the password provided from the previous command

Step 3 - Create a new K8s service account that is scoped to the specific vSphere Namespace. In my example, service account name is called tanzu-svc and it is scoped to primp-industries vSphere Namespace.

SA_NAME="tanzu-svc"
SV_NAMESPACE="primp-industries"

kubectl -n ${SV_NAMESPACE} create serviceaccount ${SA_NAME}

Step 4a - Create a ClusterRoleBinding for either the view or edit vSphere Namespace role for our service account.

SA_PERMISSION="edit" # view or edit

cat << EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ${SA_NAME}:${SA_PERMISSION}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ${SA_PERMISSION}
subjects:
- kind: ServiceAccount
  name: ${SA_NAME}
  namespace: ${SV_NAMESPACE}
EOF

Step 4b (Optional) - To configure our service account to have the "owner" vSphere Namespace role, we need to create an additional ClusterRoleBinding after completing Step 4a.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ${SA_NAME}:namespace-delete
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: vmware-system-nsop-namespace-delete-cluster-role
subjects:
- kind: ServiceAccount
  name: ${SA_NAME}
  namespace: ${SV_NAMESPACE}
EOF

At this point, you can now log out of the VCSA and the remainder steps can be performed using any account that can login to vSphere with Tanzu enabled Cluster.

Step 5 - Once you are logged in using the kubectl-vsphere plugin, specify the name of the service account, vSphere Namespace and the Control Plane address to automatically create the kubeconfig file for our service account.

SA_NAME="tanzu-svc"
SV_NAMESPACE="primp-industries"
SV_CONTROL_PLANE_ADDRESS="10.10.0.65"

SA_SECRET_NAME=$(kubectl get sa ${SA_NAME} -o jsonpath='{.secrets[0].name}')
SA_SECRET_VALUE=$(kubectl get secret/${SA_SECRET_NAME} -o jsonpath='{.data.token}' | base64 --decode)
SA_SERVER_URI="https://${SV_CONTROL_PLANE_ADDRESS}:6443"

cat << EOF > ${SA_NAME}-kubeconfig
apiVersion: v1
kind: Config
current-context: ${SV_NAMESPACE}
clusters:
- cluster:
    server: ${SA_SERVER_URI}
  name: ${SV_CONTROL_PLANE_ADDRESS}
contexts:
- context:
    cluster: ${SV_CONTROL_PLANE_ADDRESS}
    namespace: ${SV_NAMESPACE}
    user: ${SA_NAME}
  name: ${SV_NAMESPACE}
users:
- name: ${SA_NAME}
  user:
    token: ${SA_SECRET_VALUE}
EOF

Step 6 - Lastly, lets now verify our kubeconfig file is valid by running a basic kubectl command such as listing the available Tanzu Kubernetes Releases (TKR):

kubectl --kubeconfig tanzu-svc-kubeconfig get tkr

If the previous command was successful, you now have a valid kubeconfig that can be used for automation and/or GitOps purposes without having to re-authenticate every 10hrs.