Questions regarding vSphere Permissions for vSphere with Tanzu has been frequently brought up more lately and the majority of questions that I have seen, has primarily focused on the behavior of the vSphere UI Inventory. After taking a closer look and experimenting with a few permutations within my lab, I realized that most folks were simply focusing on what they were most familiar with, which is using the vSphere UI to interact with vSphere.
Although vSphere with Tanzu is tightly integrated with vSphere and the vSphere UI is certainly a primary interface, it is certainly not the only interface nor is it always the interface for end users like a developer. Depending on the needs of your end users and how your organization wishes to grant access to a vSphere Namespace, there are actually a few options that are available to you. In fact, users can interact with vSphere with Tanzu without ever logging into the vSphere UI and that is completely valid and may even be desirable for some organizations.
Note: The custom kubectl plugin for vSphere (kubectl-vsphere) which is needed to interact with vSphere with Tanzu can be downloaded by simply opening a browser (or use wget) to following URL: https://[SUPERVISOR-CLUSTER-IP]/wcp/plugin/[OS]-amd64/vsphere-plugin.zip, where OS is darwin, linux or windows (e.g. https://172.17.33.33/wcp/plugin/darwin-amd64/vsphere-plugin.zip)
Below are the results of my testing using the various vSphere Roles and Groups including the various behavior across the different consumption interfaces including the vSphere UI. To help better illustrate the results, I am also using some example personas, these are purely used as an example and may differ based on your organizational needs.
Persona: VI/Cloud Admin
In this scenario, the user is a vSphere Administrator and has the following memberships:
- vSphere Role: Administrator
- vSphere SSO Group: Administrators
- vSphere Namespace: SSO User and/or Active Directory User
The user will be able to view and manage all vSphere infrastructure including the vSphere Namespaces and the respective workloads including TKG Workload Clusters and/or VMs via the VM Service.
Here is a summary of this users access:
| Interface | Access |
|---|---|
| vSphere UI VM Inventory View | ✅ |
| vSphere UI Namespace Inventory View | ✅ |
| vSphere Namespace View | ✅ |
| Kubectl Access | ✅ |
Persona: Developers
In this scenario, the user is a consumer of the vSphere Namespace and will be able to provision both TKG Workload Clusters and/or VMs in a self-service manner using the kubectl CLI. Depending on the needs of your organization and the requirement of your end users, you have a couple of options.
Option 1 - In this first option, the user has the following memberships:
- vSphere Role: None
- vSphere SSO Group: None
- vSphere Namespace: SSO User and/or Active Directory User
The user may not have or want to login to the vSphere UI and as long as they have been granted access to a specific vSphere Namespace and given the Supervisor Cluster IP Address, they will be able to login and start deploying and managing their workloads.
Here is a summary of this users access:
| Interface | Access |
|---|---|
| vSphere UI VM Inventory View | ❌ |
| vSphere UI Namespace Inventory View | ❌ |
| vSphere Namespace View | ❌ |
| Kubectl Access | ✅ |
As an aside, although end users may not have a graphical interface such as the vSphere UI, there are many Kubernetes based UI/Terminal tools out there including my favorite, which is Octant. Simply download the client and once started, it will login using your k8s configuration file and you will now have access to all your TKG Workload Clusters and can easily switch between them.
Option 2 - In this second option, the user has the following memberships:
- vSphere Role: ReadOnly
- vSphere SSO Group: None
- vSphere Namespace: SSO User and/or Active Directory User
In addition to all the interactions described in Option 1 using kubectl, users will now also have the ability login to the vSphere UI. Although users will not see their vSphere Namespace and the respective workloads in the vSphere Inventory view, they can navigate to the Namespaces tab under the vSphere Cluster level and get a view of all their resources and workloads as shown in the screenshot below. To be honest, this is not a view I normally navigate to and I often forget about, but it does provide users with all relavent information as it pertains to their vSphere Namespace-based workloads in the vSphere UI, which is especially handy if you still have a need to manage or access your traditional VMs.
| Interface | Access |
|---|---|
| Component | Access |
| vSphere UI VM Inventory View | ✅ |
| vSphere UI Namespace Inventory View | ❌ |
| vSphere Namespace View | ✅ |
| Kubectl Access | ✅ |
Persona: DevOps/Platform Operations
In this scenario, the user may not be part of the core vSphere Administrator team but is part of a Platform Team who is responsible for managing, supporting and/or creating the TKG Workload Clusters and has the following memberships:
- vSphere Role: Non-Administrator or Custom Role
- vSphere SSO Group: ServiceProviderUsers
- vSphere Namespace: SSO User and/or Active Directory User
This persona may not exists in every organization, but it is certainly a growing trend (see this blog post for more details) and may have a need to easily view all vSphere Namespace-based workloads using the classic vSphere Inventory view.
Note: This option was initially covered in this blog post.
| Interface | Access |
|---|---|
| Component | Access |
| vSphere UI VM Inventory View | ✅ |
| vSphere UI Namespace Inventory View | ✅ |
| vSphere Namespace View | ✅ |
| Kubectl Access | ✅ |
Hey William, these all sound like fully plausible operations. But I'm hitting a permissions issue whereby when deploying Jupyterhub for kubernetes (in order to have scalable individual Jupyter notebook pods for users) When running kubectl get clusterroles I'm getting:
Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "sso:@" cannot list resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope.
There appears to be two parts to this - firstly, when deploying the Helm chart and attempting to create a role and binding, Tanzu is blocking access to Kubernetes resource verbs that are not accessible to the current user e.g. resource of bindings, verb of create. To rule out any issues with permissions this has been checked with *protected email*.
>> Question is can Tanzu be configured to allow access to these resource verbs?
{APIGroups:[""], Resources:["bindings"], Verbs:["create"]}
{APIGroups:[""], Resources:["persistentvolumes"], Verbs:["watch" "patch" "watch"]}
{APIGroups:[""], Resources:["pods/status"], Verbs:["patch" "update"]}
Secondly, 6 of the resources in the list below are not returned when performing a kubectl auth can-i --list -n= | grep subjectaccessreviews as an example
{APIGroups:[""], Resources:["pods/binding"], Verbs:["create"]}
{APIGroups:["authentication.k8s.io"], Resources:["tokenreviews"], Verbs:["create"]}
{APIGroups:["authorization.k8s.io"], Resources:["subjectaccessreviews"], Verbs:["create"]}
{APIGroups:["storage.k8s.io"], Resources:["csidrivers"], Verbs:["get" "list" "watch"]}
{APIGroups:["storage.k8s.io"], Resources:["csinodes"], Verbs:["get" "list" "watch"]}
{APIGroups:["storage.k8s.io"], Resources:["csistoragecapacities"], Verbs:["get" "list" "watch"]}
>> Question is can Tanzu be configured to allow access to the 6 resources listed?
Thanks in advance as I know it's a very specific issue, but wondered if you'd ever seen or overcome issues like this before?
If you’re attempting to deploy general workload into Supervisor Cluster, this is expected and by design, you won’t have full admin access. You should deploy TKC (k8s) using Supervisor and then you can provision your workload as expected
William, that makes more sense, I'll go back and attempt to deploy them as you say.
We were struggling time and again to give permissions and getting rebuffed so hopefully this will clear it. Thanks again 👍🏻