WilliamLam.com

How to Create Custom Roles on Standalone ESX(i) Host

by // 9 Comments

There was an interesting question on the VMTN forums that caught my eye this week in which a user posted a question about controlling access to a limited set of virtual machines for users on a standalone ESX(i) host. For those of you who are familiar with vCenter Server know that this can easily be accomplished by using/creating roles which allows for granular access to various resources within a vSphere environment such as Datacenter, Resource Pool, Host, Virtual Machine, etc. This was the suggestion from several community members and it ended up being the simple solution as the user actually had a vCenter Server.

So what if the user did not have a vCenter Server and actually had standalone ESX(i) host, would he/she be out of luck? The answer is no, you can actually utilize roles on a standalone host but the default roles that are available are pretty limited in scope. By default, there are three roles: No access, Read-only and Administrator and these roles can be applied to both local and/or Active Directory users if your ESX(i) host is joined to an AD domain.

By leveraging our good friend vimsh/vim-cmd from either the classic ESX Service Console or Tech Support Mode of ESXi, we can actually create custom roles just like we would on vCenter Server with fine grain permissions to a virtual machine(s). You can not create custom roles using the vSphere Client like you would when connecting to a vCenter Server.

We will be using the commands found under vim-cmd vimsvc/auth:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth
Commands available under vimsvc/auth/:
entity_permission_add lockdown_mode_enter role_permissions
entity_permission_remove lockdown_mode_exit role_remove
entity_permissions permissions roles
lockdown_is_enabled privileges
lockdown_is_possible role_add

Before we get started, it is good to understand what is a permission and how that relates to a particular entity such as a virtual machine. Here are some definitions from VMware:

Privilege - The ability to perform a specific action or read a specific property
Role - A collection of privileges. Roles provide a way to aggregate all the individual privileges that are required to perform a higher-level task, such as administer a virtual machine
Object - An entity upon which actions are performed. VirtualCenter objects are datacenters, folders, resource pools, clusters, hosts, and virtual machines

Permission = User/Group + Role + Object 

Basically we will be creating a custom role which contains a specific set of privileges to administrator a virtual machine and assign this role to a set of users on a standalone ESX(i) host.

To view all available privileges, you can run the following command:

~ # vim-cmd vimsvc/auth/privileges | grep privId | sed 's/^[ \t]*//;s/[ \t]*$//;s/,//;s/"//g' | awk '{print $3}' | sort
Alarm.Acknowledge
Alarm.Create
Alarm.Delete
Alarm.DisableActions
Alarm.Edit
Alarm.SetStatus
Authorization.ModifyPermissions
Authorization.ModifyRoles
Authorization.ReassignRolePermissions
DVPortgroup.Create
DVPortgroup.Delete
DVPortgroup.Modify
DVPortgroup.PolicyOp
DVPortgroup.ScopeOp
DVSwitch.Create
DVSwitch.Delete
DVSwitch.HostOp
.....

To view existing roles and what privileges are associated with a given role, you can run the following command:

vim-cmd vimsvc/auth/roles | less

From the above privilege list, you will need to select the privileges that you would like in your new role, for now you can just copy the entries into a temporarily file (/tmp/vm-privileges) as that will be used later.

In this example, I've selected the basic VM privileges that is associated with the "Virtual Machine User" found in vCenter Server but adding on additional privileges to support snapshots.

[root@vesx41-1 ~]# cat /tmp/vm-privileges
Global.CancelTask
VirtualMachine.Interact.AnswerQuestion
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.SetCDMedia
VirtualMachine.Interact.SetFloppyMedia
VirtualMachine.Interact.ConsoleInteract
VirtualMachine.Interact.DeviceConnection
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.PowerOn
VirtualMachine.Interact.Reset
VirtualMachine.Interact.Suspend
VirtualMachine.Interact.ToolsInstall
VirtualMachine.State.CreateSnapshot
VirtualMachine.State.RemoveSnapshot
VirtualMachine.State.RevertToSnapshot

Next we will need to know the virtual machine's VmId also known as the MoRefID (Managed Object ID) that uniquely identifies a given entity. To display the VmId of all virtual machines residing on your ESX(i) host, you can run the following command:
[root@vesx41-1 ~]# vim-cmd vmsvc/getallvms
Vmid Name File Guest OS Version Annotation
32 SteveJablonsky [datastore1] SteveJablonsky/SteveJablonsky.vmx freebsdGuest vmx-07
48 HansZimmer [datastore1] HansZimmer/HansZimmer.vmx dosGuest vmx-07
64 JamesNewton [datastore1] JamesNewton/JamesNewton.vmx dosGuest vmx-07
80 BrianTyler [datastore1] BrianTyler/BrianTyler.vmx dosGuest vmx-07

Next we need to identify the users in which we will create the new permissions for. In this example, I will use both a local (william) and Active Directory user (PRIMP-IND\primp):

[root@vesx41-1 ~]# id william
uid=501(william) gid=100(users) groups=100(users)

[root@vesx41-1 ~]# id PRIMP-IND\\primp
uid=37225554(PRIMP-IND\primp) gid=37224960(PRIMP-IND\domain^admins) groups=37224960(PRIMP-IND\domain^admins),37225020(PRIMP-IND\denied^rodc^password^replication^group)

In this example, I have a single ESX host that contains 4 virtual machines and the first two will only be visible to a user called "william" and the second two will only be visible to a user called "PRIMP-IND\primp":

  •  Owner: william
    • HansZimmer
    • BrianTyler
  •  Owner: PRIMP-IND\primp
    • SteveJablonsky
    • JamesNewton

We will create a virtual machine administrator role called "VMAdmin" which will then be associated with their respective virtual machines and system owners.

First we need to create the custom "VMAdmin" role by using the file that contains all the necessary privileges, we will run the following command:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/role_add VMAdmin $(cat /tmp/vm-privileges | awk '$1=$1' ORS=' ')

Note: The "role_add" operation takes in a roleName and then a list of privileges which are separated out by a space, this list can be as many as you need and by specifying it in a file, it makes it easier to edit/adjust

You can verify that the new role was created running the following command:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/roles
......
(vim.AuthorizationManager.Role) {
dynamicType = ,
roleId = 12,
system = false,
name = "VMAdmin",
info = (vim.Description) {
dynamicType = ,
label = "VMAdmin",
summary = "VMAdmin",
},
privilege = (string) [
"Global.CancelTask",
"System.Anonymous",
"System.Read",
"System.View",
"VirtualMachine.Interact.AnswerQuestion",
"VirtualMachine.Interact.ConsoleInteract",
"VirtualMachine.Interact.DeviceConnection",
"VirtualMachine.Interact.PowerOff",
"VirtualMachine.Interact.PowerOn",
"VirtualMachine.Interact.Reset",
"VirtualMachine.Interact.SetCDMedia",
"VirtualMachine.Interact.SetFloppyMedia",
"VirtualMachine.Interact.Suspend",
"VirtualMachine.Interact.ToolsInstall",
"VirtualMachine.State.CreateSnapshot",
"VirtualMachine.State.RemoveSnapshot",
"VirtualMachine.State.RevertToSnapshot"
],
}
]

Note: If you created a role that did not contain all the privileges and you need to update the role, you actually have to delete the role first. I have not found a way to "append" privileges, but removing a role is very straight forward and you can use "vim-cmd vimsvc/auth/role_remove" to do so.

Next we will associate this new role with the users and the virtual machines to create the permission mappings. If you recall earlier, we retrieved a list of virtual machines and the first column contains the VM's VmId which will be needed in this next section. We will be using the "vim-cmd vimsvc/auth/entity_permission_add" operation and if you would like to know what arguments it accept, you can just run it by itself without any arguments.To associate the first two virtual machines to the user "william" we will be running the following commands:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:80 william false VMAdmin true
[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:48 william false VMAdmin true

To associate the new two virtual machines with the user "PRIMP-IND\primp" we will be running the following commands:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:64 PRIMP-IND\\primp false VMAdmin true
[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permission_add vim.VirtualMachine:32 PRIMP-IND\\primp false VMAdmin true

Note: The VM entity syntax is in the form of vim.VirtualMachine:XX where XX is the virtual machine's VmId extracted earlier which will need to be substituted in with your own VmId

We can verify the permissions by running a few commands, we can either check all permissions created with this particular role name, in our example, it is "VMAdmin" and the command to run would be the following:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/role_permissions VMAdmin
(vim.AuthorizationManager.Permission) [
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:32',
principal = "PRIMP-IND\primp",
group = false,
roleId = 12,
propagate = true,
},
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:48',
principal = "william",
group = false,
roleId = 12,
propagate = true,
},
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:64',
principal = "PRIMP-IND\primp",
group = false,
roleId = 12,
propagate = true,
},
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:80',
principal = "william",
group = false,
roleId = 12,
propagate = true,
}
]

You can also check a specific entity such as a virtual machine by running the following command:

[root@vesx41-1 ~]# vim-cmd vimsvc/auth/entity_permissions vim.VirtualMachine:64
(vim.AuthorizationManager.Permission) [
(vim.AuthorizationManager.Permission) {
dynamicType = ,
entity = 'vim.VirtualMachine:64',
principal = "PRIMP-IND\primp",
group = false,
roleId = 12,
propagate = true,
}
]

Now that we have confirmed the new permissions, we will want to verify that these users can only see the virtual machines we have specified both in the vSphere Client and webAccess (ESX only).

Here is a screenshot of user "william" connecting to the vSphere Client:

Here is a screenshot of user "PRIMP-IND\primp"connecting to the ESX webAccess:

Contrary to the default permissions on a stand alone host, you actually can reate custom roles just like on a vCenter Server to have fine grain access controls for your various users without having to give them full administrative privledges to an ESX(i) host. If you have vCenter Server, then of course that is the recommended approach but if you do not, you still have a way 🙂

vSphere Tagging Feature Not So Invisible

by // 9 Comments

This evening I was reading a new blog post Tagging: An Invisible Feature in vSphere by Steve Jin in which he describes a hidden vSphere feature for tagging particular managed entities in your vSphere environment. The main consumer of this hidden feature is primarily VMware and it provides a level of meta data tagging to denote the importance of an entity. These tags are used in workflows in various vSphere products such as vCenter Server, vCenter Update Manager and View to just name a few.

Here is a screenshot of a VM entity using the vSphere MOB and as you can see from the tags, VMware can identify that this particular VM is running both vCenter Server and vCenter Update Manager.

If you have a VM that is a View Replica/Linked Clone, you may see these tags associated with the VM:

SYSTEM/COM.VMWARE.SIM.SVIMANAGEDOBJECT
SYSTEM/COM.VMWARE.VIM.SVIMANAGEDOBJECT

This feature can be really useful to allow users to tag particular objects in their environment of importance and easily correlate/integrate with a provisioning system and/or CMDB without having to resort to a separate database to store the meta data. You might say that custom fields exists today that can provide the same level of functionality, but they are actually pretty limited in scope. It only allows you to create custom attributes at a global vCenter level, host and virtual machine but not all managed objects such as Datastore, Networks, Resource Pool, etc.

So what if you wanted to create your own custom tags? Since these are hidden methods from VMware and they are not exposed through the public vSphere API, how might we access it? I spent a few minutes of digging and I was able to identify the two methods that pertained to tag management in the vSphere API:

addTag - Adds a tag to a managed entity
removeTag - Removes a tag to a managed entity

You can access both of these methods using the vSphere MOB as none of the official vSphere SDKs have these methods implemented. These methods are only supported when connecting to vCenter Server and are not exposed at an ESX(i) level.

In this example, I will create a custom tag at the Datacenter level called "WWW.VIRTUALLYGHETTO.COM"

Here is the path to a Datacenter object using the vSphere MOB:

If we scroll down on the page, you will notice a property called "tag" and as you can see in the screenshot below, no tags have been defined:

I will now craft the URL to the "addTag" method using the Datacenter as a point of reference:

https://reflex.primp-industries.com/mob/?moid=datacenter-1246&method=addTag

As you can see the method accepts an array of tags which can be defined within the section. All tags MUST begin with "SYSTEM/" and then a unique string following, in our example I used "SYSTEM/WWW.VIRTUALLYGHETTO.COM". Once you have created your tag, you will then click on the "Invoke Method" to add the tag. Since this method does not return anything, the return result is void.

Now if we go back to the Datacenter page and refresh the page, you should see the new tag that was just created:

To delete a tag, you will generate the same URL but replace "add" with "remove":

https://reflex.primp-industries.com/mob/?moid=datacenter-1246&method=removeTag

You will again fill in the tags to be removed and then click on "Invoke Method":

As you can see, you now have the power to create your own tags on various managed entities in your vSphere environment. If you want to automate this through a script or command line, you can create simple HTTP POST operations to the vSphere MOB or create the appropriate interface methods to the SDK bindings to support these hidden methods. I'll leave the latter as an exercise for the readers.

How to query for MACs on internal vSwitch on ESXi

by // 10 Comments

There was an interesting question this week on the VMTN community forums about querying a vSwitch on an ESX(i) host. The user was trying to locate a particular virtual machine's MAC Address due to an IP conflict that was identified. The internal VMware vSwitch is pretty much closed off as a blackbox. The vSwitch is not exposed like a traditional physical switch in which you can run commands against such as "show mac-address-table" to display the MAC addresses found on the switch.

However, you can still perform a lookup of all the MAC Addresses found on a particular ESX(i)/vCenter host by using the vSphere APIs. You can search for all virtual machines and dump out their associated MAC Addresses and correlate that back to a particular vSwitch. You can easily do this through a script such as using the vSphere SDK for Perl script: getvSwitchMacTable.pl which supports both stand vSwitch and distributed vSwitch or if you prefer a GUI, you can use the popular RVTools. I am sure there is most likely a PowerCLI solution to solving this problem as well.

The solution described above is the proper and most flexible way of solving this problem, but what if you really wanted to query the internal vSwitch and extract out the MAC Addresses that way? Well the answer is, you can so using vsish on ESXi (vsish is not available on ESX unless you have the VMware debugging RPM package installed).

Here are some of things you can view for a given vSwitch using vsish:

~ # vsish -e ls /net/portsets/vSwitch0
ports/
overlays/
uplinks/
type
mtu
unlink
link
destroy
properties
stats

The "ports" section is what we are interested in:

~ # vsish -e ls /net/portsets/vSwitch0/ports
16777217/
16777218/
16777219/
16777220/
16777358/
16777359/

When looking at a particular port, it provides quite a bit of information on what is connected and various metrics/statistics:

~ # vsish -e ls /net/portsets/vSwitch0/ports/16777220/
respool/
e1000/
vmxnet3/
pktSizes/
clusterSizes/
worlds/
coalesceDetailed/
ip
schedTeamUplink
teamUplink
blocked
injectIGMP
txCompCoalesce
txCoalesce
rxCoalesce
controlChain
notifyStats
inputStats
outputStats
vmxnet2clientStats
clientStats
gateway
setPassthru
status
stats

As you can see it is pretty tedious to go through each of the ports and it does not easily allow you to figure out what is exactly connected to the port until you view the "status" property.

I decided to write a tiny script that would allow a user to dump out all the MAC Addresses from the vSwitch(s) found on an ESX(i) host. Not only does it provide this mapping but also what is specifically using a given port whether it is mapped to internal interface or a particular virtual machine.

You can download the script vswitchInfo.sh which runs directly on ESXi's TSM (Tech Support Mode). The script can be called with the "-l" option to provide a high level dump of all MAC Addresses. Once you have identified the particular vSwitch and port, then you can get further details by specifying "-v" for vSwitch name and "-p" for the port number as displayed from the previous execution.

Here is an example output of just listing all MAC Addresses from all vSwitch(s) in an ESXi host:

Here is an example of getting more details on a particular port on a vSwitch:

Here you can see the clientName which is either a VM or interface using the port. You will also notice there is a mapping to set of pNICS that are attached to the vSwitch and various other details that I will let you explore.

You might have noticed the vSwitch port-ids looks kind of familiar? If you did, they actually are, as they part of the "networking" section in esxtop/resxtop output.

Unfortunately with esxtop/resxtop, it does not display the associated MAC Addresses, but now you have a way to easily query for details on the internal ports of a vSwitch.

Note: The second solution falls under the "not supported" category as you might have guessed.