WilliamLam.com

How should bulk operations such as adding ESXi hosts work?

by // 4 Comments

I had a meeting yesterday with a couple of Engineers to provide some feedback on a new project they were working on and they were interested in getting feedback on their  workflows but also my overall impression of the user experience (UX). One area that we had spent some time covering was on adding new resources such as an ESXi host which is performed using either a UI/API/CLI. In the case of adding an ESXi host, this was done sequentially and a user would need to specify multiple parameters like the hostname, IP Address, credentials, etc. and perform this task N-number of times until you had added all of the ESXi hosts that you wanted. Many of you are probably already familiar with this workflow as it exists in many of the current VMware products today when requiring to add a resource like an ESXi host.

I was wondering why this was the case and why not support a bulk add operation? This was not a new idea and in fact I and many others have asked about this in the past on why our own vSphere Web Client does not support bulk operations for some of the more common operations such as adding an ESXi host or provisioning a VM? In fact, I was actually reminded of this topic again this morning when I needed to mount an NFS Volume across multiple ESXi hosts and one of the new capabilities that was introduction in the vSphere Web Client over the vSphere C# Client was being able to bulk add an NFS Volume across multiple hosts.

Not only is this a useful capability to have but when you need to perform the same operation multiple times, there is also an increase of risk for making a mistake. Sure, if you mount the wrong NFS volume, you would have it wrong across all the hosts but this is no different if you were to Automate it from an API/CLI. I am sure some of you are probably saying, a simple for loop would do the trick and though I agree that would be the easiest way. However, it would be really nice that some of these operations would actually have native support for bulk operations from an API standpoint, that way this is not limited to just a UI-only feature which it is today. Even in the case of a simple for-loop, you are still executing the individual operation N-number of times versus being able to pass in a list and perform a single request and then rely on the backend implementation to properly handle the operation which could be done in parallel or sequentially depending on the type of operation.

Thinking about this a bit more, it may not be as simple as just adding bulk operation support into an API but there are also other things to consider. What happens when the resource you are adding has varying configurations, a classic example of this is adding multiple ESXi hosts. If they all have the same username and password, then you can provide the credentials once and then a list of ESXi hosts. What happens when they differ or in the worse case scenario, they all have different passwords? What happens when only a subset of the hosts are added, do you continue or fail all together?

I am actually curious to hear what others think about this and what your expectations would be? Both from a UI standpoint as well as from an Automation standpoint which includes API/CLI. Would a simple JSON/YAML configuration file that contains all hosts and credentials be acceptable with the option of having common global configurations and ability to override for individual entries? Would love to hear your thoughts on this and what you would be expecting? Perhaps this is something I can also share back with the Engineering teams as this is something I have been wanting to see for quite some time, especially when it comes to common operations like adding ESXi hosts and provisioning VMs.

vCenter Server 6.0 Tidbits Part 10: Automating SSO Admin configurations

by // 4 Comments

A common request that I have heard from customers is to have the ability to automate vCenter Single Sign-On configurations from a programmatic standpoint. Unfortunately, this is currently not possible today as a public API does not exist for SSO. Having spent some time exploring the underlying vmdir database which is just an LDAP-based system (here & here) and learning about a way to update a particular key per KB2070433 within the vmdir database which I have shown here and I have found here, I wanted to see if it was possible to query for these specific SSO Admin configurations and also be able to update these properties.

Disclaimer: Please take extreme caution when connecting to the vmdird database. You should take extreme care in making changes while in the database else you can negatively impact your environment.

There are three main sections to the SSO Admin configurations that can be seen from the vSphere Web Client:

  • Password Policies
  • Lockout Policy
  • Token Policy

For each section, I have provided the specific ldapsearch query (please refer to this article as per-requisite) which can either be run directly on the VCSA if you are using that or a system that includes the ldapsearch command. You will need to replace the text highlighted in blue with your own environment details.

Password Policies & Lockout Policy

To view the following set of configurations, here is the ldapsearch query to use:

/opt/likewise/bin/ldapsearch -h 192.168.1.70 -w 'VMware1!' -x -D "cn=Administrator,cn=Users,dc=vghetto,dc=local" -b "cn=password and lockout policy,dc=vghetto,dc=local"

automate-sso-configuration-3
Here is a screenshot of the Password Policies as seen in the vSphere Web Client and their corresponding LDAP property names:

automate-sso-configuration-0

UI Setting LDAP Attribute Name
Maximum lifetime vmwPasswordLifetimeDays
Restrict reuse vmwPasswordProhibitedPreviousCount
Maximum lenght vmwPasswordMaxLength
At least special character vmwPasswordMinSpecialCharCount
At least alphabetic character vmwPasswordMinAlphabeticCount
At least uppercase character vmwPasswordMinUpperCaseCount
At least lowercase character vmwPasswordMinLowerCaseCount
At least numeric character vmwPasswordMinNumericCount
Identical adjacent Characters vmwPasswordMaxIdenticalAdjacentChars

Here is a screenshot of the Lock Policy as seen in the vSphere Web Client and their corresponding LDAP property names:

automate-sso-configuration-1

UI Setting LDAP Attribute Name
Maximum number of failed login attempts vmwPasswordChangeMaxFailedAttempts
Time interval between failures vmwPasswordChangeFailedAttemptIntervalSec
Unlock time vmwPasswordChangeAutoUnlockIntervalSec

Token Policy

To view the following configuration, here is the ldapsearch query to use:

/opt/likewise/bin/ldapsearch -h 192.168.1.70 -w 'VMware1!' -x -D "cn=Administrator,cn=Users,dc=vghetto,dc=local" -b "cn=Tenants,cn=IdentityManager,cn=Services,dc=vghetto,dc=local" -s sub "objectclass=vmwSTSTenant"

automate-sso-configuration-4
Here is a screenshot of the Token Policy as seen in the vSphere Web Client and their corresponding LDAP property names:

Token Policy

automate-sso-configuration-2

UI Setting LDAP Attribute Name
Clock tolerance vmwSTSClockTolerance
Maximum token renewal count vmwSTSRenewCount
Maximum token delegation count vmwSTSDelegationCount
Maximum bearer token lifetime vmwSTSMaxBearerTokenLifetime
Maximum holder-of-key token lifetime vmwSTSMaxHolderOfKeyTokenLifetime

Now that we know how to query for a particular SSO Configuration, here is how you can modify one of these properties. In the example below, we will be changing the life time of a password which dictates the frequency in which you need to change an SSO user's password. Using the "Password Policies" table above, we can see the that property name is called vmwPasswordLifetimeDays

To modify an LDAP entry, we will need to first create a file that contains the change, in the example here we are going to name it change.ldif and it should contain the following where the "replace" keyword shows which property is getting modified and the next line after shows the value that it will be changed to.

dn: cn=password and lockout policy,dc=vghetto,dc=local
changetype: modify
replace: vmwPasswordLifetimeDays
vmwPasswordLifetimeDays: 30

To apply the change, we will now run the following ldapmodify command and specifying our change.ldif configuration file:

/opt/likewise/bin/ldapmodify -f change.ldif -h 192.168.1.70 -D "cn=Administrator,cn=Users,dc=vghetto,dc=local" -w 'VMware1!'

automate-sso-configuration-5
If the change was successful, you can confirm by either querying the property again using the ldapquery command or just refreshing the SSO Configurations using the vSphere Web Client.

Retrieving statistics for a Distributed Virtual Port using the vSphere API & PowerCLI

by // 2 Comments

Over the weekend I received an email from a fellow colleague over in the NSBU who was interested in retrieving statistics on an individual Distributed Virtual Port using the vSphere API and had asked if I had done this before. Although I have written about using the vSphere API to access various properties of a Distributed Virtual Switch (here & here) and migrating from a Virtual Standard Switch (VSS) to Distributed Virtual Switch (VDS) and from a VDS back to VSS (here & here), I have not done a whole lot with the vSphere API with regards to individual Distributed Virtual Ports.

Having said that, with the easy to use PowerCLI Distributed Virtual Switch cmdlets being available, it is actually not too difficult to retrieve this information using the vSphere API. For those of you who have not worked with a Distributed Virtual Switch before, to view the statistics of an individual Distributed Virtual Port, you must drill down into a specific Distributed Virtual Portgroup which you can find using either the vSphere Web/C# Client. Below is a screenshot from my vSphere Web Client.

distributed-virtual-port-stats-0
To retrieve information on a Distributed Virtual Port using the vSphere API, you can refer to the DistributedVirtualPort property. Within this object, there is a state property which you can then drill down further to retrieve statistics using the stat property. If you would like to be able to pull other properties as shown in the screenshot, you can use the runtimeInfo property which provides additional information on the Distributed Virtual Port such as connected entity, MAC Address, VLAN ID, state, etc.

The following two PowerCLI VDS cmdlets: Get-VDPortgroup & Get-VDPort will help us retrieve the information we are looking. However, out of the box these cmdlets do not support pulling statistics for a Distributed Virtual Port. To do so, we will need to access the ExtensionData property which will allows us to access the underlying vSphere API from the PowerCLI object.

Below is a simple PowerCLI script which connects to a vCenter Server and retrieves a specific Distributed Virtual Portgroup (dvPg) that we are interested in using the Get-VDPortgroup cmdlet. Once we have the dvPg, we can then identify a specific Distributed Virtual Port (dvP) by using the Get-VDPort and passing in the dvPg from the previous command as well as the specific port by using the -key property. From here, we can then access a few properties from the dvP and most importantly, the statistics which we were originally interested in. In the example below, I am displaying the entire "stat" property, but you can easily filter out the specific stats you care about.

$server = Connect-VIServer -Server reflex.primp-industries.com

# Retrieve a Distributed Portgroup
$vdportgroup = Get-VDPortgroup -Name dv-vlan333

# Retrieve a specific Distributed Virtual Port from the Distributed Portgroup using the "key" property
$vdport = Get-VDPort -VDPortgroup $vdportgroup -Key 4

# Sample Distributed Virtual Port properties
Write-Host "Link Up: " $vdport.ExtensionData.State.RuntimeInfo.Linkup
Write-Host "Connectee: " $vdport.ExtensionData.State.RuntimeInfo.LinkPeer
Write-Host "Runtime MAC Address: " $vdport.ExtensionData.State.RuntimeInfo.macAddress

# Distributed Virtual Port Stats
$vdport.ExtensionData.State.Stats

Disconnect-VIServer -Server $server -Confirm:$false

Here is an example of the output from the script above and we can see that Distributed Virtual Port "4" matches to what we see in the vSphere Web Client screenshot.
distributed-virtual-port-stats-1