WilliamLam.com

How should bulk operations such as adding ESXi hosts work?

06.04.2015 by William Lam // 4 Comments

I had a meeting yesterday with a couple of Engineers to provide some feedback on a new project they were working on and they were interested in getting feedback on their workflows but also my overall impression of the user experience (UX). One area that we had spent some time covering was on adding new resources such as an ESXi host which is performed using either a UI/API/CLI. In the case of adding an ESXi host, this was done sequentially and a user would need to specify multiple parameters like the hostname, IP Address, credentials, etc. and perform this task N-number of times until you had added all of the ESXi hosts that you wanted. Many of you are probably already familiar with this workflow as it exists in many of the current VMware products today when requiring to add a resource like an ESXi host.

I was wondering why this was the case and why not support a bulk add operation? This was not a new idea and in fact I and many others have asked about this in the past on why our own vSphere Web Client does not support bulk operations for some of the more common operations such as adding an ESXi host or provisioning a VM? In fact, I was actually reminded of this topic again this morning when I needed to mount an NFS Volume across multiple ESXi hosts and one of the new capabilities that was introduction in the vSphere Web Client over the vSphere C# Client was being able to bulk add an NFS Volume across multiple hosts.

Bulk support for adding NFS volumes across multiple ESXi host, still love this feature! Simple, yet powerful & useful pic.twitter.com/TZ7KeG52f6

— William Lam (@lamw.bsky.social | @*protected email*) (@lamw) June 4, 2015

Not only is this a useful capability to have but when you need to perform the same operation multiple times, there is also an increase of risk for making a mistake. Sure, if you mount the wrong NFS volume, you would have it wrong across all the hosts but this is no different if you were to Automate it from an API/CLI. I am sure some of you are probably saying, a simple for loop would do the trick and though I agree that would be the easiest way. However, it would be really nice that some of these operations would actually have native support for bulk operations from an API standpoint, that way this is not limited to just a UI-only feature which it is today. Even in the case of a simple for-loop, you are still executing the individual operation N-number of times versus being able to pass in a list and perform a single request and then rely on the backend implementation to properly handle the operation which could be done in parallel or sequentially depending on the type of operation.

Thinking about this a bit more, it may not be as simple as just adding bulk operation support into an API but there are also other things to consider. What happens when the resource you are adding has varying configurations, a classic example of this is adding multiple ESXi hosts. If they all have the same username and password, then you can provide the credentials once and then a list of ESXi hosts. What happens when they differ or in the worse case scenario, they all have different passwords? What happens when only a subset of the hosts are added, do you continue or fail all together?

I am actually curious to hear what others think about this and what your expectations would be? Both from a UI standpoint as well as from an Automation standpoint which includes API/CLI. Would a simple JSON/YAML configuration file that contains all hosts and credentials be acceptable with the option of having common global configurations and ability to override for individual entries? Would love to hear your thoughts on this and what you would be expecting? Perhaps this is something I can also share back with the Engineering teams as this is something I have been wanting to see for quite some time, especially when it comes to common operations like adding ESXi hosts and provisioning VMs.

vCenter Server 6.0 Tidbits Part 10: Automating SSO Admin configurations

06.03.2015 by William Lam // 4 Comments

A common request that I have heard from customers is to have the ability to automate vCenter Single Sign-On configurations from a programmatic standpoint. Unfortunately, this is currently not possible today as a public API does not exist for SSO. Having spent some time exploring the underlying vmdir database which is just an LDAP-based system (here & here) and learning about a way to update a particular key per KB2070433 within the vmdir database which I have shown here and I have found here, I wanted to see if it was possible to query for these specific SSO Admin configurations and also be able to update these properties.

Disclaimer: Please take extreme caution when connecting to the vmdird database. You should take extreme care in making changes while in the database else you can negatively impact your environment.

There are three main sections to the SSO Admin configurations that can be seen from the vSphere Web Client:

Password Policies
Lockout Policy
Token Policy

For each section, I have provided the specific ldapsearch query (please refer to this article as per-requisite) which can either be run directly on the VCSA if you are using that or a system that includes the ldapsearch command. You will need to replace the text highlighted in blue with your own environment details.

Password Policies & Lockout Policy

To view the following set of configurations, here is the ldapsearch query to use:

/opt/likewise/bin/ldapsearch -h 192.168.1.70 -w 'VMware1!' -x -D "cn=Administrator,cn=Users,dc=vghetto,dc=local" -b "cn=password and lockout policy,dc=vghetto,dc=local"

Here is a screenshot of the Password Policies as seen in the vSphere Web Client and their corresponding LDAP property names:

UI Setting	LDAP Attribute Name
Maximum lifetime	vmwPasswordLifetimeDays
Restrict reuse	vmwPasswordProhibitedPreviousCount
Maximum lenght	vmwPasswordMaxLength
At least special character	vmwPasswordMinSpecialCharCount
At least alphabetic character	vmwPasswordMinAlphabeticCount
At least uppercase character	vmwPasswordMinUpperCaseCount
At least lowercase character	vmwPasswordMinLowerCaseCount
At least numeric character	vmwPasswordMinNumericCount
Identical adjacent Characters	vmwPasswordMaxIdenticalAdjacentChars

Here is a screenshot of the Lock Policy as seen in the vSphere Web Client and their corresponding LDAP property names:

UI Setting	LDAP Attribute Name
Maximum number of failed login attempts	vmwPasswordChangeMaxFailedAttempts
Time interval between failures	vmwPasswordChangeFailedAttemptIntervalSec
Unlock time	vmwPasswordChangeAutoUnlockIntervalSec

Token Policy

To view the following configuration, here is the ldapsearch query to use:

/opt/likewise/bin/ldapsearch -h 192.168.1.70 -w 'VMware1!' -x -D "cn=Administrator,cn=Users,dc=vghetto,dc=local" -b "cn=Tenants,cn=IdentityManager,cn=Services,dc=vghetto,dc=local" -s sub "objectclass=vmwSTSTenant"

Here is a screenshot of the Token Policy as seen in the vSphere Web Client and their corresponding LDAP property names:

Token Policy

UI Setting	LDAP Attribute Name
Clock tolerance	vmwSTSClockTolerance
Maximum token renewal count	vmwSTSRenewCount
Maximum token delegation count	vmwSTSDelegationCount
Maximum bearer token lifetime	vmwSTSMaxBearerTokenLifetime
Maximum holder-of-key token lifetime	vmwSTSMaxHolderOfKeyTokenLifetime

Now that we know how to query for a particular SSO Configuration, here is how you can modify one of these properties. In the example below, we will be changing the life time of a password which dictates the frequency in which you need to change an SSO user's password. Using the "Password Policies" table above, we can see the that property name is called vmwPasswordLifetimeDays

To modify an LDAP entry, we will need to first create a file that contains the change, in the example here we are going to name it change.ldif and it should contain the following where the "replace" keyword shows which property is getting modified and the next line after shows the value that it will be changed to.

dn: cn=password and lockout policy,dc=vghetto,dc=local
changetype: modify
replace: vmwPasswordLifetimeDays
vmwPasswordLifetimeDays: 30

To apply the change, we will now run the following ldapmodify command and specifying our change.ldif configuration file:

/opt/likewise/bin/ldapmodify -f change.ldif -h 192.168.1.70 -D "cn=Administrator,cn=Users,dc=vghetto,dc=local" -w 'VMware1!'

If the change was successful, you can confirm by either querying the property again using the ldapquery command or just refreshing the SSO Configurations using the vSphere Web Client.

Creating your own 3rd Party Content Library for vSphere 6.0 & vCloud Director 5.x

06.02.2015 by William Lam // 21 Comments

If you recall a few weeks back, I had shared a custom vGhetto Content Library that I had created that contains a collection of my Nested ESXi and VSAN OVF Templates that anyone with a vSphere 6.0 or vCloud Director 5.x environment could subscribed to. This custom Content Library is also known as a 3rd Party Content Library and is one of the capabilities that I am most excited about with the new Content Library feature in vSphere 6.0. The reason I think this is such a cool capability is that I believe it opens up a wide range of use cases not only for our customers but also for our partner eco-systems enabling ease of content management and content distribution for VMware customers.

Just imagine, a vendor who might be providing Virtual Appliances (OVF/OVA), ISOs, VIBs, patches, workflows, blueprints, scripts, etc. can host a 3rd Party Content Library and as a customer you can easily subscribe to a particular repository. Once subscribed, you can either have the content automatically downloaded or synchronized on-demand. In fact, customers can also create their own 3rd party Content Library and distribute their own content within their own organization (local and remote sites) and even be able to share it with others in the VMware community.

Before I jump into the process of creating your own 3rd Party Content Library, I wanted to quickly go over the different options for publishing and subscribing to a Content Library. Below is a diagram to help you visualize the different options you have for publisher and subscriber sources.

Here is the same information as shown in the diagram but in a table format:

Publisher	Subscriber
vCenter Server	vCenter Server
vCloud Director	vCenter Server
3rd Party	vCenter Server
3rd Party	vCloud Director

Note1: Although the diagram technically shows a 5th scenario where a vCenter Server is the publisher and vCloud Director is the Subscriber, this particular flow is not entirely functional at the moment. Though it will work, it will only allow you to sync non-VM Templates and a different version of the script would be needed. This is a known issue and hopefully it will be resolved in a future update within vCloud Director.

Note2: For 2nd scenario where vCloud Director is a Publisher and vCenter Server is a Subscriber, On-Demand synchronization and overriding an existing OVF Library is currently not supported.

Here is the high level workflow for creating your own 3rd Party Content Library:

Create directory structure layout for your content
Create the JSON metadata index
Publish the 3rd Party Content Library using HTTP(s)
(Optional) Periodically update the JSON metadata index on new or updated content

Step 1 - Here is an example of what your directory structure layout should look like:

Step 2 - To assist with the JSON metadata indexing, Eric Cao, one of the Content Library Engineers was kind enough to provide the following Python script called make_vcsp_2018.py which you will need to download onto a system which has the Python interpreter running

UPDATE (07/26/18) - You can also create 3rd party content library directly on Amazon S3 or Microsoft Azure Blob using the updated versions of the script make_vcsap_2018.py and make_vcsap_2022.py respectively

UPDATE (05/29/17) - Thanks to Eric, the script now supports both Python2/3 as well as a new 4th optional command-line argument for performing md5check (Default: true)

Step 3 - The make_vcsp_2018.py script accepts two arguments: the first is the display name of the Content Library and the second argument is the path to the root directory of your Content Library. Here is an example:

python make_vcsp_2018.py vghetto-content-library /Users/lamw/desktop/vghetto-content-library

Once the script has finished building the metadata index, we can now take a look at our content and you will notice some additional files that have been added to your Content Library directory as seen in the screenshot below:

You will notice two new files at the root directory of your 3rd Party Content Library. The first is the items.json which provides reference to all the files within the library and you will see that each subsequent directory will also contain an individual item.json file describing the content within the local directories. The lib.json is the 3rd Party Content Library endpoint and is what you will be specify when subscribing to a 3rd Party Content Catalog and by providing the absolute URL to this file when setting up a new Content Library.

Step 4 - To publish the 3rd Party Content Library, you just need to host the content somewhere that supports HTTP(s), this can be as simple as using a regular Web Server to something like an Object Store such as Amazon S3.

Note: If you plan to use a non-traditional Web Server for hosting your 3rd Party Content, ensure that the root URL is the same for accessing all files and can not change. For example, you will not be able to host a 3rd Party Content Library using Dropbox as the root URL changes for each and every file.

Step 5 (Optional)- Periodically you will most likely want to add new content and files to your 3rd Party Content Library and as part of that process you will also need to update the metadata index. You can easily do so by re-running the make_vcsp_2018.py script against your existing 3rd Party Content Library. The script will only generate new metadata index for files that have been newly added and for any changes to existing files, the appropriate metadata files will only updated.

As you can see the process in creating and publishing a 3rd Party Content Library is pretty straight forward and simple. I hope to see more of our partners start to publish their own 3rd Party Content Libraries, I think it would be extremely beneficial for VMware customers and I look forward to hearing about them :). If you have created your own 3rd Party Content Library and would like to share the details, feel free to leave a comment and I will be aggregating a list below so that you can easily find them all in one place.

Published 3rd Party Content Libraries

Author	Library Content	Link
William Lam	Nested ESXi & VSAN OVF Templates	Subscribe to vGhetto Nested ESXi Template Content Library in vSphere 6.0
Alan Renouf	Some sample OVF/OVAs	Subscribe to Alan's Content Library in vSphere 6.0