WilliamLam.com

Automating SSL Certificate Expiry Validation for vCenter Server + ESX(i) Hosts

by // 4 Comments

As many of you know, it is a best practice to replace VMware's self-signed SSL certificates that are included in the vCenter Server (Windows & VCSA) and ESX(i) hosts to prevent or help reduce MiTM (Man in The Middle) attacks. If you are looking for more details on how to replace the default SSL certificates, you should take a look at the fantastic articles written by Michael Webster who details the process, provides some troubleshooting steps and best practices for SSL certificate replacement.

Replacing the default self-signed SSL certificate is just one part of the process, but you also need to check to ensure the certificates are still valid and have not expired. If you already have a process in place or a system that automatically does this for you, that is great. If you do not, you should definitely validate that your SSL certificates are valid on a regular basis.

I recently stumbled onto a nifty open source tool called ssl-cert-check that can help with validating expiration of SSL certificates found on vCenter Server(s) and ESX(i) hosts or any other SSL enabled host for that matter. This utility is just a shell script (specifically bournce shell) wrapping the common openssl utility found on most UNIX/Linux systems and does not require any login credentials to the remote hosts to validate the SSL certificate.

To use the script, you can visit the website here and download it to a system that has openssl installed (in my home lab, I used vMA).

It took me awhile to find the script, but it's located on the right side of the screen where it says "Website". You can also download it from the command-line using wget if you have direct/proxy access to the internet:

wget http://freecode.com/urls/353b752faa208fca12bc0091c742f764 -O ssl-cert-check

Note: Don't forget to set the execute permission on the script (chmod +x ssl-cert-check) else you will get permission denied when trying to run the script.

The script can be executed interactively by specifying the -s option for server and -p for the port. You can also specify the issuer of the certificate by using the -i option. Below is a screenshot of running the ssl-cert-check against a vCenter Server:

You can also run the script in batch mode by specifying -f option which accepts a list of servers in FQDN along with the port number. Using this feature of the script, you can easily run this script against all your vCenter Server(s) and ESX(i) hosts to ensure that their SSL certificates are still valid.

If you already have a list of hosts you want to check, then you can easily create a new file with the hostname and port. Though if you do not have one handy, I wrote a quick vSphere SDK for Perl script called generateESXiHostsList.pl that helps automate the creation of the output file containing all ESX(i) hosts when connecting to a vCenter Server. To use the script, you just need to have vCLI installed on a system or use vMA.

The script accepts one options which is --output which specifies the name of the output file to be created:

If we "cat" the file out, we can see it looks like the following:

vesxi50-1.primp-industries.com 443
vesxi50-2.primp-industries.com 443
vesxi50-3.primp-industries.com 443
vesxi50-4.primp-industries.com 443

Let's now run the ssl-cert-check against the list of ESX(i) hosts using the -f option and see if we have any hosts with expired certificates:

Uh oh, it looks like we have two hosts with some problems. We can see one host that already has an expired SSL certificate and another one that will be expiring in 10 days. We better take a look at these and get them replaced soon!

There are additional options in the ssl-cert-check script including the ability to email the results or run as a nagios check. You can easily schedule a cron job to automate this script to run every week and grepping for the keyword "Expiring" to alert you of any hosts that have expiring SSL certificates. As you can see, it is not only important to replace the default self-signed SSL certificates in your environment, but you need to validate on a routinely basis your your certificates are still valid.

VM Provisioning on Datastore Clusters in vSphere 5

by // 24 Comments

Last year I wrote an article called Automating Storage DRS & Datastore Cluster Management in vSphere 5 and I provided a pretty comprehensive vSphere SDK for Perl script to help administrators automate Storage DRS configurations. These past few months I have noticed an increase in interest on the VMTN developer forums relating to Storage DRS. Majority of the questions has been related to which vSphere API methods to use and how to use these methods for cloning VMs to datastore clusters.

If you have cloned a VM before, the underlying vSphere API method being used is the CloneVM_Task(), but when cloning a VM to a datastore cluster, a different API must be used.

In vSphere 5, VMware introduced several new API methods in the StorageResourceManager and the two specific ones relating to provisioning VMs are RecommendDatastores() and ApplyStorageDrsRecommendation_Task(). The process to clone a VM to a datastore cluster is a two step process:

  1. Call RecommendDatastores() which accepts very similar input as the CloneVM_Task(). In addition, you need to specify the datastore cluster also known as a Storage Pod in the vSphere API as well as the "type" (create, clone, relocate, reconfigure), in this example we will be performing a clone operation. This method will then generate a recommendation on where to place the VM which is based on the SDRS placement engine. No provisioning will occur at this point, just a placement recommendation.
  2. To perform the actual provision of the VM, you will need to call ApplyStorageDrsRecommendation_Task() which accepts a recommendation ID that was generated from the first step. Once the recommendation is applied, the provisioning of the VM will start just like it did when you called the CloneVM_Task().

Note: The RecommendDatastores() will return multiple recommendations, the best one will be first entry in the array. This is the same algorithm used when performing this same operation in the vSphere Client, it also selects the first recommendation.

Now that we understand how the APIs work, let's take a look at how we can leverage this in a script for some automation! Here is a simple vSphere SDK for Perl script called datastoreClusterVMProvisioning.pl which allows you to clone an existing VM onto a datastore cluster. You will need a system that has vCLI 5.0 installed or you can use VMware vMA 5 to run the script. You will also need to connect to a vCenter Server 5 for all SDRS operations.

The script requires 4 input parameters:

  • vmname - Name of the VM you wish to clone from
  • clonename - Name of the cloned VM
  • vmfolder - Name of a vCenter folder
  • datastorecluster - Name of a datastore cluster

Here is a screenshot of cloning an existing VM onto a datastore cluster:

The script is pretty straight forward and it can easily be adapted to include other configurations as required in your own environment.

Hopefully this gives you a better idea on how to leverage the new provisioning APIs for Storage DRS and start automating your VM deployments onto datastore clusters and get the benefits Storage DRS in your vSphere environment.

Automating Dead Space Reclamation in ESXi 5.0u1

by // 4 Comments

VMware released vSphere 5.0 Update 1 last week, which mainly included bug fixes but it also brought back one very cool feature that was initially introduced with the release of vSphere 5.0 called Thin Provisioning UNMAP primitive for an ESXi host. You can read more about the details in this article by my colleague Cormac Hogan.

As you can see from From Cormac's article, the process of reclaiming of dead space on a thin provisioned LUN is currently a manual process, but does it have to be? The answer is, No of course, we can can definitely automate this!

Disclaimer: This script is not officially supported by VMware, please test this in a development environment before using on production system. This script is provided as an example on you can automate this manual process.

Before you proceed, please understand that the UNMAP operation can potentially take a few minutes up to a few hours depending on the size of your datastore and how your array handles this operation. You should consider performing this operation during a maintenance window or during off peak hours else you could impact VMs residing on the datastore. You should also ensure you have a VAAI-capable storage array before performing running this script.

I wrote a simple shell script called reclaimMyDeadsSace.sh which needs to be executed on the ESXi Shell via SSH. The script will also perform some validation such as ensuring you are running ESXi 5.0 Update 1 and that your host is in maintenance mode as a per-caution to ensure no running VMs are on the host during this process.

You will only need to run the script on one of the hosts connected to all the datastores you wish to reclaim dead space on. You may use scp or WinSCP to transfer the script to your ESXi host and ensure you set the execute permission on the script (chmod +x reclaimMyDeadSpace.sh)

The script can be executed in two ways:

  1. Identify ALL VMFS3 and VMFS5 volumes and perform the reclaim based on the percentage entered by the user
  2. Reclaim on specific datastores specified by the user as well as the percentage to be reclaimed (this is recommended, that way script does not choose all datastores including local ones)

Here is an example of selecting ALL VMFS3 and VMFS5 datastores to reclaim 60% of free space:

Here is an example of selecting just 4 datastores specified in a file and we will be reclaiming 60% of free space:

In this example, we created a file called "datastore_list.txt" (you may name the file anything you want) which contains the following:
iSCSI-1
iSCSI-2
iSCSI-3
iSCSI-4

So if you are using thin provisioned LUNs and would like to reclaim some of that dead space back and have a VAAI-capable storage array, be sure to check out the UNMAP functionality in ESXi 5.0u1.