vSphere API – Page 26

A kitten is also harmed, every time you edit a VMX file by hand

03.25.2014 by William Lam // 4 Comments

Following up on a recent article that I wrote a kitten dies every time you query the VCDB, I wanted to raise another topic that could also bring harm to kittens as well as making me cringe (JK on the harming of kittens). The topic, as you might have guessed is about editing a Virtual Machine's VMX configuration file by hand which comes up some what frequently from customers. There are many valid reasons on why a customer would need to perform such an operation such as enabling Change Block Tracking for efficient Virtual Machine backups to the hardening of a Virtual Machine based on the vSphere Security Hardening Guideline, the list just goes on and on. Outside of extreme troubleshooting scenarios (debatable unless directed by VMware GSS), you should be leveraging the vSphere API to automate these changes and instead of manually editing the VMX file by hand.

Disclaimer: No kittens were harmed during the writing of this article

There are several disadvantages when editing a VMX file by hand:

Very error prone, you might get it right for the first VM but what about 500 VMs with couple of settings each?
VMX editing requires the VM to be powered off
Requires ESXi host to have SSH enabled unless you plan to manually download/re-upload the edited VMX file
Changes to VMX file must also be reloaded for changes to take affect

I think there are two main reasons customers are manually editing VMX files by hand. First, it might seem to be the quickest way but looking at the list of disadvantages, you will soon realize it really is not when you need to scale this change beyond a single Virtual machine. The second reason I think is the perceived intuitiveness of the change. I mean, all configuration changes made to a Virtual Machine whether it is using the vSphere Web/C# Client or vSphere API will need to eventually persist onto disk and those changes is what is saved into Virtual Machine's VMX configuration file. However, that is not the only way nor would I say the "correct" way of modifying your Virtual Machines when it comes to these and other settings. As I have already mentioned, you can already use the vSphere Web/C# Client to make these changes which in turn leverage the vSphere API. So why would it be any different if you were to perform these operations outside of a UI?

In fact, the vSphere API abstracts away the complexity of needing to edit the VMX file and provides nice programmatic/scripting interfaces to perform some of these Virtual Machine configurations. Taking our two existing examples, Change Block Tracking is actually exposed through a simple boolean property called changeTrackingEnabled and you can just call the ReconfigVM_Task() method to enable this functionality to ensure that you have efficient Virtual Machine backups. You do not need to manually tweak each Virtual Disk since that will automatically be handled by vSphere once this has been configured.

The second example is around hardening your Virtual Machines based on the vSphere Security Hardening Guide. Luckily, to implement those settings among other Virtual Machine Advanced Settings, it just simply an array property called extraConfig that accepts a list of key/value pairs. In fact, you can easily automate the configuration of one or dozen parameters using the scripts found in this blog article called Accessing Virtual machine Advanced Settings which I wrote a couple years back. If you want to automatically harden your Virtual Machines based on a set of pre-defined settings, you can also take a look at my Automate the hardening of your Virtual Machine VMX Configurations article.

The point that I am trying to make here is that though these changes may eventually end up in the Virtual Machines VMX configuration file, there are better, easier and more efficient ways of making these changes through the use of the vSphere API and that is one of the many benefits of using an API. If you have gotten this far in the article, then I would like to conclude with some of the benefits I mentioned earlier that you will not find if you were to manually edit a VMX file by hand. The first is that the Virtual Machine Advanced Settings can be applied while the Virtual Machine is up and running, yes the changes will not go into effect, but you can at least stage the changes. If really can not take down time of the Virtual Machine to completely power off and power on for the changes to reload, another trick that I learned from a friend in the VMware Community is to vMotion the Virtual Machine from one host to another. By doing a vMotion, a new VMX will be created on the destination host and the Virtual Machine Advanced Settings will then become active as a new VMX process is spawned. This is not something you can do if you manually edited the VMX file by hand 🙂

Exploring VSAN APIs Part 6 – Modifying Virtual Machine VM Storage Policy

03.20.2014 by William Lam // 6 Comments

One of the biggest benefit of VSAN is the ability to specify granular storage policies on a per Virtual Machine basis. These storage policies is managed through VMware's Storage Policy Based Management system and is automatically enforced by VSAN to ensure compliance. A VM Storage Policy can be assigned during the initial deployment of a Virtual Machine or it can be modified afterwards, for example if the Virtual Machine's SLA's has changed because the workload has changed. From the vSphere Web Client, modifying a Virtual Machine's VM Storage Policy is simply selecting the VM Storage Policy and re-applying which is also available programmatically through the vSphere API.

Using the vSphere API method ReconfigVM_Task(), you will be able to modify the VM Storage Policy for the VM Home Namespace and/or individual Virtual Disks. To modify the VM Home Namespace, there is a property defined at the root of the Virtual Machine config spec called vmProfile which accepts the VM Storage Policy ID extracted from the SPBM API. To modify the VM Storage Policy for an individual Virtual Disk, you will need to set the profile property which is exposed on a Virtual Device with the VM Storage Policy ID. To demonstrate this functionality, I have created a sample vSphere SDK for Perl script called changeVMStoragePolicy.pl

Disclaimer: These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

In my environment, I have a Virtual Machine called VM1 which has been defined with a VM Storage Policy called "Copper" as seen in screenshot below:

Let's say I want to change the Virtual Machine's VM Storage Policy to another policy called "Aluminum", I first need to extract the VM Policy ID from SPBM API and then pass it into the script like the following:

./changeVMStoragePolicy.pl --server vcenter55-1.primp-industries.com --username root --vmname VM1 --profileid cd6908b2-0704-4733-ad9b-a9a8f200ab0a

Once the Virtual Machine has been reconfigured, we can then take a look in our vSphere Web Client and we can see the VM Storage Policy has now been changed and VSAN will automatically enforce these new requirements.

If you wish to assign a VM Storage Policy as part of a new Virtual Machine creation, you just need to set the vmProfile and profile properties which is similar to a reconfiguration operation.

A kitten dies, every time you query the VCDB

03.19.2014 by William Lam // 14 Comments

Okay, maybe I am being a bit dramatic 😉 However, this is what crosses my mind every time I hear someone trying to query for something in the VCDB (vCenter Server Database) instead of using the vSphere API. Every so often I see a customer request asking for a specific SQL Query to find something in the VCDB. It just baffles me on why someone would want to go through such pain, not to mention this is not supported nor a recommended best practice.

Disclaimer: No kittens were harmed during the writing of this article

There are many disadvantages by going directly to the VCDB versus going through the vSphere API:

There are no guarantees on backwards compatibility of the database schema or database views from release to release
The data is in a very raw form, there is no abstraction which is usually provided by an API to make consumption easier
Risk of crafting queries that could negatively impact the performance of vCenter Server
Not officially supported by VMware, unless directed by VMware GSS
Pulling hair out while trying to understand the internal relationships of various objects, especially with a complex system like vCenter Server

Although the vSphere data is eventually persisted in the VCDB, the main purpose of the vSphere API, like many other software APIs is to provide a well defined interface for interacting with the underlying platform. I think there are still many vSphere/System Administrators who fear the word "API". Instead of being afraid of APIs, we should all embrace it and start to better understand how they work as it is just a way of interacting with the underlying system.

I can speak from my own personal experience, the first task for me out of college and into the real world was to look into building an application impact assessment report when an vSphere HA event occurred. We knew which Virtual Machines were restarted, but we did not have a good idea of what applications were affected. I had to correlate the impacted Virtual Machines with a list of applications and application owner from our homegrown CMDB which ran on MS SQL Server. At the time, I did not know any better and I thought best way of getting this information was to go directly to the VCDB (at the time this was Virtual Center 2.5). I found myself creating super complex SQL queries and trying to reverse engineer the relationships between all the different tables to get basic information from vCenter Server. If I knew what I know now back then, I would have gone straight to learning the VMware APIs, as I could have gotten everything I needed in just a couple of lines of code. After completing the project among few others which I leverage the VCDB directly, I realized that I needed to learn this VMware API. This early lesson has greatly helped me in my career to further automate IT/VMware infrastructure and help us move towards this new world of a Software-Defined Datacenter.

The diagram below provides an overview of some of the ways the vSphere API can be consumed by customers. An example is the vSphere Web Client and the legacy vSphere C# Client, this is a UI interface that customers can use to interact with the vSphere platform which in turns uses the vSphere API. There are also a variety of CLIs and scripting/programming languages that are built for easily extracting information and performing operations against vSphere that could be as simple as one-liner. The vSphere API can also be consumed by 3rd party companies to further abstract and provide new and interesting ways of interacting with vSphere, a great example of this is the CloudPhysics Card Builder platform that makes creating custom vSphere reports a breeze.

Note: The diagram above is just an example of some of the SDK/CLI/UIs that VMware provides to our customers. This is not meant as an exhaustive list and you can find all SDKs/CLIs for vSphere here.

Hopefully with this information, customers will better understand the benefits of using the vSphere API versus going directly to the VCDB.