WilliamLam.com

Exploring VSAN APIs Part 7 - VSAN Datastore Folder Management

03.26.2014 by William Lam // 1 Comment

In addition to storing your Virtual Machines, you may also want to use the VSAN Datastore to store your operating system ISOs in case you do not have an external repository and would like to keep everything local. If you use the vSphere Web Client to perform this operation, you will notice that you will need to first create a top-level directory before you can upload an ISO or a file to the VSAN Datastore.

The reason for this is that the VSAN Datastore does not support files in the top level directory, so you will have to first create a top-level directory and then upload the files under that directory. To create these top-level directories, a new DatastoreNameSpaceManager is introduced in the vSphere 5.5 API which manages the creation and deletion of these directories. Once these top-level directories exists, you can then use the regular fileManager API to manage your files and sub-directories within the VSAN Datastore. To demonstrate the creation of a top-level directory and sub-directory in a VSAN Datastore, I have created a sample vSphere SDK for Perl script called vsanDatstoreFolderMgmt.pl

Disclaimer: These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

In this example, I will create a top-level directory in the VSAN Datastore called ISO and then a sub-directory under that called Linux by running the following command:

./vsanDatstoreFolderMgmt.pl --server vcenter55-1.primp-industries.com --username root --vsan-datastore vsanDatastore --root-folder ISO --sub-folder Linux

The script will first call the DatastoreNameSpaceManager CreateDirectory() API method which will then create the top-level directory and then using the fileManager's MakeDirectory() API method to create the sub-directory. If we take a look at our VSAN Datastore using the vSphere Web Client, we can see that our new top-level directory has been created along with our sub-directory.

For deleting sub-directories and the top-level directories, there is the DeleteDatastoreFile_Task() and DeleteDirectory() API method respectively.

A kitten is also harmed, every time you edit a VMX file by hand

03.25.2014 by William Lam // 4 Comments

Following up on a recent article that I wrote a kitten dies every time you query the VCDB, I wanted to raise another topic that could also bring harm to kittens as well as making me cringe (JK on the harming of kittens). The topic, as you might have guessed is about editing a Virtual Machine's VMX configuration file by hand which comes up some what frequently from customers. There are many valid reasons on why a customer would need to perform such an operation such as enabling Change Block Tracking for efficient Virtual Machine backups to the hardening of a Virtual Machine based on the vSphere Security Hardening Guideline, the list just goes on and on. Outside of extreme troubleshooting scenarios (debatable unless directed by VMware GSS), you should be leveraging the vSphere API to automate these changes and instead of manually editing the VMX file by hand.

Disclaimer: No kittens were harmed during the writing of this article

There are several disadvantages when editing a VMX file by hand:

Very error prone, you might get it right for the first VM but what about 500 VMs with couple of settings each?
VMX editing requires the VM to be powered off
Requires ESXi host to have SSH enabled unless you plan to manually download/re-upload the edited VMX file
Changes to VMX file must also be reloaded for changes to take affect

I think there are two main reasons customers are manually editing VMX files by hand. First, it might seem to be the quickest way but looking at the list of disadvantages, you will soon realize it really is not when you need to scale this change beyond a single Virtual machine. The second reason I think is the perceived intuitiveness of the change. I mean, all configuration changes made to a Virtual Machine whether it is using the vSphere Web/C# Client or vSphere API will need to eventually persist onto disk and those changes is what is saved into Virtual Machine's VMX configuration file. However, that is not the only way nor would I say the "correct" way of modifying your Virtual Machines when it comes to these and other settings. As I have already mentioned, you can already use the vSphere Web/C# Client to make these changes which in turn leverage the vSphere API. So why would it be any different if you were to perform these operations outside of a UI?

In fact, the vSphere API abstracts away the complexity of needing to edit the VMX file and provides nice programmatic/scripting interfaces to perform some of these Virtual Machine configurations. Taking our two existing examples, Change Block Tracking is actually exposed through a simple boolean property called changeTrackingEnabled and you can just call the ReconfigVM_Task() method to enable this functionality to ensure that you have efficient Virtual Machine backups. You do not need to manually tweak each Virtual Disk since that will automatically be handled by vSphere once this has been configured.

The second example is around hardening your Virtual Machines based on the vSphere Security Hardening Guide. Luckily, to implement those settings among other Virtual Machine Advanced Settings, it just simply an array property called extraConfig that accepts a list of key/value pairs. In fact, you can easily automate the configuration of one or dozen parameters using the scripts found in this blog article called Accessing Virtual machine Advanced Settings which I wrote a couple years back. If you want to automatically harden your Virtual Machines based on a set of pre-defined settings, you can also take a look at my Automate the hardening of your Virtual Machine VMX Configurations article.

The point that I am trying to make here is that though these changes may eventually end up in the Virtual Machines VMX configuration file, there are better, easier and more efficient ways of making these changes through the use of the vSphere API and that is one of the many benefits of using an API. If you have gotten this far in the article, then I would like to conclude with some of the benefits I mentioned earlier that you will not find if you were to manually edit a VMX file by hand. The first is that the Virtual Machine Advanced Settings can be applied while the Virtual Machine is up and running, yes the changes will not go into effect, but you can at least stage the changes. If really can not take down time of the Virtual Machine to completely power off and power on for the changes to reload, another trick that I learned from a friend in the VMware Community is to vMotion the Virtual Machine from one host to another. By doing a vMotion, a new VMX will be created on the destination host and the Virtual Machine Advanced Settings will then become active as a new VMX process is spawned. This is not something you can do if you manually edited the VMX file by hand 🙂

Quick Tip - Don't always assume your local HDs will be claimed correctly

03.21.2014 by William Lam // Leave a Comment

For whatever reason I woke up super early this morning around 4am PST and since I could not go back to sleep, I figure I might as well finish re-building one of my lab environments. I was trying to create a VSAN Disk Group using ESXCLI, but ran into the following error: Stamping non-local disks requires a pre-created vsan cluster If I run the following ESXCLI command, we can clearly see the Hitachi disks that I have are not being recognized as a local device:

esxcli storage core device list

I already know that some disks may not be recognized as a local device which VSAN requires and this is nothing new, in fact Duncan Epping even shows you how to get around this in this article here. Seeing that it was so early in the US, I figure Duncan was probably awake and wanted to run a few things by him. It was while he was looking around in the system did he notice a stranger issue.

It turns out my local Hitachi disks were being claimed by the VMW_SATP_DEFAULT_AA (ALUA) SATP plugin which was kind of strange as I would have expected to be claimed by VMW_SATP_LOCAL instead. I decided to take a look at the SATP claim rules to see why this was the case by running the following ESXCLI command:

esxcli storage nmp satp rule list

It turns out, for identifying Hitachi storage devices, the SATP rules is quite generic and keys off of the vendor name only and hence the assignment of the ALUA SATP plugin is choosen.

VMW_SATP_DEFAULT_AA HITACHI system inq_data[128]={0x44 0x46 0x30 0x30} VMW_PSP_RR
VMW_SATP_DEFAULT_AA HITACHI system

In my opinion this can be problematic as the plugin can potentially cause strange IO or pathing behaviors as it expects one thing but is getting something completely different. I suspect this plugin was probably meant for an Hitachi storage array like an HDS instead of a local device. Once I was able to narrow this down, I just needed to create a new rule that would override the system defaults.

To verify this, I created a new rule based on the vendor name just for testing purposes which will have priority over the default system rules. When creating custom SATP rules, you can filter on a variety of attributes such as the model, transport, controller, target, adapter, etc. Depending on your use case, you may want to be generic or quite specific. The following ESXCLI command will create the new rule and assign it the VMW_SATP_LOCAL plugin:

esxcli storage nmp satp rule add -V HITACHI -P VMW_PSP_FIXED -s VMW_SATP_LOCAL

Once that rule has been added, I can now finish my original task which is to mark my Hitachi drives as "local" and reclaim these devices by running the following ESXCLI commands:

esxcli storage nmp satp rule add -s VMW_SATP_LOCAL -o "enable_local" -d [DEVICE]
esxcli storage core claiming reclaim -d [DEVICE]

If we now take a look at our device, we can see that are rule was activated and if we perform the following ESXCLI command, we can now see the device is "local":

esxcli storage core device list

The lesson here, do not assume your local devices and potentially even remote devices will be claimed correctly. There maybe vendor best practices that differ from the out out box rules and you should always do your due diligence to either verify yourself or work with your vendors to confirm the configurations.