vmx

Configuring a "Whitelist" for VM advanced settings in vCloud Director

05.10.2014 by William Lam // 3 Comments

The ability to preserve a Virtual Machine's "virtual hardware personality" during an export has been around since the release of vSphere and vCloud Director 5.1 also known as a Lossless OVF Export. However, when it comes time to import that Virtual Machine back into your environment, will it be a "lossless" import? For vSphere, the answer is yes as long as you click on the "Accept extra configuration options" during the import of your OVF/OVA in the vSphere Web Client.

For vCloud Director however, the answer is sadly no. If you try to import an OVF/OVA that contains additional VM advanced settings, vCloud Director will automatically strip away these parameters before importing the Virtual Machine. As annoying as this is for end users, there is a very good reason why this happens.

In a public cloud you do not know who your users are and whether they have malicious intentions. Since the "Extra Configuration" of a Virtual Machine can not be validated, the safest way to guard against this type of a security risk is by sanitizing the content prior to upload.

Although I have known about this default behavior in vCloud Director for some time now, it was only in a recent meeting with some of our internal folks managing our OneCloud environment did I realize how annoying this behavior was for "private" clouds as well. I was looking to import an existing OVF that contains several VM Advanced settings, but realized this would not be supported due to the sanitization of the OVF.

The other issue is that vApps within OneCloud are also exported/re-imported into other vCloud Director environments. Even if our Cloud Administrator were to make a one time exception and manually add these these VM Advanced Settings, they would automatically be stripped away the next time they were re-imported into another environment. After our meeting, I thought there had to be a better way! (at least I hope)

Disclaimer: This may or may not be officially supported by VMware, please use at your own risk.

In doing a quick bit of research, it turns out this is actually a pretty elegant solution that allows you to create a "Whitelist" of trusted VM advanced settings that would be accepted as part of the import process in vCloud Director. To configure this "Whitelist", you will need to add a new category called ExtraConfigWhitelist into the vCloud Director Database for each and every VM advanced setting that you wish to allow.

To demonstrate this functionality, I have the latest vCloud Director appliance (I believe this should also work on 5.1 as well) running and I would like to add the following VM advanced settings to the whitelist:

scsi0:0.virtualSSD
scsi0:1.virtualSSD
scsi0:2.virtualSSD
disk.enableUUID

Before starting, you will need to SSH to our vCloud Director instance and stop the VCD service by running the following command:

/etc/init.d/vmware-vcloud stop

Next, you will need to switch over to the "oracle" user by running the following command:

su - oracle

sqlplus vcloud/VCloud

Once you have logged in, you will need to insert the following rows into the "config" table of VCD Database:

INSERT INTO config (config_id, cat, name, value, sortorder) VALUES (seq_config.NextVal, 'ExtraConfigWhitelist', 'scsi0:0.virtualSSD', 'true', 0);
INSERT INTO config (config_id, cat, name, value, sortorder) VALUES (seq_config.NextVal, 'ExtraConfigWhitelist', 'scsi0:1.virtualSSD', 'true', 0);
INSERT INTO config (config_id, cat, name, value, sortorder) VALUES (seq_config.NextVal, 'ExtraConfigWhitelist', 'scsi0:2.virtualSSD', 'true', 0);
INSERT INTO config (config_id, cat, name, value, sortorder) VALUES (seq_config.NextVal, 'ExtraConfigWhitelist', 'disk.enableUUID', 'true', 0);

Note: The syntax used here is for an Oracle database. You may need to tweak it if you are using MS SQL Server.

Finally, for the changes to go live you just need to start up your VCD instance:

/etc/init.d/vmware-vcloud start

Once the VCD service has returned, you can now import any OVF/OVA that contains a list of approved VM advanced settings and VCD will ensure these settings are preserved as part of the import. I hope this solution is something our OneCloud team will consider as it solves my immediate problem but I also think this can greatly benefit the overall use base, especially those building out content.

Note: Even though this "whitelist" option is available, I would still recommend you carefully determine which VM advanced setting is required for your environment and minimize the size of the list as much as possible from an operational perspective, especially when it comes to new installation or upgrades.

A kitten is also harmed, every time you edit a VMX file by hand

03.25.2014 by William Lam // 4 Comments

Following up on a recent article that I wrote a kitten dies every time you query the VCDB, I wanted to raise another topic that could also bring harm to kittens as well as making me cringe (JK on the harming of kittens). The topic, as you might have guessed is about editing a Virtual Machine's VMX configuration file by hand which comes up some what frequently from customers. There are many valid reasons on why a customer would need to perform such an operation such as enabling Change Block Tracking for efficient Virtual Machine backups to the hardening of a Virtual Machine based on the vSphere Security Hardening Guideline, the list just goes on and on. Outside of extreme troubleshooting scenarios (debatable unless directed by VMware GSS), you should be leveraging the vSphere API to automate these changes and instead of manually editing the VMX file by hand.

Disclaimer: No kittens were harmed during the writing of this article

There are several disadvantages when editing a VMX file by hand:

Very error prone, you might get it right for the first VM but what about 500 VMs with couple of settings each?
VMX editing requires the VM to be powered off
Requires ESXi host to have SSH enabled unless you plan to manually download/re-upload the edited VMX file
Changes to VMX file must also be reloaded for changes to take affect

I think there are two main reasons customers are manually editing VMX files by hand. First, it might seem to be the quickest way but looking at the list of disadvantages, you will soon realize it really is not when you need to scale this change beyond a single Virtual machine. The second reason I think is the perceived intuitiveness of the change. I mean, all configuration changes made to a Virtual Machine whether it is using the vSphere Web/C# Client or vSphere API will need to eventually persist onto disk and those changes is what is saved into Virtual Machine's VMX configuration file. However, that is not the only way nor would I say the "correct" way of modifying your Virtual Machines when it comes to these and other settings. As I have already mentioned, you can already use the vSphere Web/C# Client to make these changes which in turn leverage the vSphere API. So why would it be any different if you were to perform these operations outside of a UI?

In fact, the vSphere API abstracts away the complexity of needing to edit the VMX file and provides nice programmatic/scripting interfaces to perform some of these Virtual Machine configurations. Taking our two existing examples, Change Block Tracking is actually exposed through a simple boolean property called changeTrackingEnabled and you can just call the ReconfigVM_Task() method to enable this functionality to ensure that you have efficient Virtual Machine backups. You do not need to manually tweak each Virtual Disk since that will automatically be handled by vSphere once this has been configured.

The second example is around hardening your Virtual Machines based on the vSphere Security Hardening Guide. Luckily, to implement those settings among other Virtual Machine Advanced Settings, it just simply an array property called extraConfig that accepts a list of key/value pairs. In fact, you can easily automate the configuration of one or dozen parameters using the scripts found in this blog article called Accessing Virtual machine Advanced Settings which I wrote a couple years back. If you want to automatically harden your Virtual Machines based on a set of pre-defined settings, you can also take a look at my Automate the hardening of your Virtual Machine VMX Configurations article.

The point that I am trying to make here is that though these changes may eventually end up in the Virtual Machines VMX configuration file, there are better, easier and more efficient ways of making these changes through the use of the vSphere API and that is one of the many benefits of using an API. If you have gotten this far in the article, then I would like to conclude with some of the benefits I mentioned earlier that you will not find if you were to manually edit a VMX file by hand. The first is that the Virtual Machine Advanced Settings can be applied while the Virtual Machine is up and running, yes the changes will not go into effect, but you can at least stage the changes. If really can not take down time of the Virtual Machine to completely power off and power on for the changes to reload, another trick that I learned from a friend in the VMware Community is to vMotion the Virtual Machine from one host to another. By doing a vMotion, a new VMX will be created on the destination host and the Virtual Machine Advanced Settings will then become active as a new VMX process is spawned. This is not something you can do if you manually edited the VMX file by hand 🙂

How to change hardware serial number for Mac OS X Guest?

10.25.2013 by William Lam // 5 Comments

There was an interesting question that was asked the other day about changing the hardware serial number for an Apple Mac OS X guest as the generated serial number is not compatible with services such as Apple Caching Service or iMessage. I recall seeing this question get asked awhile back, but I could not immediately find the answer but thanks to Darius Davis (VMware Engineer) who provided the quick answer.

We have a facility to generate a "short" serial number which should be suitable for recent Apple software. The option is enabled by default for OS X 10.9 guests. To enable it for earlier guest OS versions, you'll need to power off your virtual machine and edit its configuration to add the following option:

SMBIOS.use12CharSerialNumber = "TRUE"

As mentioned by Darius, if you are running Mac OS X VM prior to 10.9 (Mavericks) you will need to add the following advanced VM setting by first powering it off and then add the above setting. There are two recommended ways of performing this change using either the vSphere C# Client or vSphere Web Client and instructions are listed below.

Note: Though you can also edit the VMX configuration file by hand, for those that are not familiar on how to reload the configuration file, it is best you use the UI.

vSphere C# Client:
Edit Settings -> VM Options -> Advanced -> Edit Configurations

vSphere Web Client:
Edit Settings -> Options -> Advanced -> General -> Configuration Parameters

Once you have added the advanced setting, you can now power back on your Mac OS X VM and when you click on the "About this Mac" option on the upper left hand side of the Apple icon you should see the Mac OS X version string. Click on the version string twice and you should now see the serial number that is generated which should not be longer than 12 characters.

As of writing this article the latest Mac OS X 10.9 (Mavericks) is not yet officially on the VMware HCL for latest release of ESXi 5.5 as the OS just came out recently, however it is still possible to create the a Mac OS X 10.9 guest using the new vSphere Web Client. Be sure to keep your eyes on the VMware HCL for support of Mac OS X 10.9 on ESXi 5.5 here.