WilliamLam.com

How to run a VM on Dropbox Storage

by // 2 Comments

In my previous blog post How to backup VMs in ESX onto Dropbox, I showed you how to backup your virtual machines on ESX to your Dropbox account, but who said it should stop there? I realized after playing with Dropbox on ESX, I had a crazy idea of trying to run a virtual machine that was stored in my Dropbox account. Since I am using the free Dropbox account, I have a maximum of 2GB of online storage and decided to spin up a tiny Linux virtual machine and upload it to my Dropbox account. Not surprised, I was able to register the virtual machine and fire it up and it worked like a charm.

Now, this got me thinking .... if I can run a virtual machine from Dropbox via ESX, could I get another ESX host to see this same Dropbox account? The answer is YES! One feature of Dropbox is the ability to access your files across multiple devices: Windows, Linux, Mac OSX desktop, iPhone, iPad, Andriod and of course ESX. For demonstration purposes, I created two virtual ESX 4.1 hosts called west and east and authorized both hosts to access my Dropbox account.

Here is a screenshot of the two ESX hosts via "My Computers" tab on Dropbox account:

Here is a screenshot of the small 1GB Linux Debian virtual machine I created and uploaded to my Dropbox account:

Here is a screenshot of both ESX hosts (east and west) accessing the directory of the small Linux virtual machine:

I registered the virtual machine on the west coast ESX host and here is a side by side screenshot of both vSphere Clients:

As you can see, the virtual machine is powered on and running. If we take a look at where the virtual machine configuration file and disks is stored, you will see it is currently on the Dropbox account accessed by "west" coast ESX host:

Here is a screenshot showing the registration of the small Linux VM on west and there is currently nothing registered on east:

Let's say we had a failure on the west coast ESX host and we need to access this virtual machine via east coast ESX host. I registered the virtual machine and powered it on and now the system is back up and running again:

Again, we can see the configuration and virtual disk is now being accessed from the Dropbox account by the east coast ESX host:

We also confirm that the virtual machine is now registered on the east coast ESX and there is nothing running on the west coast ESX host:

As you can see from this simple demonstration, you can easily run a virtual machine and have it accessible via multiple ESX host, though I would not recommend you go out and start putting your production VMs on Dropbox, but it is an interesting idea for cloud storage 😉

Note: While testing, I found the synchronization process between the two ESX host to be slightly off and were not always up to date when a change was made. I had to restart the Dropbox daemon several times after I created the virtual machine for the other ESX host to see the updated files. I am pretty sure it is an issue with the daemon versus dropbox, since I can see the files updated on the web browser immediately. I would be very careful to ensure that only one host is accessing the files, else you could see some discrepancies.

How to backup VMs in ESX onto Dropbox

by // 5 Comments

I previously wrote about backing up virtual machines directly from ESX and ESXi onto both Amazon S3 and Google Storage, but there was actually a third online file hosting company that I wanted to get working, Dropbox. Dropbox is a relatively new file hosting company that launched back in 2008 and has gain popularity for its ease of use and ability to access and share files across multiple devices. While researching Dropbox initially, I discovered there was a python-based CLI which I was hoping would install and function on ESX(i). This unfortunately did not pan out due to the various python library dependencies including a newer version of python.

While scouring the web, I recently found out that Dropbox actually released a Linux client binary and I thought I'd kick the tires and see if I could get it running on ESX(i). After a few minutes of testing, I found out that it was possible to get it running on classic ESX, but there are still certain python dependencies that prevent the Dropbox client to run on ESXi.

Before you begin, you will need to sign up for a free Dropbox account. With the free account you automatically get 2GB of free online storage, if you want more, you can pay for up to 100GB of online storage. The following has been validated on ESX 4.1, I have not tested this on any other ESX version and your results may vary. ESXi is not supported as mentioned earlier.

1. Download the latest Dropbox Linux Client here.

2. You will need to upload the Dropbox tar ball to your ESX host, you can use scp on UNIX/Linux or winSCP if you are using a Windows system.

3. You should not have the tar ball file sitting in the root directory of your ESX host:

4. You will now need to extract the contents of the tar ball, by running the following:

tar -zxvf dropbox-lnx.x86_64-0.7.110.tar.gz

5. You should now have a hidden directory called .dropbox-dist in your current working directory:

6. If you are starting the Dropbox client for the first time, it will default to using home directory of the current user to access your Dropbox share. In our case, it will be stored as /root/Dropbox which is probably not what you want. We will actually update our home directory to point to a VMFS volume path, by setting the HOME environmental variable:

As you can see, now our new home directory is set to a VMFS datstore. Once the Dropbox client starts, it will create a 64bit encoded string of the path which is stored in a configuration file once you have authorized the addition of this system to your Dropbox account.

Note: Once the default Dropbox path is set, you need to ensure that environmental HOME dir is always set to the one you specified above, else when you start Dropbox it will think it is a new setup.

7. We will now start the Dropbox daemon and ensure that we run it in the background:

If you have successfully started the Dropbox daemon, a unique URL will be generated based on your system which is used to authorize this system to access your Dropbox account. Take the URL and paste it into a browser, it should ask you to login to authorize the system.

8. If this is the first time you are using Dropbox, once you have signed in, you will be brought to the files tab in which all the folders and files that are currently accessible to you. By default, you will have a Public and Photos folder in which both are empty:

9. If you go back to your ESX host, you will now notice some output regarding nautils, you can ignore this error as the packages are not required for functionality:

10. Now, if you remember when we set the home directory to trick Dropbox to put folders under a VMFS datastore, you will now see a new directory called Dropbox which will contain the Public and Photos folder you saw on your web browser:

11. For demo purposes, I created a Backup directory in Dropbox root folder on the ESX host which will be visible from your web browser:

I then created a dummy 1MB VMDK in the same folder as if you were copying a VM to Dropbox account:

You can now go back to your web browser and see the VMDK file that was just created in the Backup directory:

There you have it, you can now transfer files or backup your virtual machines from your ESX host to your Dropbox account.

Earlier I mentioned that there are configuration files that tells the Dropbox client that this system is authorized to connect to your Dropbox account and it stores both the system ID along with the Dropbox path. If you do a long listing in your VMFS volume that was used to store your Dropbox folder, you will notice a hidden directory called .dropbox:

There are two database files, dropbox.db that contains the files and folder structures as it is being synced down to the host and host.db which contains the system's ID and Dropbox path which is encoded as 64bit string. You can decode and verify the path by using this website: http://webnet77.com/cgi-bin/helpers/base-64.pl

Here is what the host.db file looks like:

The second line contains the Dropbox path and you should not try to edit this file manually as it may cause synchronization issues. If you look at the Dropbox documentation, there is a python script that allows you to change the path but it requires sqlite3 to be available which is not available by default on ESX.

Automating ESXi 4.1 Kickstart Tips & Tricks

by // 25 Comments

While testing the new kickstart functionality in ESXi 4.1, I ran into a few issues trying to convert a classic ESX 4.x deployment to ESXi 4.1. I thought I share some of the tips and tricks I have learned, so others will not encounter the same issues.

Before diving in and creating an ESXi 4.1 kickstart configuration, make sure you spend some time going over the documentation provided by VMware, specifically the ESXi Installable and vCenter Server Setup Guide. 

UPDATE: For ESXi 5, Check out ESXi5 Kickstart Tips & Tricks

Tip #1

If you are going to specify a ks.cfg (kickstart configuration file) in your pxelinux file, make sure that the kickstart entry is appended after the *vmkboot.gz* but before *vmkernel.gz* entry as highlighted in green in the screenshot. If you place it anywhere else in the boot line option, you will receive an error that is not easy to diagnose. Also you want to make sure you add triple dashes (---) after the kickstart line following the required syntax for the boot options as highlighted in orange in the screenshot.

Tip #2

While developing and testing your ks.cfg, you may want to use the new dryrun parameter which parses your kickstart configuration file looking for syntax and formatting errors. In dryrun mode, no installation will be performed but you will be provided with a log of whether your ks.cfg had any errors, warnings or was successful in being validated.

The following screenshot shows a warning where I purposely left out --hostname entry which is generally recommended within the "network" portion of the ks.cfg:

If there are other errors or warnings, they will be displayed within this screen and you can login to the host to view the log for more details (esxi_install.log):

To login to the host, you will press "enter" and you will be prompted for login (press Alt+F1 to go to login screen). The username by default will be "root" and the password is blank, just press enter for the password:

Once logged in, you will want to take a look at the esxi_install.log for more details on how your ks.cfg is being processed and if there are any errors or warnings discovered by the parser:

Tip #3

If you want to enable both local and remote (SSH access) Tech Support Mode on your ESXi host, you now have the ability to do this via host services. You can use the vim-cmd (vimsh) utility to enable these services and both local and remote TSM is disabled by default.

Note: If you want to enable either local and/or remote TSM, you need to make sure you enable and start the service for you to actually be able to SSH into your ESXi host.

Tip #4

With classic ESX, if you needed to transfer additional packages or files to your host, you could easily mount an NFS volume, with ESXi, an NFS client is not available. If need to transfer files for configuration purposes, you can utilize the wget utility.

The syntax for wget is the following:

wget http://webserver/file -O /tmp/file

Tip #5

I have been told by support that you could not configure syslog for your ESXi host without relying on external tools such as vCLI, PowerCLI or vSphere Client. I have found that you actually can configure syslog configurations, though you have to dig a little bit into vim-cmd (vimsh) as it is not available using any of the local esxcfg-* commands. There only three syslog options as provided via vSphere Client in the Advanced Host Configurations: Syslog.Remote.Hostname, Syslog.Remote.Port and Syslog.Local.DatastorePath

Here is the syntax for the syslog options:

vim-cmd hostsvc/advopt/update Syslog.Remote.Hostname string syslog.primp-industries.com
vim-cmd hostsvc/advopt/update Syslog.Remote.Port int 514
vim-cmd hostsvc/advopt/update Syslog.Local.DatastorePath string "[datastoreName] /logfiles/hostName.log"

Note: Currently you can only configure one syslog server for your ESXi host to forward logs to.

Tip #6

Another new new kickstart parameter introduced with ESXi 4.1 is --level that is used in conjunction with %firstboot stanza. This parameter specifies the specific order in which the kickstart firstboot configurations should run with respect to the other startup scripts when your ESXi host first boots. By default, if you leave this out, VMware will automatically create a script called firstboot_001 and number it 999 which will be the very last script to execute. It is a good idea to move any post configurations to the very end, since most of post configuration may rely on specific VMware CLIs and services which must be started up before executing. You of course can change level, but be careful about moving it too early in the boot process.

Here is an example of changing the level to 998:

Once the host has booted up, you can login to see the script that was created from your %firstboot stanza under /etc/vmware/init/init.d

Note: As you can see, the firstboot script has now changed to 998. You will also notice two other scripts set at level 999 that handles updating the password if you decide to set a root password from the default blank, which you should. These custom scripts are generated after the initial build and upon the next reboot, these will be automatically removed.

Tip #7

You may have noticed in Tip #6, we changed the --level to 998, by default all three of these init scripts are set to boot order 999. This was actually done on purpose, the reason being as described earlier, the root password is blank by default. One issue that I found while testing is the inability to enable "Management Traffic" for a VMkernel interface. You can easily enable vMotion and FT Traffic for a VMkernel interface using vim-cmd (vimsh), but you can not for Management Traffic. One way I solved this problem is creating a python script which connects to the local ESXi MOB and enables Management Traffic on a particular VMkernel interface. I have shared this specific script on the on the VMTN communities which can be found here. The script is actually based on modified version that was initially created by Justin Guidroz who blogged about it here.

Here is the snippet that would be included in the %firstboot in which does not require you to expose the root password as it is empty by default:

ESXi 4.1

import sys,re,os,urllib,urllib2
 
# connection info to MOB
url = "https://localhost/mob/?moid=ha-vnic-mgr&method=selectVnic"
username = "root"
password = ""
 
#auth
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None,url,username,password)
authhandler = urllib2.HTTPBasicAuthHandler(passman)
opener = urllib2.build_opener(authhandler)
urllib2.install_opener(opener)
 
#execute method
params = {'nicType':'management','device':'vmk0'}
e_params = urllib.urlencode(params)
req = urllib2.Request(url, e_params)
page = urllib2.urlopen(req).read()
__ENABLE_MGMT_INT__
 
python /tmp/enableVmkInterface.py

ESXi 4.1 Update 1 ( Requires CSRF code update)

cat > /tmp/enableVmkInterface.py << __ENABLE_MGMT_INT__
import sys,re,os,urllib,urllib2
 
# connection info to MOB
url = "https://localhost/mob/?moid=ha-vnic-mgr&method=selectVnic"
username = "root"
password = ""
 
# Create global variables
global passman,authhandler,opener,req,page,page_content,nonce,headers,cookie,params,e_params
 
#auth
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None,url,username,password)
authhandler = urllib2.HTTPBasicAuthHandler(passman)
opener = urllib2.build_opener(authhandler)
urllib2.install_opener(opener)
 
# Code to capture required page data and cookie required for post back to meet CSRF requirements  ###
req = urllib2.Request(url)
page = urllib2.urlopen(req)
page_content= page.read()
 
# regex to get the vmware-session-nonce value from the hidden form entry
reg = re.compile('name="vmware-session-nonce" type="hidden" value="?([^\s^"]+)"')
nonce = reg.search(page_content).group(1)
 
# get the page headers to capture the cookie
headers = page.info()
cookie = headers.get("Set-Cookie")
 
#execute method
params = {'vmware-session-nonce':nonce,'nicType':'management','device':'vmk0'}
e_params = urllib.urlencode(params)
req = urllib2.Request(url, e_params, headers={"Cookie":cookie})
page = urllib2.urlopen(req).read()
__ENABLE_MGMT_INT__
 
python /tmp/enableVmkInterface.py

As you can see, we first create the python script and then we execute it. This allows us to call other utilities within the Busybox console without having to specify the interpreter to be python, we can just use busybox as the interpreter.

Tip #7a Here is an alternative solution to enable management traffic type on ESXi - Another way to enable management traffic on ESXi

Tip #8

If you tried to configure NTP by echoing your NTP servers into /etc/ntpd.conf and restarting ntpd, you will notice that the changes do not take effect. The only way I have been able to get this to work is by issuing another reboot which is specified at the very end of the %firstboot which will then be picked up upon boot up by the host.

Tip #9

If you would like customize the DCUI Welcome Screen, take a look at my blog post How to add a splash of color to ESXi DCUI Welcome Screen.

Tip #10

If you want to update the default datastore name from "datastore1" to something more useful such as [hostname]-local-storage-1, you can use vim-cmd (vimsh) to do so. Here is the syntax for the command if you want to use the short hostname and append "-local-storage-1" (this should be done in the %firstboot section of your ks.cfg): vim-cmd hostsvc/datastore/rename datastore1 "$(hostname -s)-local-storage-1"

Tip #11

SNMP is another one of those configurations that can not be configured and started up via normal services as you would have done in classic ESX. You can make the appropriate edits to the configuration file and you will need to reboot the host for the changes to take affect just like NTP configurations. You will need to edit /etc/vmware/snmpd.xml and add that to your firstboot section. Here is an example of snmpd.xml file: 

<config>
  <snmpsettings> 
    <communities>public1;private1</communities> 
    <enable>true</enable> 
    <port>163</port> 
    <targets>192.168.1.5 public1;192.168.1.6@163 private1</targets
  </snmpsettings>
</config>

Tip #12

A VMTN user recently posted an issue when applying patches during firstboot, that the init scripts were not being removed and the scripts continue to execute upon every reboot. The problem was that the boot.cfg were not being properly updated under /vmfs/volumes/Hypervisor{1,2}. I did some testing and found that if you created a second customization script and perform the patching as the very last step and issued a reboot, that you would not run into this problem.  Here is a small snippet of what your ks.cfg would like look with 2 custom init scripts, one configured at 998 and the other configured at 9999:

%firstboot --unsupported --interpreter=busybox --level=998
# do your customization
# in this section
 
%firstboot --unsupported --interpreter=busybox --level=9999
# do your patching
# in this section
 
#issue one final reboot
reboot
Tip #13

Here is a post on Automating Active Directory Domain Join in ESXi Kickstart

Tip #14

Here is a post on Automating Active Directory User Management in ESXi Kickstart As you can see, there are quite a few hacks I had to go through to get an equivalent kickstart build for ESXi 4.1 compared to classic ESX. I am sure there are other gotchas, but these are the ones I had encountered. Overall, ESXi 4.1 has greatly improved in terms of automating an unattended installation and configuration from ESXi 4.0, but there is definitely more work that needs to be done by VMware before users can easily transition from classic ESX to ESXi.

Tip #15
Here is a post on How to automatically add ESX(i) host to vCenter in Kickstart

In additional to VMware's documentation which is still limiting, here are additional tools and links that will be useful when creating your ks.cfg for ESXi 4.1: