WilliamLam.com

Cloning vMA still not supported?

by // 5 Comments

Cloning and/or moving vMA is not supported by VMware, this has been the case since VIMA 1.0. If you take look at the "Known Issues" in the vMA release notes, you will see there is no still workaround. Users may find it useful to be able to clone vMA for backup purposes or to deploy another vMA without having to re-download or re-uploading .OVF from your local desktop/VMware's website.

If you ever tried to clone VIMA 1.0 and boot the system up, you will notice a warning message during the init process when vMA verifies the system's UUID. If it detects that it has been cloned, it will actually disable itself and makes the vMA components unusable until you reinitialize vMA using vimaclean. This is still the case with vMA 4.1, except the vMA components are no longer disabled when you perform a clone.

You will just see the warning message during the init process, but all functionality continues to work. Before the vMA specific components starts up, it makes a call to /opt/vmware/vma/bin/verifyuuid. It verifies the system UUID which you can extract using dmidecode, this is the same UUID found within the .vmx entry called uuid.bios. When you clone a VM, a new UUID will be generated which causes an issue when it tries to verify.

Prior to vMA 4.x, VIMA 1.0 stored all it's configurations within a set of XML files located in /etc/vmware/viconfig, these files held the config for both the vi-fastpass targets and vilogger config and policies. With the release of vMA 4.0, these configurations were migrated to a sqlite3 database which is stored in /var/opt/vmware/vMA/vMA.db. Within this database, it also stored as the old flat file configuration, the system UUID of vMA. This is what verifyuuid was checking against to see if it was cloned or moved, you can actually query the vMA DB to extract this information using a simple SQL query.

To resolve the warning message, you just need to update the new UUID in the vMA DB using an update statement.

Instead of manually typing all that out, I actually created a very simple script that queries for the current system UUID and then updates the vMA DB with the correct UUID. Here is a sample execution:

Download: updatevMAUUID.sh

I suspect that VMware may have allowed cloning of vMA, but just did not update their release notes? The only thing you have to do is manually fix the UUID if you do not want to see the warning message. If you would like to fix this for VIMA 1.0, you will need to do the same thing except you need to modify the UUID found in /etc/vmware/viconfig/viconfig.xml and restart vMA for the changes to effect.

How to automate & cron vi-fastpass scripts on vMA 4.1

by // 3 Comments

vMA's vi-fastpass and vifptarget utility is meant to be used interactively on the command line, this requires the user to set a particular target (ESX,ESXi or vCenter host) and then execute either a vCLI and/or vSphere SDK for Perl script. If you want to stay in the context of a particular server (e.g without having to manually specify --server parameter), you would need to manually change the target. There have been two common set of questions that have re-appeared several times on the VMTN forums:

1) How to automate a particular script and leverage vi-fastpass without having to change the context for each ESX or ESXi host

2) How to automate a particular script and leverage vi-fastpass via cron without having to provide username/password 

    Before we dive into the solutions, the key in using vi-fastpass in a non-interactive fashion is how vifptarget is called from a script. When you call the vifptarget, it is actually an alias to source the following shell script located in /opt/vmware/vma/bin/vifptarget. You can see this by using the "which" utility on a particular command.

    This alias is defined in /etc/bashrc

    You can even redefine the alias name to whatever you like, so long as it's aliasing the actual vifptarget script. Properly sourcing vifptarget script is the key in solving Question 1. If you look at /etc/bashrc, you will also notice that LD_LIBRARY_PATH environmental variable for shared libraries is also defined, this will be the key in solving Question 2 along with properly sourcing vifptarget. You will see these referenced in the example scripts. 

    There are two solutions for Question 1:

    Solution A:

    If you are leveraging vCenter Server and the ESX(i) hosts are being managed by vCenter, then you just need to add vCenter to vi-fastpass and run a simple for loop across the hosts.

    Example:
    2 x ESXi 4.1 hosts being managed by vCenter

    • esx4-4.primp-industries.com
    • esx4-5.primp-industries.com

    Execute esxcfg-ntp to configure 2 x NTP servers for both ESX(i) host

    • ntp1.ucsb.edu
    • ntp2.ucsb.edu

    Here is a shell script that performs the configuration on the two ESX(i) host being managed by vCenter and reads in a file containing the name of the ESX(i) hosts:

    example1.sh

    Download: example1.sh

    Solution B:

    In this solution, vCenter is not available or you each ESX(i) host being managed by vMA's vi-fastpass. The script will be very similar, but instead of setting vCenter as the target, you will be setting an ESX(i) host. What is interesting that may not be well known is if you are managing more than one ESX(i) vi-fastpass target, you only need to set the target once and you can perform the same operation on all other host. The caveat being, if you want to switch out of the default context of your target, you need to specify --server parameter. This script will actually do it for you behind to scenes.

    Example:
    2 x ESXi 4.1 hosts being managed by vMA

    • esx4-4.primp-industries.com
    • esx4-5.primp-industries.com

    Execute esxcfg-ntp to configure 2 x NTP servers for both ESX(i) host

    • ntp1.ucsb.edu
    • ntp2.ucsb.edu

    Here is a shell script that performs the configuration on the two ESX(i) host being managed by vMA and reads in a file containing the name of the ESX(i) hosts:

     example2.sh

    Download: example2.sh

    After executing either SolutionA or SolutionB, you now should have the configurations applied to your ESX(i) host as you would if you manually ran it on each host. This is just an example, you can easily substitute multiple esxcfg-* and/or vSphere SDK for Perl scripts. Here is also another example script for a user on the VMTN forums on configuring a vSwitch and adding portgroups.

    Solution for Question 2:

    Now that we know how to automate a particular operation using vi-fastpass in a script, the next logical question is how do we automate it using a cronjob. I will not go into details of how cron works, take a look at the link for more details. One thing I will mention is that cron does not have all the same PATH environmental variables defined as you normally would expect. You should always use full path to scripts and binaries and any shared library modules that is required to execute a particular utility. What this means for leveraging vi-fastpass via a cronjob is that you need to specify the LD_LIBRARY_PATH. In solving Question 1, all we had to do was properly source the vifptarget and the reason we did not have to specify the shared library path is that it was already defined as a default for vi-admin user. This is not the case for cron and you will need to specify that in either the script you are executing or within the crontab.

    Example:

    2 x ESXi 4.1

    • esx4-4.primp-industries.com
    • esx4-5.primp-industries.com

    Execute vmware-cmd -l to extract virtual machines on both ESX(i) host and store the output that to a file called /tmp/vms every 5 minutes. Here is what the cron entry should look like for example3.sh

    example3.sh

    Download: example3.sh

    After 5 minutes, you should now see a new file under /tmp/vms that contains all virtual machines registered on your ESX(i) host. Again, you can easily apply this to any other vCLI and/or vSphere SDK for Perl script.

    Here another example that gets asked quite often in scheduling the popular VMware vSphere Health Check Script, here is the crontab entry for automatically running the healthcheck script every hour against your vCenter server.

    example4.sh

    Download: example4.sh

    Again, the key is to make sure you specify the full path to not only the script you would like to run but also if there is any type of output, that you specify the full path to the output.

    How to increase capacity of /var/log on vMA 4.1

    by // 4 Comments

    This question pops up from time to time on the VMTN community forums on how to increase the size of /var/log on vMA. By default, the /var/log partition is configured to ~500MB, which is a decent size for system logs, but one of the features of vMA is vilogger which adds a syslog capability for your ESX and ESXi hosts. This allows you to ship all your host logs over to vMA, the problem is the logs are also stored in /var/log along with vMA's system logs. If you are managing several dozen hosts, you can easily fill up this partition using the default configured size.

    With vMA 4.0, it was pretty trivial to increase/resize partitions within vMA using LVM and I had written a quick guide on how to do this - How to increase/resize vMA Disks. With the release of vMA 4.1, VMware kind of went backwards and decided not to leverage LVM and lost the benefits with using a volume manager. Resizing the partitions can still be accomplished but it requires a few additional steps and tools. I will show you two methods in which you can increase your /var/log partition, primarily to accommodate larger number of hosts to utilize the vilogger functionality.

    Option1: Resize /var/log using gparted 

    Here is what a default vMA disk layout looks like and note the size of /var/log

    First, you will want to shutdown your vMA host, you may also want to backup any scripts/configs you may have on your vMA host in case you make a mistake and can not recover. Next, you will need to increase the existing virtual disk, by default it is configured for 5GB. In this example, I will increase the disk capacity to 10GB bringing the total to 15GB. You will also need to download a gparted live CD ISO, in this example I am using gparted 0.3.4-11, it is what I have available on my datastore.

    Next, you will want to attach the gparted ISO to vMA, you probably will need to add a CD-ROM drive since vMA does not come with one by default. Once you power vMA on, you will need to hit ESC to and select boot from CD-ROM. You should be able to just accept all the defaults and once gparted is loaded, you should see the following screen or something similar depending on the version of gparted you are using.

    As you can see, it is the partition layout of your vMA host including the unallocated space, which in this case is 10GB. At a high level, we need to resize the partitions inside the extended partition first before we can resize /var/log partition. First you will select/highlight the "extended (/dev/sd4)" partition which is colored in baby blue color and you will move the right arrow and extend it all the way to the right and then click "Resize/Move" button.

    Next, you will select/highlight "/ (dev/sd5)" partition which is colored in blue. Here we want to keep / (root) the same default size of 3.39GB, so we are just going to shift the entire box from the left to far right. You just need to bring the cursor on top of the dark blue section and drag it to the right. Make sure you do not move the left or right arrows, else you will be changing the size of root which is not what we want. Once you have it like the picture below, click on the "Resize/Move" button.

    Now you will select/highlight "extended (/dev/sd4)" partition again and you will move the left arrow and drag it towards the right like the picture below and again click on the "Resize/Move" button once you are done.

    At this point, your screen should look like the following, if it does not for what ever reason you can "undo" all the changes, since none of these changes take affect until you apply them. 

    Now, we will finally increase the size of "/var/log (/dev/sd3)" and consume the full amount of space that we have allocated.

    Once you are done, we are now ready to apply these changes, the "apply" button is at the top.

    After this, it can take up to several minutes depending on your allocation but if everything went well, you should see this success screen at the end and you just need to reboot the system to see the new changes.

    One your vMA host is up, you now should be able to run "df -h" again and you should see that /var/log has now increased with the additional space we assigned it.

    Option2: Add a new virtual disk and reconfigure vilogger syslog location

    If you did not want to go through the process in option 1 and just want to add another virtual disk and dedicate that for vilogger syslog, you can actually change the path of the logs in /etc/vmware/vMA/vMA.conf. You can control not only the vilogger syslog but also vi-fastpass logs and vilogger daemon logs, but we will only change the "vMALogCollector" which is basically the vilogger syslog component.

    I will assume you already have a disk added to vMA and you have created the appropriate mount point (If you are unsure how, do a search online). In this example, I have a 15GB partition in /vmasyslog that I will use as the new home for all my vilogger syslog files.

    After you have made the edit to vMA.conf, you will need to restart vilogger daemon and you will need to use sudo and the service command.

    Now, you can enable vilogger for your ESX and ESXi host and you should see all logs being redirected to this new partition instead of the default /var/log/vmware/*