New Instant Clone Architecture in vSphere 6.7 – Part 2

In the previous article, I provided an overview of the new "Parentless" Instant Clone feature which was introduced in vSphere 6.7 and some of the architectural differences between prior versions of Instant Clone. In this post, I will show you how to use the new Instant Clone feature, which is currently only available with the vSphere API.

There are two important parts when using Instant Clone:

1. A user defined script which runs within the GuestOS and is responsible for customizing the network identity of the Instant Clone. This script is not just limited to network configurations but can also be used to customize other OS and/or application settings.
2. The deployment script which runs outside of the GuestOS and instantiates new Instant Clones using the vSphere API. This script is responsible for passing in data (network configuration, OS and/or application settings) to the GuestOS which can then be accessed directly by the user defined script running within each Instant Clone for actual customization.

To demonstrate the use of the Instant Clone API, I will be using PowerCLI and have also created an Instant Clone PowerCLI module called InstantClone.psm1. However, the API is not limited to just PowerCLI and can be consumed using any of the new vSphere 6.7 SDKs. For our example, we will be exercising the creating Instant Clones from a "Frozen" Source VM workflow since that is the most efficient method, especially as you scale to larger number of Instant Clones. For those that wish to experiment with the other workflow of creating Instant Clones from a "Running" Source VM, feel free to take a look at the previous post for the required steps.

For my setup, I will be using an Ubuntu Server 16.04 system as my Source VM and I will create 30 Instant Clones from this system. Each Instant Clone will immediately be available for use and will be fully customized and reachable from a network perspective (e.g. unique IP and MAC Address). Here is a screenshot of one of my deployments to give you an idea of how blazing fast Instant Clone is 🙂

Here is a short video that I just recorded that demos the Instant Clone APIs using PowerCLI (this demo is using an Ubuntu VM configured with 2GB memory where as the original screenshot was 1GB)

Demo of Instant Clone in vSphere 6.7 from lamw on Vimeo.

Guest OS Customization Script

Here is an example OS customization script that I have created called customize.sh which can be used with Ubuntu 16.x system. You will need to download and upload the shell script to the GuestOS. The script is broken down into three parts:

1. Pre-Freeze - This is simliar to the "prepare" step in prior Instant Clone versions where you would prep the GuestOS and remove any uniquely identifying information and/or flushing data from the IP stack. In my example, I just need to flush and disable the primary network interface. This will be required as we will reconfigure the network identity. You can add other commands, but these would run up to freezing the SourceVM
2. Freeze - This is initiated by running: vmware-rpctool "instantclone.freeze" which requires that you have VMware Tools running within the GuestOS. Once this operation completes, the VM will be in a frozen state and will no longer be executing commands beyond this point. Execution will resumed within the Instant Clone exactly after this command, which in the script will be L27  
3. Post-Freeze - This is simliar to the "post" step in prior Instant Clone versions where execution will continue from where it had left off in the script prior to initiating the freeze operation. At this point, through the use of VMware Tools and the passing of data from the deployment script, you should now have a set of guestinfo* properties that can be accessed by the script and this portion is now responsible for consuming that data and then applying the network customization along with any other configurations. L30-34 demonstrates how to retrieve the custom key/value pairs that have been defined outside of the GuestOS and storing that into local variables. L37-43 will update the new MAC Address of the Instant Clone. The rest of the code should be pretty self-explanatory which handles the re-IP of the Guest along with configuring the hostname and restarting network services.

To facilitate ease of debugging OS customizations, I highly recommend logging all commands within the script and example of this is done in L6-7. Once an Instant Clone has been provisioned, you will find /root/ic-customization.log file that contains the execution of all commands, including the guestinfo data that was passed from the deployment script.

Instant Clone Deployment Script

Below is an example deployment script written in PowerCLI that uses my Instant Clone module and instantiates new Instant Clone and passes in guestinfo* properties that will then be consumed by the customization script running within each Instant Clone.

Import-Module InstantClone.psm1

$SourceVM = "Ubuntu-SourceVM"

$numOfVMs = 1
$ipNetwork = "192.168.30"
$ipStartingCount=50
$netmask = "255.255.255.0"
$dns = "192.168.30.1"
$gw = "192.168.30.1"

$StartTime = Get-Date
foreach ($i in 1..$numOfVMs) {
    $newVMName = "IC-$i"

    $guestCustomizationValues = @{
        "guestinfo.ic.hostname" = "$newVMName"
        "guestinfo.ic.ipaddress" = "$ipNetwork.$ipStartingCount"
        "guestinfo.ic.netmask" = "$netmask"
        "guestinfo.ic.gateway" = "$gw"
        "guestinfo.ic.dns" = "$dns"
        "guestinfo.ic.sourcevm" = "$SourceVM"
    }
    $ipStartingCount++
    New-InstantClone -SourceVM $SourceVM -DestinationVM $newVMName -CustomizationFields $guestCustomizationValues
}

$EndTime = Get-Date
$duration = [math]::Round((New-TimeSpan -Start $StartTime -End $EndTime).TotalMinutes,2)

Write-Host -ForegroundColor Cyan  "`nTotal Instant Clones: $numOfVMs"
Write-Host -ForegroundColor Cyan  "StartTime: $StartTime"
Write-Host -ForegroundColor Cyan  "  EndTime: $EndTime"
Write-Host -ForegroundColor Green " Duration: $duration minutes"

L1 - Imports Instant Clone module

L3 - This is the name of the Source VM that you will Instant Clone from

L5 - This defines the number of Instant Clones you wish to deploy, for testing purposes, leave this as 1 so you can verify it works before deploying more VMs

L6-10 - This defines the network configuration that will be used to customize your Instant Clones. This should be fairly straight forward and it simply increments from $ipStartingCount variable

L12-26 - This is where loop through the number of Instant Clones to create and define a hashtable of the guestinfo* properties which are simply key/value pair strings that will then be sent down into the GuestOS using VMware Tools. The keys need to be prefixed with guestinfo.X (so it can be accessed within the GuestOS) where X can be any custom string. Make sure your OS customization script is aware of the keys you have defined, so that they can be processed correctly. In my example, I have defined the following properties:

• guestinfo.ic.hostname - Hostname of the Instant Clone
• guestinfo.ic.ipaddress - IP Address for the Instant Clone
• guestinfo.ic.netmask - Network netmask for the Instant Clone
• guestinfo.ic.gatway - Network gateway for the Instant Clone
• guestinfo.ic.dns - DNS for the Instant Clone
• guestinfo.ic.sourcevm - Name of the Source VM, this can be used both internally and externally of the Instant Clone to trace the source VM that was used

L28-34 - Calculates the deployment duration and outputs that to the console

Consuming Instant Clone Scripts

Below are the detailed instruction for consuming both the OS customization and deployment script.

Step 1 - Install an Ubuntu Server 16.04 VM and upload the customize.sh script. To make debugging easier, I found it useful to initially create a poweredOn snapshot so you can easily revert back to the pristine state of your Source VM. Once you understand the workflow, then this snapshot is not required and you can start from a powered on state.

Step 2 - Launch a VM Console to your Source VM and then login. We are now going to run the customize.sh script which after execution will look like it is not doing anything, this is expected. Notice the cursor will stop blinking, but the VM will still be running. This means that our script has executed and has reached the frozen command.

Note: This steps can also be automated using either Guest Operations API (Invoke-VMScript for PowerCLI users) or using the VM keystrokes API. One thing to be aware of if you plan to use Invoke-VMScript is that it expects the command that runs in the guest to complete, however when the script performs the freeze operation, the process will actually freeze and not complete. One potential workaround is to background the process when you run it, which would allow the cmdlet to complete, this is not something I have tried yet.

Step 3 - Update the PowerCLI deployment script (linked above) and fill out the specific details to your environment and go ahead and run the script. If everything was configured correctly, you should start seeing new Instant Clones deployed in your environment.

Lets now take a closer look at one of our deployed Instant Clones to get an idea of what has just happened as its so fast 🙂

If you open the VM console to the Instant Clone, you should see some of the output from the script. The first section are all the commands that executed prior to the freeze operation and this happened in the Source VM up to the freeze operation. Everything after that output is actual execution within the Instant Clone which has resumed from where the Source VM had left off.

If you recall earlier, we also talked about logging all customization which can be useful for debugging and troubleshooting purposes. Below is a screenshot of what the logs look like and I have highlighted in yellow the key/values that have been passed by our deployment script directly into the GuestOS, which you can see here.

One other thing I wanted to mention is that because we are using ExtraConfig property of a VM, these key/values can be queried both inside the GuestOS via VMware Tools but it can also be extracted outside of the VM simply using the vSphere API.

Here is a PowerCLI snippet on how to query all these properties:

(Get-VM -Name IC-1).ExtensionData.Config.ExtraConfig | where {$_.key -match "guestinfo."}

Note: If you happen to have sensitive values in these guestinfo.* properties or you wish to NULL these values out, you can actually reconfigure these to be empty strings which can be done using either the vSphere API or you can actually do so within the GuestOS.

Community Repository for Instant Clone OS Customization Scripts

In case you did not pick this up while going through this blog post, the most complicated part of using Instant Clone is not actually its APIs, but rather the OS specific customization script to handle things like MAC Address updates and network identity configuration. This means, that Instant Clone is not actually constrained by a particular OS, but is quite flexible. As long as you can run VMware Tools and develop a script to handle the network identity, Instant Clone can be used. In fact, you can probably Instant Clone a Windows XP or MacOS system 😉

Given the OS customization script will be the most interesting part as you develop and consume Instant Clone, I thought it would be useful to create a community repository of working scripts that can be used with Instant Clone. I have included my Ubuntu customization script, but my hope is that the community will also contribute code that has worked for them to also help others. You can find the Github repo at: https://github.com/lamw/instantclone-community-customization-scripts

Converting Instant Clone to Full Clone

Simliar to a Linked Clone, customers can also take an Instant Clone and then convert that to a Full Clone using the PromoteDisks_Task() vSphere API and ensure you set the unlink property to True. Unlike a Linked Clone, which can be converted while the VM is powered on, for a Instant Clone, you must first power it off before calling the API. I know this is something Engineering is considering for future to support online full clone conversion.