WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

How to VMFork aka Instant Clone Nested ESXi?

08.03.2015 by William Lam // 15 Comments

vmfork-aka-instant-clone
The VMware Fling's team recently released an update to the existing PowerCLI Extensions which now exposes the new VMFork aka Instant Clone capability that was introduced in vSphere 6.0. The Fling contains a set of PowerCLI Extension Modules which in turn provides new PowerCLI cmdlets for accessing the Instant Clone feature. The idea behind the Fling is to help VMware understand how customers would like to consume the Instant Clone feature not only from a CLI point of view but also from an API and UI standpoint. Prior to this, Instant Clone was only available through the use of either Horizon View or the Big Data Extensions product. I think this is a great opportunity for customers and partners to help shape how Instant Clone should be consumed more generally.

One of the use cases I had in my mind when I had first heard about the Instant Clone feature was to be able to quickly instantiate new Nested ESXi VMs. When I got the opportunity to help test out early prototypes of the Instant Clone cmdlet to help provide feedback and usability improvements, I knew I had to give Nested ESXi a try!

Requirements:

  • Fresh installation of Nested ESXi 6.0 in VM (unconfigured)
  • PowerCLI 6.0 Release 1
  • Instant Clone PowerCLI Extensions Fling
  • Nested ESXi 6.0 Instant Clone Scripts

High level process:

  1. A "preparation" script will be manually uploaded & executed within the Nested ESXi VM (Parent VM) to prep the system for Instant Cloning
  2. As the Parent VM is quiesce, both the pre/post customization script will be uploaded to the Parent VM automatically. The "pre-customization" is also then executed within the Parent VM which properly setups the library path to the VMware Tools binary (applicable to ESXi 6.0 only) and is then placed in a ready state for creating Instant Clones
  3. As new Instant Clone (Child VMs) are spun up, the "post-customization" script is automatically executed to add additional configurations and most importantly ensure newly created Instant Cloned Nested ESXi VMs have unique network identities

Note: For Instant Cloning regular OSes, only step 2 and 3 are really needed. Due to a known issue with VMware Tools for Nested ESXi, I have found that it is easier to prepare the Nested ESXi VM prior to quiescing and creating Instant Clones from the Parent VM.

Instructions:

Step 1 - Download and install both PowerCLI 6.0 Release 1 & Instant Clone PowerCLI Extensions Fling.

Step 2 - Perform a fresh Nested ESXi 6.0 installation in a VM, do not configure additional settings outside of enabling ESXi Shell and SSH.

Step 3 - Download the Nested ESXi 6.0 Instant Clone Scripts which contains the following four files:

  • prep-esxi60.sh - Prepares the Nested ESXi VM and ensures that new Child VMs will not retain the Parent VM's MAC Address which is baked in several places
  • pre-esxi60.sh - Pre-customization script which is used to properly setup the library paths to use the VMware Tools daemon to retrieve guest properties from PowerCLI script
  • post-esxi60.sh - Post-customization script which is used to apply networking configuration and hostnames for example
  • vmfork-esxi60.ps1 - An example PowerCLI script which issues the Instant Clone cmdlets

Note: For out of the box use, the only script that needs to be modified is the PowerCLI "vmfork-esxi60.ps1" script, the rest of the scripts should work or require very little to no modifications assuming you have followed the instruction thus far.

Step 4 - Upload the prep-esxi60.sh to Nested ESXi 6.0 VM (Parent VM) and then execute it using either the ESXi Shell over SSH or through a VMRC session. If you use SSH, you will notice that the script hangs, that is because the VMkernel interface is deleted as part of the script.

Step 5 - Next, we need to make a few edits to the vmfork-esxi60.ps1 script to update the name of your ESXi VM, along with its credentials and the full path to both the pre and post customization scripts. Below is an example of the variables that you will need to edit:

$parentvm = 'vESXi6'
$parentvm_username = 'root'
$parentvm_password = 'vmware123'
$precust_script = 'C:\Users\lamw\Desktop\vmfork\esxi60\pre-esxi60.sh'
$postcust_script = 'C:\Users\lamw\Desktop\vmfork\esxi60\post-esxi60.sh'

The section shown below will also need to be edited which contains the customization properties which are then passed down to the guestOS for configuration as part of the Instant Clone process.

 $configSettings = @{
 'hostname' = "$vmname.primp-industries";
 'ipaddress' = "192.168.1.$_"; 
 'netmask' = '255.255.255.0'; 
 'gateway' = '192.168.1.1';
 }

Step 6 - Lastly, it is time to run the script by issuing the following command:

.\vmfork-esxi60.ps1

instant-clone-nested-esxi-0
If everything was successful, you should see a couple of new powered on Instant Cloned Nested ESXi VMs that have been fully customized and ready for use!

instant-clone-nested-esxi-1
Note: There have been a couple of times where newly Instant Clone VMs have not been properly customized and when looking in the Instant Clone logs under /var/tmp/quiesce.log you may find "Unable to fork" error message. I usually have to re-quiesce the Parent VM which I do so by reverting back to a snapshot that captures the state after Step 4. Once I re-run the PowerCLI script, I am able to successfully deploy N-Number of Instant Clone Nested ESXi VMs. For additional best practices and tips/tricks, be sure to check out this blog post here.

Big thanks to Jim Mattson for some of his earlier research and work on this topic which made implementing these scripts much easier.

Categories // Automation, ESXi, Nested Virtualization, vSphere 6.0 Tags // ESXi 6.0, Fling, instant clone, nested, nested virtualization, PowerCLI, vmfork

Heads up: ESXi 5.x & 6.0 unable to detect newer Apple Mac Pro 6,1 local SSD Device

07.27.2015 by William Lam // 12 Comments

Over the last couple of weeks there have been several reports coming in from customers that the local SSD device found in newer Apple Mac Pro 6,1 were no longer being detected by ESXi. Starting with ESXi 5.5 Patch03 and ESXi 6.0, the Apple Mac Pro 6,1 was officially supported but it looks like the latest versions of the Mac Pro 6,1 that are being shipped contain  a slightly different local SSD device which is not recognized by ESXi.

This was not the first time that this has happened, when the 2014 Mac Mini were first released, they too had a similar issue in which a custom VIB was required to get the internal device to get recognized by ESXi. There is an internal bug (PR 1487494) that is currently tracking the issue and if you are also experiencing this problem, please file an SR and have the GSS Engineer attach your case to this bug.

In the meantime, there is an unofficial workaround which was discovered by one of my readers (Mr. Spock) that by installing the community SATA-XACHI VIB over on Andreas Peetz VIB Depot site, both ESXi 5.5 and ESXi 6.0 will then recognize the local SSD Device. You will need to either use VMware Image Builder or Andreas ESXi Customizer tool to create a custom image if you decide to install ESXi directly on the local SSD device. I personally would recommend installing ESXi on USB device, this would allow you to install the VIB as a post-installation and not requiring a custom ESXi image.

Categories // Apple, ESXi, vSphere 5.5, vSphere 6.0 Tags // apple, ESXi 5.5, ESXi 6.0, mac, mac pro

Using PowerCLI to invoke Guest Operations API to a Nested ESXi VM

07.14.2015 by William Lam // 1 Comment

In my opinion, the Guest Operations API in vSphere is still one of the most powerful Virtual Machine capabilities that is available to vSphere Administrators and anyone else who integrates with the vSphere Platform. The Guest Operations API allows users to perform guest operations (running commands, transferring files, etc) directly within the guestOS as if you were logged in. Valid guest credentials are still required and once authenticated, the operations are then proxied through VMware Tools. Networking is not even required which makes this a handy feature for troubleshooting and can even extend into application level provisioning through a single API.

Obviously, I am a huge fan of this capability and have used it myself on more than one occasion. However, one limitation that I discovered awhile back when using the Guest Operations API with Nested ESXi VMs is that it threw some very strange memory related errors. It was only recently did I find out that there was a known issue with the VMware Tools for Nested ESXi, both the installable VIB and the pre-installed version in ESXi 6.0 on how the guest operations are executed. The good news is that for now, there is a simple workaround that can be applied when using the Guest Operations API.

You will need to add the following option, which runs the command under a specific resource group in the ESXi Shell:

'++group=host/vim/tmp'

Here is an example if I were to run the 'echo' command:

/bin/echo '++group=host/vim/tmp' "hello world"

A more interesting example would be to issue ESXCLI commands using the Guest Operations API, perhaps configuring the welcome message?

/bin/python '++group=host/vim/tmp' '/bin/esxcli.py system welcomemsg set -m "vGhetto Was Here"'

Notice, we need to pass in the resource group command to the "python" binary versus ESXCLI binary. The reason for this is that /bin/esxcli is really just a symlink to /bin/esxcli.py which is just a Python wrapper. The actual command being launched is the python interpreter and the arguments to the command is /bin/esxcli.py and the ESXCLI arguments itself.

For those who prefer to consume the Guest Operations API without having to directly use the vSphere API, you can use PowerCLI and use the Invoke-VMScript cmdlet. The problem with that is due to the way the cmdlet has been abstracted, the necessary underlying API details can not be accessed due to certain assumed defaults which can not be overridden or extended. This is a general problem that I have seen in more than one occasion where the abstraction is to make the consumption of the API simpler but in certain cases, it can also inhibit the use of the underlying API feature.

In this case, we will actually need to call into the vSphere API and using PowerCLI as an example, I have created a script called runGuestOpsInNestedESXiVM.ps1 which implements the specific Guest Operations APIs to issue commands to a Nested ESXi VM.

Here is an example of running the script and configuring the welcome message using ESXCLI:

guest_operations_api_nested_esxi

Categories // Automation, ESXi, PowerCLI, vSphere, vSphere 6.0 Tags // guest operations, nested, nested virtualization, vix, vix api, vmware tools

  • « Previous Page
  • 1
  • …
  • 25
  • 26
  • 27
  • 28
  • 29
  • …
  • 51
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • VMware Flings is now available in Free Downloads of Broadcom Support Portal (BSP) 05/19/2025
  • VMUG Connect 2025 - Minimal VMware Cloud Foundation (VCF) 5.x in a Box  05/15/2025
  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...