WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

How to VMFork aka Instant Clone Nested ESXi?

08.03.2015 by William Lam // 15 Comments

vmfork-aka-instant-clone
The VMware Fling's team recently released an update to the existing PowerCLI Extensions which now exposes the new VMFork aka Instant Clone capability that was introduced in vSphere 6.0. The Fling contains a set of PowerCLI Extension Modules which in turn provides new PowerCLI cmdlets for accessing the Instant Clone feature. The idea behind the Fling is to help VMware understand how customers would like to consume the Instant Clone feature not only from a CLI point of view but also from an API and UI standpoint. Prior to this, Instant Clone was only available through the use of either Horizon View or the Big Data Extensions product. I think this is a great opportunity for customers and partners to help shape how Instant Clone should be consumed more generally.

One of the use cases I had in my mind when I had first heard about the Instant Clone feature was to be able to quickly instantiate new Nested ESXi VMs. When I got the opportunity to help test out early prototypes of the Instant Clone cmdlet to help provide feedback and usability improvements, I knew I had to give Nested ESXi a try!

Requirements:

  • Fresh installation of Nested ESXi 6.0 in VM (unconfigured)
  • PowerCLI 6.0 Release 1
  • Instant Clone PowerCLI Extensions Fling
  • Nested ESXi 6.0 Instant Clone Scripts

High level process:

  1. A "preparation" script will be manually uploaded & executed within the Nested ESXi VM (Parent VM) to prep the system for Instant Cloning
  2. As the Parent VM is quiesce, both the pre/post customization script will be uploaded to the Parent VM automatically. The "pre-customization" is also then executed within the Parent VM which properly setups the library path to the VMware Tools binary (applicable to ESXi 6.0 only) and is then placed in a ready state for creating Instant Clones
  3. As new Instant Clone (Child VMs) are spun up, the "post-customization" script is automatically executed to add additional configurations and most importantly ensure newly created Instant Cloned Nested ESXi VMs have unique network identities

Note: For Instant Cloning regular OSes, only step 2 and 3 are really needed. Due to a known issue with VMware Tools for Nested ESXi, I have found that it is easier to prepare the Nested ESXi VM prior to quiescing and creating Instant Clones from the Parent VM.

Instructions:

Step 1 - Download and install both PowerCLI 6.0 Release 1 & Instant Clone PowerCLI Extensions Fling.

Step 2 - Perform a fresh Nested ESXi 6.0 installation in a VM, do not configure additional settings outside of enabling ESXi Shell and SSH.

Step 3 - Download the Nested ESXi 6.0 Instant Clone Scripts which contains the following four files:

  • prep-esxi60.sh - Prepares the Nested ESXi VM and ensures that new Child VMs will not retain the Parent VM's MAC Address which is baked in several places
  • pre-esxi60.sh - Pre-customization script which is used to properly setup the library paths to use the VMware Tools daemon to retrieve guest properties from PowerCLI script
  • post-esxi60.sh - Post-customization script which is used to apply networking configuration and hostnames for example
  • vmfork-esxi60.ps1 - An example PowerCLI script which issues the Instant Clone cmdlets

Note: For out of the box use, the only script that needs to be modified is the PowerCLI "vmfork-esxi60.ps1" script, the rest of the scripts should work or require very little to no modifications assuming you have followed the instruction thus far.

Step 4 - Upload the prep-esxi60.sh to Nested ESXi 6.0 VM (Parent VM) and then execute it using either the ESXi Shell over SSH or through a VMRC session. If you use SSH, you will notice that the script hangs, that is because the VMkernel interface is deleted as part of the script.

Step 5 - Next, we need to make a few edits to the vmfork-esxi60.ps1 script to update the name of your ESXi VM, along with its credentials and the full path to both the pre and post customization scripts. Below is an example of the variables that you will need to edit:

$parentvm = 'vESXi6'
$parentvm_username = 'root'
$parentvm_password = 'vmware123'
$precust_script = 'C:\Users\lamw\Desktop\vmfork\esxi60\pre-esxi60.sh'
$postcust_script = 'C:\Users\lamw\Desktop\vmfork\esxi60\post-esxi60.sh'

The section shown below will also need to be edited which contains the customization properties which are then passed down to the guestOS for configuration as part of the Instant Clone process.

 $configSettings = @{
 'hostname' = "$vmname.primp-industries";
 'ipaddress' = "192.168.1.$_"; 
 'netmask' = '255.255.255.0'; 
 'gateway' = '192.168.1.1';
 }

Step 6 - Lastly, it is time to run the script by issuing the following command:

.\vmfork-esxi60.ps1

instant-clone-nested-esxi-0
If everything was successful, you should see a couple of new powered on Instant Cloned Nested ESXi VMs that have been fully customized and ready for use!

instant-clone-nested-esxi-1
Note: There have been a couple of times where newly Instant Clone VMs have not been properly customized and when looking in the Instant Clone logs under /var/tmp/quiesce.log you may find "Unable to fork" error message. I usually have to re-quiesce the Parent VM which I do so by reverting back to a snapshot that captures the state after Step 4. Once I re-run the PowerCLI script, I am able to successfully deploy N-Number of Instant Clone Nested ESXi VMs. For additional best practices and tips/tricks, be sure to check out this blog post here.

Big thanks to Jim Mattson for some of his earlier research and work on this topic which made implementing these scripts much easier.

Categories // Automation, ESXi, Nested Virtualization, vSphere 6.0 Tags // ESXi 6.0, fling, instant clone, nested, nested virtualization, PowerCLI, vmfork

Heads up: ESXi 5.x & 6.0 unable to detect newer Apple Mac Pro 6,1 local SSD Device

07.27.2015 by William Lam // 12 Comments

Over the last couple of weeks there have been several reports coming in from customers that the local SSD device found in newer Apple Mac Pro 6,1 were no longer being detected by ESXi. Starting with ESXi 5.5 Patch03 and ESXi 6.0, the Apple Mac Pro 6,1 was officially supported but it looks like the latest versions of the Mac Pro 6,1 that are being shipped contain  a slightly different local SSD device which is not recognized by ESXi.

This was not the first time that this has happened, when the 2014 Mac Mini were first released, they too had a similar issue in which a custom VIB was required to get the internal device to get recognized by ESXi. There is an internal bug (PR 1487494) that is currently tracking the issue and if you are also experiencing this problem, please file an SR and have the GSS Engineer attach your case to this bug.

In the meantime, there is an unofficial workaround which was discovered by one of my readers (Mr. Spock) that by installing the community SATA-XACHI VIB over on Andreas Peetz VIB Depot site, both ESXi 5.5 and ESXi 6.0 will then recognize the local SSD Device. You will need to either use VMware Image Builder or Andreas ESXi Customizer tool to create a custom image if you decide to install ESXi directly on the local SSD device. I personally would recommend installing ESXi on USB device, this would allow you to install the VIB as a post-installation and not requiring a custom ESXi image.

Categories // Apple, ESXi, vSphere 5.5, vSphere 6.0 Tags // apple, ESXi 5.5, ESXi 6.0, mac, mac pro

How to create a custom ESXi VIB to execute a script once?

07.16.2015 by William Lam // 16 Comments

Earlier this week I received a question from a customer who was interested in creating a custom ESXi VIB that could execute a specific script within the payload as part of the installation and only running it once. This was a fairly common request that I have seen in the past and as far as I knew, this type of behavior was not possible. What was unique about this particular custom inquiry was that they mentioned they found some references on this being possible. Being the curious person I am, I decided to take another look and reach out to a few folks in Engineering.

After speaking with one of the Engineers familiar with our VIB specification, to my surprise I learned that this type of behavior was actually indeed possible but was not very well documented externally. The typical use case for this is usually to apply certain configurations which are not exposed through the traditional ESXi interfaces like the vSphere API/CLI which includes Host Profiles. One example that comes to mind is being able to deploy a VIB across several hundred ESXi hosts that could configure a specific ESXi Advanced Setting which might be marked hidden. Another example would be updating a configuration file like /etc/vmware/config or running a series of ESXi Shell commands which can not available in the vSphere API and require the use of SSH and the ESXi Shell. There are many other examples, but this should give you an idea of some of the common use cases that I have heard from customers.

Here is what is required to execute a specific script as part of the VIB installation:

  • Created an "init" script which needs to be stored under /etc/init.d as part of your VIB payload (e.g. /etc/init.d/lamw-special-vib)
  • You must ensure that the "live-install-allowed" param is set to true in your VIB's descriptor.xml file (this assumes the changes can be applied without requiring reboot to take affect)
  • The init script will need to parse special keywords passed as command-line arguments

Here is a table showing the pair of special keywords that are passed to the init script as command-line arguments during a VIB install, upgrade or remove:

Arg1 Arg2 VIB Operation
start install VIB install
start upgrade VIB upgrade
stop remove VIB removal

It is up to creator of the init script to handle the different VIB operations by parsing the command-line arguments which would then determine the operations that would get executed within the script. This not only allows you to control the commands that are executed during an installation of a VIB but it also allows you specify the commands to run upon the removal of a VIB which is quite handy for properly cleaning up an uninstall. In addition, since these special keywords are not passed as part of the ESXi boot up process, the commands within the script will not execute and ensures it only runs once during the install.

Here is very simple shell script (you could also do this in Python as well) which demonstrates how to handle the three different types of VIB operations and then uses the "logger" utility to write some output to syslog:

#!/bin/sh

ARG1=$1
ARG2=$2

if [[ "${ARG1}" == "start" ]] && [[ "${ARG2}" == "install" ]]; then
 # commands to run go here #
 /bin/logger "William's custom script ran on start-install"
elif [[ "${ARG1}" == "start" ]] && [[ "${ARG2}" == "upgrade" ]]; then
 /bin/logger "William's custom script ran on start-upgrade"
 # commands to run go here #
elif [[ "${ARG1}" == "stop" ]] && [[ "${ARG2}" == "remove" ]]; then
 /bin/logger "William's custom script ran on stop-remove"
 # commands to run go here #
fi

As you can see, this allows you to perform a variety of tasks through the use of a custom ESXi VIB which is a great way to be able to roll out a set of changes that may not be possible using either the vSphere API or Host Profiles for example. Best of all, this solution does not require the use of SSH which is great since many customers already disable this by default.

For more information on creating a custom VIB, be sure to check out my blog post here and you can even use this Docker image I created for building custom ESXi VIBs.

Categories // Automation, ESXi Tags // ESXi, host profile, vib, vib author

  • « Previous Page
  • 1
  • …
  • 344
  • 345
  • 346
  • 347
  • 348
  • …
  • 560
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...