I was fortunate to have been given early access to the VMFork (Instant Clone) PowerCLI cmdlets to help provide early feedback and usability improvements before it was released to customers. Having spent some time with the Fling, I have learned a thing or two about how Instant Cloning works and some of the caveats or gotchas while creating the customization scripts that are used as part of the Instant Clone workflows. I wanted to put together a quick reference on some of my findings as well as well as other recommendations from Engineering who have worked closely with the Instant Clone feature.

The idea is to have this as a living document which I will update as new tips and tricks are identified.

Best Practices

• Ensure VMware Tools is installed inside the guestOS and also good time to ensure you are running the latest
• Both Pre/Post Customization scripts are uploaded to /var/tmp by the Enable-InstantCloneVM cmdlet
• Do not delete Child VMs directly on ESXi, manage it through vCenter Server. There is currently a known issue in which deleting Child VMs will also delete the Parent VM's disk
• Additional custom variables can be passed to the post-customization script by adding to the -ConfigParams array of variables.
  • An example could be passing in two custom properties called "foo" and "bar" which would look like:

  @{foo = "val1";bar ="val2"}

  • To retrieve the variable "foo" and "bar" from within the post-customization script, you would issue the following commands:

  vmtoolsd --cmd "info-get guestinfo.fork.foo"
  vmtoolsd --cmd "info-get guestinfo.fork.bar"

• A Forked Child VM will also have a duplicate MAC Address which needs to be updated as it is not automatically picked up.
  • You can either manually set it by retrieving the guestinfo.fork.ethernet0.address with the post-customization script.
  • An easier way would be to reload it based on the guestOS type. On a Linux system, you can use the modprobe command like the following (Submited by George Hicken):

  modprobe -r vmxnet3;modprobe vmxnet3

• A Forked Child VM may also have identical kernel entropy pools which means semi-predictable RNG, possibly including TCP sequence numbers (Submited by George Hicken)
• A Forked Child VM's system clock may also be out of date (until you call hwclock --hctosys or similar) which can cause problems with ordering of file timestamps (Submited by George Hicken)
• Shared host keys if you are using a PKI system or identical asset identifiers in the case of Windows and any sort of AD infrastructure would also need to be either removed prior or updated after a Child VM is created (Submited by George Hicken)
• Instant Cloning Nested ESXi has been a bit tricky due to a known issue with the VMware Tools for Nested ESXi. I have found that manually preparing the guest prior to Instant Cloning has yield better results. For more information on how to Instant Clone Nested ESXi, check out the blog post here
• Powering off the Parent VM means that the VM is no longer quiesced and this also means that new Child VMs can not be instantiated until all existing Forked Child VMs have been powered off and the Parent VM has been re-quiesced
• If you plan on downloading or installing additional software packages on the Parent VM, it is recommended that you perform that operation directly in the VM and not within the pre-customization script. I have noticed that if pre-customization takes too long, the quiesce operation eventually fails even though the operations within the pre-customization script executed successfully.
• To ensure Forked Child VMs do not contain duplicate disk ID's from Parent VM such as setting up a VSAN environment using Instant Clone Nested ESXi, add the disks after Forked Child VMs have been created.
• For additional OS Customization Scripts, be sure to check out the Instant Clone community customization script repository and consider contributing back scripts that you have developed.
• When you hard reset or power off on a child VM it will respawn from the parent, soft resets will not respawn (Submitted by Alan Renouf)

Troubleshooting

• Instant Clone guestOS logs are stored in /var/tmp/quiesce.log
• Consider enabling tracing within your customization scripts. An example of this for a shell script is using

  set -x

• Add additional echo or print statements like Start/Stop of certain sections like Pre/Post which can aide in reviewing the Instant Clone logs as seen in the screenshot above
• For Instant Cloning Nested ESXi guestOSes, I recommend taking a snapshot after you have prepared the guest and removed any system specific information. This allows you to quickly revert back to a known state for ease of debugging. I found this to be very useful to be able to start back a known clean state while developing the customization scripts for Instant Cloning Nested ESXi
• A known issue that is mentioned in the documentation of the Instant Clone cmdlets is after enabling a ParentVM for Instant Cloning, is that it is no longer available for migration to another ESXi host. The reason for this is that after powering off the VM, the "parentEnabled" boolean flag is still set to "true" which prevents the migration. Currently, there is not a work around but hopefully this will be resolved in a future update of the cmdlets. You can see this by running the following PowerCLI snippet:

  (Get-VM "MyParentVM").ExtensionData.Config.ForkConfigInfo