WilliamLam.com

How to properly clone a Nested ESXi VM?

by // 53 Comments

I often hear from users that they would like to be able to just clone from an existing Nested ESXi VM that has already been configured and just create additional Nested ESXi VM instances from that. For me personally, I do not have a use case for this since I just deploy additional ESXi instances using an automated Kickstart deployment. However, I can see why this would be useful for anyone that does not have an automated deployment or just want to quickly deploy additional Nested ESXi instances by just cloning from an existing image and then manually change the networking configuration afterwards.

UPDATE (07/01/21) - As of ESXi 7.0 Update 2, cloning an ESXi boot volume (Nested or Physical) is no longer safe and can lead to data corruption. Please refer to the following two VMware KB articles for more information on this topic https://kb.vmware.com/kb/84280 and https://kb.vmware.com/kb/84349 

First off, cloning of a Nested ESXi VM is possible and you can already do this today. You will get a brand new Virtual Machine that will have a unique MoRef ID, InstanceUUID, BIOS UUID and MAC Addresses for each of the virtual network adapters which you can see an example of this from the screenshot below.

Everything from outside of the guest OS looks great as we would expect but there is actually two issues from within ESXi that you may not be aware of.

  • The first issue is that you will get a duplicated MAC Address of the VMkernel interface(s) because the Nested ESXi configuration is exactly the same.
  • The second issue is having a duplicated ESXi System UUID, also known as a VMkernel UUID which should normally be unique and can sometimes be used for tracking purposes. You can see this System UUID by running the following ESXCLI command: esxcli system uuid get or by looking in esx.conf configuration file.

To properly clone an existing Nested ESXi VM, you will need to perform the following two operations within the Nested ESXi VM prior to cloning.

First Configuration - There is an advanced ESXi setting called FollowHardwareMac that will automatically update the VMkernel's MAC Address whenever the Virtual Machine's virtual network adapter MAC Addresses changes. To do so, you will need to run the following ESXCLI command:

esxcli system settings advanced set -o /Net/FollowHardwareMac -i 1

Second Configuration - The other modification that is required is to delete the existing System UUID entry in /etc/vmware/esx.conf configuration file. This will ensure a new System UUID will automatically be generated when the system boots up. To do so, open esx.conf and delete the entire /system/uuid line entry as seen in the screenshot below. Here is a quick snippet you can run without needing to open up VI:

sed -i 's#/system/uuid.*##' /etc/vmware/esx.conf

To ensure the file is persisted, run /sbin/auto-backup.sh

Once both configurations have been performed you are now ready to start cloning additional Nested ESXi instances. You will still need to login to each Nested ESXi VM and manually change the IP Address and hostname which you of course can leverage the Guest Operations API if you have VMware Tools for Nested ESXi installed.

If you plan on joining your "cloned" Nested ESXi instances to a vCenter Server and the ESXi hosts contains a local datastore, you will not be able to add the hosts to the same Datacenter/Cluster. The reason for this is that the cloned ESXi hosts will have a duplicated VMFS UUID. To fix this, you just need to re-signature the VMFS volume by using the following ESXCLI command:

esxcli storage vmfs snapshot resignature -l [VMFS-VOLUME]

How to get rid of "Quicks stats on hostX is not up-to-date" message

by // 10 Comments

After upgrading my Apple Mac Mini to ESXi 5.5 I noticed the following warning message being displayed after joining my ESXi host to my vCenter Server:

"Quick stats on mini is not up-to-date" 

I have seen this warning message in the past and it usually goes away within a few minutes of connecting to a vCenter Server. However, this time it did not go away and as you can see from the screenshot, I have had my ESXi host up for 4 days now and the message is still there.

UPDATE (9/30) - This looks to be a known issue in vSphere 5.5 and there is a permanent fix which has been documented in the following VMware KB 2061008. Restarting the management service will not prevent the warning message from coming again, please refer to the KB for the solution.

I do not know about you, but I like to have a clean environment and I get annoyed when I see warning/error messages in the UI. From what I can tell, vCenter Server was able to collect the "quick stats" from the ESXi host but perhaps there was a communication problem at some point or just a glitch?

In any case, is is pretty easy to fix the problem, you just need to restart the management service on the ESXi host and this will force a refresh of the stats. You have three ways of doing this:

  1. Restart management service using DCUI connecting to your ESXi console
  2. Restart management service using DCUI via command-line
  3. Restart management service using command-line script

Option 1 is pretty straight forward and both Option 2 and 3 can be performed on the command-line via an SSH session to your ESXi host if you have enabled SSH.

Option 2 - To launch the DCUI, just type "dcui" on the command-line and you will be able to interact with the DCUI as you normally would from the console and restart the management service:

Option 3 - To restart just using the command-line, you can run the following command to restrat the management service:

/sbin/services.sh restart

ESXi Simple Web (aka less ghetto webAccess)

by // 10 Comments

Do you remember webAccess for those of you who were fortunate enough to have worked with classic ESX? I do and I personally have found that feature in the past to be very useful especially when you need to perform a quick VM power operation and you just have access to a web browser. Recently, one of my readers, Wei Kin Huang left a comment on my blog about a very cool solution that he built called ESXi Simple Web which is based off of some earlier work I did with a VMware Fling called Ops Panel for ESX for which I created a Ghetto webAccess for ESXi.

In his own words, ESXi Simple Web provides the following:

Basic controls straight from the esxi host in the browser

This allows you to run basic operations against the ESXi MOB api from a simple web interface straight from the ESXi host without the need for the vCenter web client.

Inspired by virtuallyGhetto's Ghetto webAccess for ESXi

Wei's ESXi Simple Web definitely looks prettier and from what I can tell, it looks to be using Bootstrap, a popular web front-end framework for the UI. The web interface talks to the vSphere MOB which basically communicates to the vSphere API for providing the summary details and basic VM operations.

You can try out ESXi Simple Web by downloading the latest version over on his Github repo. Once downloaded, you just need to extract the content and SCP the "web" directory over to the ESXi host under /usr/lib/vmware/hostd/docroot

Here are a couple more screenshots of ESXi Simple Web running on my Apple Mac Mini with ESXi 5.5:

There is a Virtual Machines tab that you can drill down and get some information about your VMs and perform basic power on/off operations.

You can also drill down into an individual VM and get a nice thumbnail of the VM console as well as additional configurations for that VM.

If you are interested in adding additional functionality, you can contribute by sending a pull requests to esxi-simple-web Github project.