WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

Quick Tip - Increasing capacity on a Nested VSAN Datastore

03.21.2014 by William Lam // 2 Comments

The other day I needed to increase the capacity on one of my Nested VSAN Datastores as one of our users required a larger VSAN datastore than it was initially configured for. I was expecting to be able to just increase the size of the underlying VMDKs like I would for a traditional Nested ESXi environment and rescan in ESXi to pick up the new capacity without any downtime. It turns out, this is was not exactly the case for a Nested VSAN environment.

increase-capacity-nested-vsan-datastore-0
Disclaimer: Nested Virtualization is not officially supported by VMware

When you first setup VSAN, regardless of how the disks were claimed, VSAN will consume the entire device (SSD or MD). The capacity that VSAN initially detects will then be used to create the necessary partition as part of the VSAN Disk Group creation. VSAN assumes that the capacity for the underlying devices would never change as in the "real" world, disks do not auto-magically get larger 🙂 and this is a valid assumption. In a Nested ESXi environment however, it can auto-magically get larger but VSAN was not built for this use case. What ends up happening is that the underlying devices can be "hot-extended" but the existing VSAN Disk Group can not detect this new capacity.

Having said that, there are two ways you can increase your VSAN datastore:

Option 1 - If you wish to preserve your VSAN Datastore, you can hot-add additional VMDK(s) to your existing VSAN Disk Group or if it is full, you can create a new disk group and add additional VMDK(s). This will modify your setup slightly if you wanted a particular set of disk groups but will allow you to preserve your data.

Option 2 - The latter option requires the deletion and re-creation of the VSAN Datastore which is not ideal if you already have data on it. You will need to increase the capacity of the underlying VMDKs and then re-create your VSAN Datastore, but this way you can keep the existing number of disks and disk groups you initially created your Nested ESXi environment with.

In my scenario, I could not destroy the VSAN Datastore as I had someone using it and so I opted for option #1. Here is what my configuration looked like before which was a single VSAN Disk Group with 1xSSD and 1xMD:

increase-capacity-nested-vsan-datastore-1
I then added an additional 10GB VMDK to each of my Nested ESXi hosts and issue a rescan so the ESXi host would pickup the new device:

increase-capacity-nested-vsan-datastore-2
In just a few seconds, I can see my new storage device. I can now head over to the VSAN management page which is located at the vSphere Cluster and once I refresh, I can see that VSAN has automatically added the new "MD" into the existing disk group and my storage has automatically expanded!

increase-capacity-nested-vsan-datastore-3

Categories // Nested Virtualization, VSAN, vSphere 5.5 Tags // nested virtualization, VSAN, vSphere 5.5

Re: Host is in a VSAN enabled cluster but does not have VSAN service enabled

03.18.2014 by William Lam // Leave a Comment

I recently noticed a couple of people hitting a warning message when configuring VSAN and specifically when running VSAN in a Nested ESXi environment (which is not officially supported by VMware). The warning message is displayed on the summary page of the ESXi host which states the following:

Host is in a VSAN enabled cluster but does not have VSAN service enabled

vsan-minimal-memory
The reason you are seeing this issue is related to the amount of memory you have allocated for your Nested ESXi VM. During the VSAN Beta, the minimal amount of memory was 4GB but it looks like that has changed to 5GB with the GA of vSphere 5.5 Update 1 last week. I know we could have done better job with the error message and communicate the actual underlying issue (will ensure we have an FR filed for this).

However, the fix is quite simple, just shutdown your Nested ESXi VM and then change it to 5GB and this message will go away. It is also worth noting that as you increase the number of disks and disk groups in your ESXi hosts, there will be an increase in memory. I would highly recommend you take a look at the official VSAN Design & Sizing guide to properly size out your real VSAN environments.

For basic functional testing and education of VSAN (not including running additional VMs), running a Nested ESXi VM with 5GB will be sufficient. You can also take a look at the my VSAN Nested ESXi OVF template which can just download and install ESXi 5.5 Update 1 without any issues.

Categories // Nested Virtualization, VSAN, vSphere 5.5 Tags // nested virtualization, VSAN, vSphere 5.5

How to properly clone a Nested ESXi VM?

12.06.2013 by William Lam // 53 Comments

I often hear from users that they would like to be able to just clone from an existing Nested ESXi VM that has already been configured and just create additional Nested ESXi VM instances from that. For me personally, I do not have a use case for this since I just deploy additional ESXi instances using an automated Kickstart deployment. However, I can see why this would be useful for anyone that does not have an automated deployment or just want to quickly deploy additional Nested ESXi instances by just cloning from an existing image and then manually change the networking configuration afterwards.

UPDATE (07/01/21) - As of ESXi 7.0 Update 2, cloning an ESXi boot volume (Nested or Physical) is no longer safe and can lead to data corruption. Please refer to the following two VMware KB articles for more information on this topic https://kb.vmware.com/kb/84280 and https://kb.vmware.com/kb/84349 

First off, cloning of a Nested ESXi VM is possible and you can already do this today. You will get a brand new Virtual Machine that will have a unique MoRef ID, InstanceUUID, BIOS UUID and MAC Addresses for each of the virtual network adapters which you can see an example of this from the screenshot below.

Everything from outside of the guest OS looks great as we would expect but there is actually two issues from within ESXi that you may not be aware of.

  • The first issue is that you will get a duplicated MAC Address of the VMkernel interface(s) because the Nested ESXi configuration is exactly the same.
  • The second issue is having a duplicated ESXi System UUID, also known as a VMkernel UUID which should normally be unique and can sometimes be used for tracking purposes. You can see this System UUID by running the following ESXCLI command: esxcli system uuid get or by looking in esx.conf configuration file.

To properly clone an existing Nested ESXi VM, you will need to perform the following two operations within the Nested ESXi VM prior to cloning.

First Configuration - There is an advanced ESXi setting called FollowHardwareMac that will automatically update the VMkernel's MAC Address whenever the Virtual Machine's virtual network adapter MAC Addresses changes. To do so, you will need to run the following ESXCLI command:

esxcli system settings advanced set -o /Net/FollowHardwareMac -i 1

Second Configuration - The other modification that is required is to delete the existing System UUID entry in /etc/vmware/esx.conf configuration file. This will ensure a new System UUID will automatically be generated when the system boots up. To do so, open esx.conf and delete the entire /system/uuid line entry as seen in the screenshot below. Here is a quick snippet you can run without needing to open up VI:

sed -i 's#/system/uuid.*##' /etc/vmware/esx.conf

To ensure the file is persisted, run /sbin/auto-backup.sh

Once both configurations have been performed you are now ready to start cloning additional Nested ESXi instances. You will still need to login to each Nested ESXi VM and manually change the IP Address and hostname which you of course can leverage the Guest Operations API if you have VMware Tools for Nested ESXi installed.

If you plan on joining your "cloned" Nested ESXi instances to a vCenter Server and the ESXi hosts contains a local datastore, you will not be able to add the hosts to the same Datacenter/Cluster. The reason for this is that the cloned ESXi hosts will have a duplicated VMFS UUID. To fix this, you just need to re-signature the VMFS volume by using the following ESXCLI command:

esxcli storage vmfs snapshot resignature -l [VMFS-VOLUME]

Categories // ESXi, Nested Virtualization Tags // clone, ESXi, nested, nested virtualization, uuid

  • « Previous Page
  • 1
  • …
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • VMware Flings is now available in Free Downloads of Broadcom Support Portal (BSP) 05/19/2025
  • VMUG Connect 2025 - Minimal VMware Cloud Foundation (VCF) 5.x in a Box  05/15/2025
  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...