WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple
You are here: Home / VSAN / How to quickly setup and test VMware VSAN (Virtual SAN) using Nested ESXi

How to quickly setup and test VMware VSAN (Virtual SAN) using Nested ESXi

09.02.2013 by William Lam // 48 Comments

Last week at VMworld 2013, VMware announced the release of vSphere 5.5 which includes a variety of exciting new features.  One of the most anticipated feature introduced in this release is VMware Virtual SAN (VSAN) which will be available initially as a public beta. One question that I heard repeatedly throughout the VMworld conference was whether it would be possible to test VSAN in a nested ESXi environment? The answer is absolutely! This is a great way to learn about VSAN and how works from a functional perspective before procuring the necessary hardware.

Disclaimer: Running VSAN in a nested ESXi environment is not officially supported nor is it a replacement for actual testing on actual physical hardware.

Before getting started, I would highly recommend you check out the following resources from my good friend Cormac Hogan which includes a detailed VSAN walk through as well what looks to be an awesome series of articles on how VSAN works:

  • VSAN Walkthrough
  • VSAN Part 1 - A first look at VSAN
  • VSAN Part 2 - What do you need to get started

Requirements:

  • Environment running either vSphere 5.1 or 5.5 and access to the vSphere Web Client.

Configuration:

Nested ESXi VM configured with the minimal resources:

  • 2 vCPU
  • 5GB Memory (ESXi 5.5 now requires a minimum of 4GB vs 2GB as with previous releases but VSAN requires minimum of 5 with recommended 6)
  • 2GB Disk for ESXi 5.5 installation
  • 4GB Disk for an "Emulated" SSD
  • 8GB Disk for HDD

Easy Method:

Instead of having you go through the process of building a Nested ESXi VM with all the prerequisites that includes steps from here and here. I have pre-built a VSAN Nested ESXi VM template (217Kb) that you can just download and import into your environment and being the installation process.

Download either:

  • Single VSAN Nested ESXi VM Template
  • 3-Node VSAN Nested ESXi VM Template
  • 32-Node VSAN Nested ESXi VM Template

and connect to your vCenter Server 5.1 or 5.5 using the vSphere Web Client and import the OVF into your environment (do not use the vSphere C# Client as the import does not persist VHV configuration). Once you have imported the VM, you can then mount the ESXi 5.5 ISO and begin the installation. All three VMDKs have been thin provisioned and you can change the capacity during deployment.

Slightly Harder Method:

If you wish to build the Nested ESXi VM yourself, then you can follow these instructions:

Step 1 - Create a new VM and when you get to the compatibility screen, select either "ESXi 5.1 or greater" or "ESXi 5.5 or greater" depending on the version of vSphere you are running

Step 2 - For the GuestOS select "Other" and "Other (64-bit)"

Step 3 - We will need to customize the following virtual hardware configuration:

  • Change vCPU to 2
  • Click on CPU drop down and enable "Expose hardware assisted virtualization to the guest OS"
  • Change Memory to 4GB
  • Change the initial VMDK to 2GB or whatever value you wish to use for ESXi installation
  • Add second VMDK with 4GB or whatever value you wish to use for "emulated" SSD
  • Add third VMDK with 8GB of whatever value you wish to use for the HDD
  • Click on the VM Options tab at the top and select the "Advanced" drop down box. We will need to add the following entry scsi0:1.virtualSSD = 1 For more details please refer to this article

Step 4 - Click okay to provision the VM and once it has been deployed you will need to re-configure the guestOS to "VMware ESXi 5.x" using the vSphere C# Client for vSphere 5.1 or vSphere Web Client for vSphere 5.5. At this point, you will have the same VM image as in the Easy Method and you are now ready to install ESXi 5.5

When you install ESXi 5.5, you should see the following three disks as shown in the screenshot below, ensure you install ESXi on the 2GB disk:

Prior to enabling VSAN on the particular vSphere Cluster, make sure you enable the new VSAN traffic type on one of your VMkernel interfaces for each of your ESXi hosts, this is required for VSAN communication.

If all the prerequisites have been met, you can now easily enable VSAN by simply checking the VSAN box when editing the vSphere Cluster. In just a few minutes you should see diskgroups automatically created (assuming you selected Automatic mode) consuming both the emulated SSD and HDD and the creation of the vsanDatastore which will be available on all ESXi hosts within that vSphere Cluster.

You can also use the same method for emulating an SSD running in a Nested ESXi to functional test the new VMware Flash Read Cache (vFRC) feature.

More from my site

  • How to run Nested ESXi on top of a VSAN datastore?
  • Quick stats for the VSAN HCL
  • How to run Nested ESXi on the vCloud Hybrid Service?
  • VSAN Flash/MD capacity reporting
  • OVF template for creating Nested ESXi 3 or 32 node VSAN Cluster

Categories // VSAN, vSphere 5.5 Tags // nested, ssd, vflash, vFRC, Virtual SAN, VSAN, vSphere 5.5

Comments

  1. *protectedGabriel Chapman says

    09/06/2013 at 2:32 pm

    Isn't this essentially testing the GUI?

    Reply
    • *protectedWilliam Lam says

      09/06/2013 at 2:42 pm

      Not sure I follow your comment?

      Reply
  2. *protectedGabriel Chapman says

    09/09/2013 at 5:06 pm

    Sorry for not being clear. Without the hardware available to test the actual functionality of vSAN, running it in nested ESXI isn't really going to do much other than show you the configuration screens. I think in order to get the full benefit, this is a technology that needs to be deployed and actually tested in regards to performance, etc.

    Reply
  3. *protectedJason Ruiz says

    09/25/2013 at 3:09 pm

    Just to note, if it's less than 4GB(I chose 1 for the lab), it won't work and it won't error out

    Reply
  4. *protectedVirtu-Al says

    01/22/2014 at 12:23 am

    Just a note that I was testing this in fusion and on SSD, a few things needed to be different:

    1. The template doesnt import so you need to create the VM manually
    2. As i was running the ESX VMs on SSD they automatically show as SSD without the change
    3. You need to change the third disk to be not SSD by using scsi0:2.virtualSSD = 0

    Reply
  5. *protectedCorbonno Klok says

    02/01/2014 at 11:58 am

    for some reason i do not get the nics in that position that all three hosts do see theier local drive's to bundle them as just one box of disks.
    the summery of the SAN Cluster still say "Misconfiguration detected" although all three ESX5.5i boxes do have a vmkernel for vsan configured all on one vlan on just one fysical nic.
    any idea how to solve this ??

    Reply
    • *protectedWilliam Lam says

      02/02/2014 at 9:28 pm

      Not exactly sure I follow what issues you're having.

      Reply
    • *protectedBrian says

      04/04/2014 at 10:34 pm

      I saw the same message,it was related to having a second vMkernel port group configured for vSAN. I removed the second portgroup and enabled vSAN on the primary management portgroup and fixed it right away.

      Reply
  6. *protectedcrvinva says

    02/24/2014 at 11:33 pm

    I'm also running into the "Misconfiguration detected" vSAN network status error in my nested ESXi environment right after I turn on virtual SAN. The C# client says "Host cannot communicate with all other nodes in the VSAN enabled cluster" on the Summary tab. I'm able to ping across all three vSAN DHCP IPs from the nested ESXi hosts. I'm using vCenter 5.5.0 Build 1440532 and ESXi 5.5.0 Build 1439689 for both nested and physical ESXi. I'm using the template above. Anything else I can check to help understand what's failing? Any log files? All three nested ESXi virtual hosts are running on the same physical host. Can this still be a multicast issue? Thanks.

    Reply
    • *protectedAlex says

      03/03/2014 at 8:02 pm

      I narrowed the issue down to one specific item: subnet mask must not be 255.255.0.0. Subnet mask 255.255.255.0 works fine.
      When using DHCP (as this WMware YouTube video shows) subnet mask 255.255.0.0 is used, and vSAN communication fails.

      Reply
  7. *protectedStephen says

    03/05/2014 at 5:56 am

    For those experiencing the "Misconfiguration detected" network status error on their VSAN setup running in a Nested ESXi environment, make sure you have Promiscuous Mode enabled (accepted) on the Port Group/vSwitch that the nested host VMs are attached to. This is normal for a Nested ESXi environment, but might have been missed during a new setup to test VSAN.

    Reply
    • *protectedArwed Schmidt says

      06/01/2014 at 9:04 am

      I also ran into this infamous problem. Turned out that my mistake was to convert a preconfigured ESXi-VM to a template and deployed the other VMs from that. Does *NOT* work with VSAN!
      Now the template contains exerything but ESXi iteslf. Has to be installed after boot in every new VM.

      Reply
  8. *protectedMiLo says

    03/13/2014 at 12:34 pm

    I'm running into the issue that VSAN is unable to create LSOM file system for the data disk, i'm using a SSD and used the NO-SSD parameter, for the rest there are no error

    Reply
    • William Lam says

      03/13/2014 at 2:10 pm

      Are you using my OVF template & is this 5.5u1?

      Reply
  9. *protectedTest says

    03/13/2014 at 9:42 pm

    I too get the "Unable to create LSOM file system". I created the disks myself and did not use the OVF yet.

    Reply
    • William Lam says

      03/14/2014 at 4:37 am

      How much memory did you allocate, I suspect you may not have enough memory which is why you're seeing this problem. You can always use the OVF template which will work OOTB

      Reply
  10. *protectedmkm says

    06/16/2014 at 11:13 pm

    I have a two node cluster physical servers and I used your 3-Node OVF. So now I have a vApp with three VMs on which I have now installed ESXi 5.5. I am unclear of next steps -

    "Prior to enabling VSAN on the particular vSphere Cluster, make sure you enable the new VSAN traffic type on one of your VMkernel interfaces for each of your ESXi hosts, this is required for VSAN communication"

    I do not see an option of vSAN traffic type on my physical servers. Also, should I move the nested servers out from vApp into a new cluster so I can enable vSAN on that cluster?

    Reply
    • William Lam says

      06/17/2014 at 1:03 am

      The VSAN Traffic Type is not on your physical servers 🙂 This is in the VMkernel interface, you can do this using the vSphere Web Client by browsing to your ESXi host and selecting Networking and VMkernel Adapters and you can add "Virtual SAN" as a traffic type.

      The other option is to do this via the command-line via ESXCLI using "esxcli vsan network ipv4 add -i "

      Reply
      • *protectedmkm says

        06/19/2014 at 12:20 am

        I had missed the fundamental step of creating a new cluster and adding the new VMs, as hosts, to the cluster. All all settings I enabled vSAN and got a vsandatastore and diskgroups for individual hosts but none of the eligible disks were pulled in. Looking at tasks I see "A general system error occurred. Can not add disk: VSAN is not licensed on this host".

        I am using the default 60 day trial license for VSAN but I thought that would work.

        Reply
  11. *protectedCharles says

    06/18/2014 at 8:03 am

    Hi William, I deployed the nested VM OVF, attached to vCenter, applied VSAN license and done the VSAN network creation, however for some reason none of the disk is eligible for VSAN from the web client.. Did I missed out anything?
    Thanks

    Reply
    • William Lam says

      06/18/2014 at 1:52 pm

      How did you import the OVF, remember it needs to be done using the vSphere Web Client as mentioned in the article. Else you will have issues

      Reply
  12. *protectedcharles says

    06/19/2014 at 12:08 am

    Hi William,
    Yes, imported the Off via the Web client...
    Cheers,

    Reply
    • William Lam says

      06/19/2014 at 1:52 am

      That's quite strange. Did you edit the Nested ESXi VMs such as disk size or memory? When ESXi is installed, do you see the other disks and are they marked appropriately?

      Reply
  13. *protectedEAS says

    08/25/2014 at 6:39 pm

    William - Trying to setup a VSan and ran into an issue. I was hoping you can shed some light on my question.

    3 Identical servers, each having a Fusion IO used for cache, 1 pair raid 1 SAS drives for VM Host, 1 Pair Raid 1 SSD for Disk Group 1 and 1 pair Raid 1 SAS HD for Disk Group 2. Can I share the Fusion IO cache for both Disk Groups? The system seems to not want to allow me to configure it like this.

    Reply
  14. *protectedYusein says

    02/18/2015 at 4:53 pm

    Hi

    I´m trying to deploy the Nested-ESXi-3-Node-VSAN-Template in an ESXi 5.1.0 but this message appears:

    "The ovf package requires support for vApps with multiple VMs.
    Details: Line 23: Unsupported element 'VirtualSystemCollection'"
    Do i need to deploy it in a server with different capabilities or a diferent version of ESXi?
    What type of server do i need to make it work?

    Any advice please.
    Thanks in advance.

    Regards

    Reply
    • William Lam says

      02/18/2015 at 4:56 pm

      You will need to deploy the OVF against a vCenter Server using the vSphere Web Client.

      Reply
      • *protectedYusein says

        03/19/2015 at 1:25 am

        Thanks for your reply William, sorry for the delay.

        Reply
  15. *protectedJohn Washabaugh says

    06/07/2015 at 4:34 pm

    I set this up like most others, on an ssd. Everything configured fine, got errors to go away with enabling igmp on my Asus home router but as soon as I put any mild load on it, like vmotion or install win on new VM, I get the host communication error and whatever I'm doing fails. Anyone seeing this? Did I miss configure something?

    Reply
  16. *protectedAndrew Zimmerman says

    04/06/2016 at 1:35 pm

    I did all of that, but it's complaining about upgrading the disks from 1 to 2 (I'm using esxi 6 & vSphere 6).
    I'm unable to upgrade them.
    If you could do a tutorial on doing VSAN 6, that would be great.
    Thanks.

    Reply
  17. *protectedDanilo says

    12/04/2017 at 8:10 am

    Thanks for these templates! I ended up creating my own manually for 6.5. One thing I'm seeing is an issue with NICs (may not be an issue but me). So i have 4 NICs in a Supermicro E200-8D server. I can't seem to utilize the 3 additional NICs on the nested hosts. Not sure if I'm missing something so for now I have vmotion, vsan & management all using the same vmkernel port. Not preferred but for the lab and in a pinch its working fine. Any ideas on how I can leverage the additional NICs?

    Reply

Trackbacks

  1. Welcome to vSphere-land! » VSAN Links says:
    03/01/2014 at 12:47 am

    […] VSAN configuration steps with nested ESXi hypervisors (ESX Virtualization) How to quickly setup and test VMware VSAN (Virtual SAN) using Nested ESXi (Virtually […]

    Reply
  2. How to bootstrap vCenter Server onto a single VSAN node Part 1? | virtuallyGhetto says:
    03/08/2014 at 3:04 pm

    […] publicly available in the very near future. I have been doing some testing in my lab with VSAN, not Nested VSAN, but on actual physical hardware. While getting started, I hit an interesting challenge given my […]

    Reply
  3. TinkerTry IT @ home | Build your own VMware vSphere 5.5 Datacenter with ESXi and vCSA says:
    03/12/2014 at 6:33 pm

    […] successfully since April of 2011. Going forward, a tinkering with vSphere Flash Read Cache and VSAN sounds fun to […]

    Reply
  4. Re: Host is in a VSAN enabled cluster but does not have VSAN service enabled | virtuallyGhetto says:
    03/18/2014 at 11:32 pm

    […] a couple of people hitting a warning message when configuring VSAN and specifically when running VSAN in a Nested ESXi environment (which is not officially supported by VMware). The warning message is displayed on the […]

    Reply
  5. vdq – A useful little VSAN utility | virtuallyGhetto says:
    03/20/2014 at 2:48 pm

    […] re-building a couple of my Nested ESXi VMs for VSAN using some newer builds, I came across a nifty little VSAN utility called vdq which I assume stands […]

    Reply
  6. Quick Tip – Increasing capacity on a Nested VSAN Datastore | virtuallyGhetto says:
    03/21/2014 at 9:48 pm

    […] other day I needed to increase the capacity on one of my Nested VSAN Datastores as one of our users required a larger VSAN datastore than it was initially configured […]

    Reply
  7. How to configure vSAN on nested ESXi hosts with SSD hard disk | Wu's Blog says:
    03/26/2014 at 3:15 pm

    […] lot of articles introduce vSAN feature and steps by steps guide. I referred William Lam's article & Duncan's article to configure vSAN on my lab, I was true I exactly followed his steps […]

    Reply
  8. Consolidated list of all Virtual SAN (VSAN) deep dive resources. | says:
    04/09/2014 at 1:16 am

    […] How to quickly setup and test VSAN using Nested ESXi (William Lam) […]

    Reply
  9. OVF template for creating Nested ESXi 3 or 32 node VSAN Cluster | virtuallyGhetto says:
    04/15/2014 at 3:16 pm

    […] week I had to build a couple of Nested VSAN environments for testing and of course I used my VSAN Nested ESXi OVF template to help expedite the deployment. […]

    Reply
  10. TinkerTry IT @ home | Build your own VMware vSphere Datacenter in under an hour with the free ESXi 5.5 hypervisor says:
    05/03/2014 at 3:36 am

    […] successfully since April of 2011. Going forward, a tinkering with vSphere Flash Read Cache and VSANsounds fun to […]

    Reply
  11. Creating a VSAN Cluster with PowerCLI | Online News Portal says:
    05/04/2014 at 7:24 am

    […] I wrote a post on the PowerCLI Blog which showed the cmdlets and some examples on how you might use these, recently I had the chance to put them to use when I needed to stand up a new VSAN cluster, I was able to easily build myself a test cluster with the help of Williams post here. […]

    Reply
  12. VMware Link Collection | Life says:
    10/31/2014 at 4:18 am

    […] – VSAN (Virtual-blog) STO5027 VMware Virtual SAN Technical Best Practices (virtualarchitect.nl) How to quickly setup and test VMware VSAN (Virtual SAN) using Nested ESXi (Virtually Ghetto) Additional steps required to completely disable VSAN on ESXi host (Virtually […]

    Reply
  13. How to quickly setup and test VMware VSAN using Nested ESX | Virteam says:
    11/24/2014 at 12:29 pm

    […] via How to quickly setup and test VMware VSAN (Virtual SAN) using Nested ESXi | virtuallyGhetto. […]

    Reply
  14. How to configure an All-Flash VSAN 6.0 Configuration using Nested ESXi? | Uğur PEK says:
    02/28/2015 at 10:15 pm

    […] process but should not be a substituted for actual hardware testing. You will need a minimum of 3 Nested ESXi hosts and they should be configured with at least 6GB of memory or more when working with VSAN […]

    Reply
  15. How to deploy vSphere 6.0 (VCSA & ESXi) on vCloud Director and vCloud Air? | virtuallyGhetto says:
    10/21/2015 at 7:20 pm

    […] installed image). To help expedite the deployment of Nested ESXi in vCloud Air, I have built several Nested ESXi OVF Templates that that you can use. You will also need to upload an ESXi 6.0 ISO or whichever version of ESXi […]

    Reply
  16. Building a VMware home lab with AutoLab. | Ryan Birk – Virtual Insanity says:
    11/02/2016 at 2:23 pm

    […] all VMware features are supported. Fault Tolerance and vSAN will give you a bit of grief trying to configure it. The two links listed will help you get it […]

    Reply
  17. VMware VSAN configuration steps with nested ESXi hypervisors - ESX Virtualization says:
    01/21/2017 at 8:06 am

    […] even simplify the creation of the template of ESXi 5.5 VM with disks, you can even download William Lam's OVF template from his post, where the ESXi has one of the disks already tagged as SSD. (saves you […]

    Reply
  18. Server virtualization nested and tiered hypervisors - StorageIOblog says:
    09/29/2017 at 6:57 pm

    […] knowledge base piece, two are from William Lam (@lamw) Virtual Ghetto (getting started here and VSAN here) and the other is from Duncan Epping @DuncanYB Yellow Bricks […]

    Reply

Thanks for the comment!Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025