WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple
You are here: Home / VSAN / Required ESXi advanced setting to support 16+ node VSAN Cluster

Required ESXi advanced setting to support 16+ node VSAN Cluster

03.13.2014 by William Lam // Leave a Comment

This evening I came across a very interesting VSAN knowledge base article KB 2073930 which states: to support a VSAN Cluster with greater than 16+ nodes, an ESXi Advanced Setting must be configured. The wording in the current KB is a bit confusing as it mentions the advanced setting must be applied to all hosts and then it mentions it should only be applied to the "additional" hosts. I have reached out to engineering and confirmed that the setting should be applied to ALL ESXi hosts and the KB will be corrected shortly. After applying this setting, you will also need to reboot your ESXi host for the changes to take affect.

The advanced setting we are interested in is called CMMDS.goto11 and you can check its current value by running the following ESXCLI command:

esxcli system settings advanced list -o /CMMDS/goto11

The following ESXCLI command will configure the required setting (default it is 0):

esxcli system settings advanced set -o /CMMDS/goto11 -i 1

The reason this setting is disabled by default is that it does consume a tiny amount of memory, we are talking about sub ~150MB per ESXi host. Though this amount is extremely small, the VSAN Engineers wanted to be very cognizant of smaller VSAN deployments and not have to require additional resources if it is not necessary. In my opinion if you plan to eventually scale beyond 16+ nodes in a VSAN Cluster and you can afford the additional memory overhead, then I would recommend enabling this setting as part of your default configuration so you do not need to do it later. If not, you can always do it later, but you know how that goes 🙂

To assist with this change, I have created two scripts: vSphere SDK for Perl and PowerCLI which will automate the configuration of this advanced setting for ALL ESXi hosts within a VSAN Cluster.

Disclaimer:  These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

You can download the sample vSphere SDK for Perl script called: enable16PlusVSANNodeSupport.pl

Here is an example of how you would run it:

./enable16PlusVSANNodeSupport.pl --server vcenter55-1.primp-industries.com --username root --cluster VSAN-Cluster

vsan-cluster-support
UPDATE (07/27/17) - Here is a PowerCLI script called VSANLargeClusterAdvSettings.ps1 which contains two functions (Get-VsanLargeClusterAdvancedSetting and Set-VsanLargeClusterAdvancedSetting) for retrieving and updating the necessary ESXi Advanced Settings to enable large vSAN Clusters for ESXi hosts running on 5.5, 6.0 & 6.5. There are two different KBs depending on the version of ESXi, please see https://kb.vmware.com/kb/2073930 and https://kb.vmware.com/kb/2110081 for more details.


Finally, some fun VSAN trivial from the engineering team. You may have noticed the advanced settings is called "goto11" which I was initially scratching my head over on why the engineers would name it that. I initially though it was either binary 11 OR it was a literal GoTo statement. It turns out, the answer can be found in this YouTube video here. Thanks to Dinesh Nambisan for sharing this awesome little tidbit. I wonder what other surprises lies in the code 🙂

More from my site

  • A killer custom Apple Mac Mini setup running VSAN
  • Automate VSAN Observer offline mode configurations
  • Does VSAN work with Free ESXi?
  • ESXi 5.5 Kickstart script for setting up VSAN
  • Does reinstalling ESXi with an existing VSAN Datastore wipe your data?

Categories // VSAN, vSphere 5.5 Tags // cmmds, VSAN, vSphere 5.5

Thanks for the comment!Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...