VSAN – Page 34

Exploring VSAN APIs Part 5 – VSAN Host Status

03.17.2014 by William Lam // 1 Comment

Another useful bit of information from a troubleshooting perspective is to be able to quickly get summary of all your ESXi hosts participating in a VSAN cluster. Critical information such as the health, state, VSAN Node/Cluster UUID for each ESXi host can be queried through the vsanSystem using the QueryHostStatus() vSphere API method. To demonstrate this simple operation, I have created a vSphere SDK for Perl sample script called vsanHostDiskMapping.pl.

Disclaimer: These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

To show the ESXi host status, you will need to specify a vSphere Cluster running VSAN and use the following command:

./vsanHostStatus.pl --server vcenter55-1.primp-industries.com --username root --cluster VSAN-Cluster

As you can see this is an easy way to quickly get the health of ESXi host participating in the VSAN cluster.

Required ESXi advanced setting to support 16+ node VSAN Cluster

03.13.2014 by William Lam // Leave a Comment

This evening I came across a very interesting VSAN knowledge base article KB 2073930 which states: to support a VSAN Cluster with greater than 16+ nodes, an ESXi Advanced Setting must be configured. The wording in the current KB is a bit confusing as it mentions the advanced setting must be applied to all hosts and then it mentions it should only be applied to the "additional" hosts. I have reached out to engineering and confirmed that the setting should be applied to ALL ESXi hosts and the KB will be corrected shortly. After applying this setting, you will also need to reboot your ESXi host for the changes to take affect.

The advanced setting we are interested in is called CMMDS.goto11 and you can check its current value by running the following ESXCLI command:

esxcli system settings advanced list -o /CMMDS/goto11

The following ESXCLI command will configure the required setting (default it is 0):

esxcli system settings advanced set -o /CMMDS/goto11 -i 1

The reason this setting is disabled by default is that it does consume a tiny amount of memory, we are talking about sub ~150MB per ESXi host. Though this amount is extremely small, the VSAN Engineers wanted to be very cognizant of smaller VSAN deployments and not have to require additional resources if it is not necessary. In my opinion if you plan to eventually scale beyond 16+ nodes in a VSAN Cluster and you can afford the additional memory overhead, then I would recommend enabling this setting as part of your default configuration so you do not need to do it later. If not, you can always do it later, but you know how that goes 🙂

To assist with this change, I have created two scripts: vSphere SDK for Perl and PowerCLI which will automate the configuration of this advanced setting for ALL ESXi hosts within a VSAN Cluster.

Disclaimer: These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

You can download the sample vSphere SDK for Perl script called: enable16PlusVSANNodeSupport.pl

Here is an example of how you would run it:

./enable16PlusVSANNodeSupport.pl --server vcenter55-1.primp-industries.com --username root --cluster VSAN-Cluster

UPDATE (07/27/17) - Here is a PowerCLI script called VSANLargeClusterAdvSettings.ps1 which contains two functions (Get-VsanLargeClusterAdvancedSetting and Set-VsanLargeClusterAdvancedSetting) for retrieving and updating the necessary ESXi Advanced Settings to enable large vSAN Clusters for ESXi hosts running on 5.5, 6.0 & 6.5. There are two different KBs depending on the version of ESXi, please see https://kb.vmware.com/kb/2073930 and https://kb.vmware.com/kb/2110081 for more details.

Finally, some fun VSAN trivial from the engineering team. You may have noticed the advanced settings is called "goto11" which I was initially scratching my head over on why the engineers would name it that. I initially though it was either binary 11 OR it was a literal GoTo statement. It turns out, the answer can be found in this YouTube video here. Thanks to Dinesh Nambisan for sharing this awesome little tidbit. I wonder what other surprises lies in the code 🙂

Exploring VSAN APIs Part 4 – VSAN Disk Mappings

03.10.2014 by William Lam // Leave a Comment

A useful feature that is provided in the vSphere Web Client for VSAN is the ability to visualize the VSAN disk mappings for each ESXi host contributing to the VSAN cluster. This information will come in handy for troubleshooting as well as checking or servicing a disk failure.

You can easily retrieve the VSAN Disk Groups for each ESXi host and their respective SSD and HDD within that disk group by looking at the following property vsanSystem->config->storageInfo->diskMapping. There is a variety of information for each device such as the name, capacity, state, health, etc. of each disk. To demonstrate the above operation, I have created a vSphere SDK for Perl sample script called vsanHostTrafficTypeMgmt.pl

Disclaimer: These scripts are provided for informational and educational purposes only. It should be thoroughly tested before attempting to use in a production environment.

To show the disk mappings, you will need to specify a vSphere Cluster running VSAN using the following command:

./vsanHostDiskMapping.pl --server vcenter55-1.primp-industries.co --username root --cluster VSAN-Cluster

In the above output, you will see the VSAN Disk Groups for each ESXi host along with their respective device name and capacity. Here is another screenshot for a VSAN environment that contains multiple VSAN Disk Groups:

This is just a very tiny sub-set of the available properties for a disk, for more information take a look at the scsiLun definition within the vSphere API Reference.