WilliamLam.com

Quick Tip - smartd configurable polling interval in vSphere 6.0

by // 1 Comment

In vSphere 5.1, one of the major storage enhancements that was part of the new I/O Device Management (IODM) framework was the addition of SMART (Self Monitoring, Analysis And Reporting Technology) data for monitoring FC, FCoE, iSCSI, SAS protocol statistics, this is especially useful for monitoring the health of an SSD device. The SMART data is provided through a SMART daemon which lives inside of ESXi and runs every 30 minutes to gather statistics and diagnostic information from the underlying storage devices and provides the information through the following ESXCLI command:

esxcli storage core device smart get -d [DEVICE]

Screen Shot 2015-02-20 at 4.14.06 AM
If you would like to learn more about IODM and SMART, be sure to check out Cormac Hogan's in-depth article here.

The default polling interval for the SMART daemon in vSphere 5.1 was not configurable and 30 minutes was the system default. For most customers, the out of the box configuration should be sufficent. However, for some customers who wish to have greater flexibility in the polling frequency, the default can now be adjusted in vSphere 6.0. The smartd process now includes a new -i option which specifies the polling interval.

[root@mini:~] smartd -h
smartd: option requires an argument -- 'h'
smartd <options>
-i   Polling interval (in minutes) for smartd
(default Polling interval is 30 minutes)

If you wish to change the default, you will need to modify the /etc/init.d/smartd init script and include the interval option. One issue that I have found is that changes to the init script do not persist reboots as modifications to these files should not be performed by users. In the case of adjusting the polling interval, we need to add the additional option for smartd startup.

We can still accomplish this by adding the following to /etc/rc.local.d/local.sh make the necessary adjustments and restarting the smartd process:

SMARTD_POLL_INTERVAL=35
/etc/init.d/smartd stop
sed -i "s/^SMARTD_SCHED_PARAM.*/SMARTD_SCHED_PARAM=\"-i ${SMARTD_POLL_INTERVAL} ++group=smartd\"/g" /etc/init.d/smartd
/etc/init.d/smartd start

Note: The -i option is only visible when smartd process is not running

If you wish to see the changes live immediately, then you can run /etc/rc.local.d/local.sh command once or this will automatically happen upon ESXi booting up. If we perform a process look up using "ps", we can see that our smartd is now configured to poll every 35 minutes instead of the default 30.

Screen Shot 2015-02-18 at 6.00.52 PM

Quick Tip - "poor mans" history command in ESXi

by // 1 Comment

When I am logged into the ESXi Shell, I often forget that history command is not implemented in ESXi which can be helpful when recalling the list of operations that had been executed in the past. I especially rely on the history command when I am tinkering around with things and once I am successful with the end result, I can easily go back and see the exact steps I took. Recently, I tried running history command only to be let down again as I forgot it was not implemented 🙁

I was thinking there had to be a way on ESXi and then it hit me! Starting with ESXi 5.1, all operations executed in ESXi Shell and Console were automatically logged to /var/log/shell.log. The information I was looking was there but instead of having to manually view the contents of the log, I could simply create an "alias" to  a history command which could display the last N-number of entries using the tail command.

Here is an example alias to "history" command to view the last 50 lines in /var/log/shell.log:

alias history="tail -50 /var/log/shell.log"

To make the alias permanent and persist across reboots, we just need to add the entry to /etc/profile.local

Now, I can run the history command on ESXi and get exactly what I want.

Screen Shot 2015-02-18 at 9.29.41 PM
Note: Entries in /var/log/shell.log contain more operations executed by all users. You can further refine the aliased command to search only for the current user, such as the root account.

Updated VSAN 6.0 Nested ESXi OVF Templates for 64 Nodes, All-Flash Array & Fault Domain Testing

by // 22 Comments

During the development of vSphere & VSAN 6.0, I built several new OVF templates to be able to help quickly deploy different configurations of VSAN to help test and provide feedback to Engineering. I figure I would share out these new templates as it may also benefit others in the community who wish to quickly setup VSAN 6.0 and give it spin in their own home lab or development environment using Nested ESXi.

Disclaimer: Nested ESXi is not officially supported by VMware, please use at your own risk.

One thing to be aware of when running VSAN 6.0 using Nested ESXi is that you will need a minimum of 6GB of memory for each ESXi VM. If you wish to run additional workloads on top or create larger VSAN Clusters, you may need to add additional memory on each ESXi VM. This is not really a problem for real physical hardware but can be a problem for smaller lab environments when running Nested ESXi.

Here are the four new VSAN 6.0 Nested ESXi OVF Templates:

Note: If you decide to build a 64-Node Cluster, you will need run the following ESXCLI command (reboot is required) to go beyond 32 nodes up to 64 node:

esxcli system settings advanced set -o /VSAN/goto11 -i 1

Here is a few screenshots of running VSAN 6.0 using the above OVF Templates:

vsan-6.0-nested-esxi
vsan-6.0-fault-domain