WilliamLam.com

Quick Tip - How to enable memory overcommitment in VMware Fusion?

by // 20 Comments

There was an interesting internal thread that I came across yesterday where someone was asking if VMware Fusion supported memory overcommitment like VMware Workstation? In VMware Workstation, memory overcommitment can be adjusted by going to Edit->Preferences->Memory which provides three different options as shown in the screenshot below. This setting specifies the percentage of memory that would be reserved from the physical host memory for each Virtual Machine.

vmware-fusion-memory-overcommit-0
These options map to the following % of host memory reservation:

Option % of Host Memory Reserved Per VM
Fit all virtual machine memory into reserved host RAM 100
Allow some virtual memory to be swapped 50
Allow most virtual memory to be swapped 25

Disclaimer: Be aware, that default safe guards have been put in place to ensure optimal VM performance. If you decide to change these settings and allow memory overcommitment, it can potentially degrade performance of both your VMs as well as host system. Make sure you understand the changes before applying them.

You can also specify a custom value by editing the VMware Workstation configuration file located in: C:\ProgramData\VMware\VMware Workstation\config.ini and modifying or adding the following property:

prefvmx.minVmMemPct = P

where P is the percentage of configured VM memory that should fit into the host memory. The smallest value that P can be is 1. Below is a screenshot of a 32GB VM running on a Macbook Air with latest version of Fusion (8.5.3) which only has 8GB of physical memory and the value that I had used for this demonstration is 1.

Going back to VMware Fusion, memory overcommitment is also possible but the option to configure it is not available in the VMware Fusion UI. You will need to add the above setting into the VMware Fusion configuration file located in /Library/Preferences/VMware\ Fusion/config which does not exist by default. You will need to restart Fusion/Workstation for the change to go into effect.

In addition to the change, if you do decide to overcommit your memory, it was also mentioned that you may also want to disable Hard Disk buffering for optimal performance. You can make this change in the Advanced Settings of the the VM as seen in the screenshot below.

vmware-fusion-memory-overcommit-1
You can also just add the following property to the Virtual Machine's VMX configuration file:

hard-disk.hostBuffer = "disabled"

Memory commitment can be a wonderful tool, especially for lab environments. If you combine this with SSD storage and if swapping does occur, the impact may be acceptable so that you can run a few more VMs. Thanks to Regis Duchesne & Jesse Pool for sharing this handy tidbit!

vimtop: esxtop for the VCSA 6.0

by // 6 Comments

A couple of weeks back I learned about a really cool new tool called vimtop located in the new VCSA 6.0 from fellow colleague Nick Marshall. If you have ever used esxtop before with ESXi, then you will feel right at home with vimtop which is purpose built to provide performance information and statistics about VCSA and the applications running under it. This will definitely be a handy a tool to be aware of when needing to troubleshoot performance issues or bottlenecks in the VCSA.

Disclaimer: While testing vimtop, I found that some of the command-line options are not currently functional and probably why the current version is at 0.5 with tag of "Alpha". I have been told vimtop is still in active development and I suspect Engineering wanted to get something out to customers to try out and get feedback as they continue to iterate and add more features.

To launch vimtop, you will need to SSH to a VCSA 6.0 system and type "vimtop" in either the applianceshell or in a regular bash shell.

vimtop0
At first glance, vimtop looks very similar to esxtop but you will quickly notice there are many cool new UI improvements which really makes navigating the interface much simpler. The first thing that should stand out to you is the use of colors to help improve the readability of all the metrics. You will also notice that you can quickly navigate through current list view by either scrolling up and down or side to side using the directional arrow keys. When a item is selected is also clearly highlighted which is a huge plus in my opinion when needing to troubleshoot and watch for a particular entry or stat.

Here is a screenshot selecting a specific row in vimtop, you can also do this for a column as well:

vimtop-1-up-down
There are three primary views in vimtop: Processes, Disks & Networks statistics which can be toggled using keyboard shortcuts. In fact, all navigation is performed through a series of global keyboard shortcuts similar to esxtop. There is actually a quite a few of them and you can quickly see the list by hitting the "h" key at any time for the help menu.

Here is the complete list of keyboard shortcuts for your reference

Keyboard Key Description
esc Clear existing selection and jump back to Process view
w Write the configure out the current settings goes to a configuration file located in vimtop/vimtop.xml
s Set the refresh interval (seconds)
f Display all available CPUs overview
t Display Tasks currently managed by the appliance
g Expand top 4 physical CPUs currently available to the appliance
h Help menu
u Show/Hide the unit headers
i Show/Hide the top line
o Network view
p Pause the screen
l Select a particular column
delete Remove selected column
PgUp/PgDn Select first and last row and scroll to it
- Collapse selected item
+ Expand selected item
home/end Select first and last column and scroll to it
left/right arrow Select column
up/down arrow Select row
enter Display more info about a select item
< Move selected column to the left
> Move selected column to the right
k Disk View
m Display memory overview information
n Show/Hide the name headers
c Add new column
d Add selected column in descending order or to switch column to descending order
x Select optimal column width
z Clear sort order
a Add selected column in ascending order or to switch column to ascending order
q Quit
~ Display vimtop in Back/White mode

If you are more of a visual person, I have also created a visual keyboard layout of all the vimtop commands which might be handy to print out and post on your wall. I actually got this awesome idea from one of our internal Wikis and I have created a new layout to match all the commands that are currently in vimtop.

vimtop-shortcut-keys
For each of the three views, you can also add and remove different columns just like you could with esxtop using the "c" character. You can then select or de-select columns by using the spacebar for the metrics you wish to be displayed in the current view.

add-column
I figure it would also be useful to have a table of all the metrics and their definitions as it is a bit difficult to read while in vimtop itself.

ProcessES

Metric ID Description
PID Process identifier
CMD Command name used to start the process as it is seen by the underlying system
CMDLINE The full command line of this process used during startup
NAME User readable name of the process
THREADS Number of native threads currently running in the process
%CPU (CPU Usage) Current CPU usage in percent for this process
MHZ Current CPU usage in MHz for this process
CPU Total CPU time used by the process during last measurement cycle (sum of cpu.used.system and cpu.used.user)
SYS CPU time spent by process in the system (kernel) routines
USR CPU time spent by process in the user land
%MEM (Memory Usage) Physical memory usage in percent for this process
MEM Physical (resident) memory used by this process
VIRT Total virtual memory size of this process (the complete working set including resident and swapped memory)
SHR Size of the shared code - these are any shared objects (so or DLL) loaded by the process
TEXT Code segment size of the process without any shared libraries
DATA Data segment size of the process (for managed process like JVM this includes the managed code also)
FD Total number of file descriptors opened by the process
FILS Number of all file objects opened by the process (sum of files directories and links)
FILE Number of regular files currently opened by the process
DIR Number of directories currently opened by the process
LNK Number of symbolic links currently opened by the process
DEVS Number of devices (char or block) opened by the process
CHAR Number of descriptors opened to character devices
BLCK Number of descriptors opened to block devices
CHNS Number of all communication channels opened by the process (either sockets or FIFOs)
SCKS Number of sockets (TCP|UDP|raw) currently opened by the process
FIFO Pipes (named or not) opened by the process

DiskS

Metric ID Description
DISK/PART Storage disk / partition identifier
IOS Number of I/O operations currently in progress on this disk (should go to zero)
IOTIME Milliseconds spent doing I/O operations on this disk / partition (increases for a nonzero number of I/O operations)
LAT disk / partition access latency (in milliseconds) calculated using the total amount of time spend doing I/O divided by the total amount of I/O operations done during last measurement interval
READS Number of reads issued to this disk / partition and completed successfully during last measurement interval
RDMRG Adjacent to each other reads on this disk / partition merged for efficiency
READ Number of reads per second issued to this disk / partition
RDSCTRS Number of sectors read successfully from this disk / partition during last measurement interval
WRITES Number of writes issued to this disk / partition and completed successfully during last measurement interval
WRMRG Adjacent to each other writes on this disk / partition merged for efficiency
WRITE Number of writes per second issued to this disk / partition
WRSCTRS Number of sectors wrote successfully to this disk / partition during last measurement interval

NetworkS

Metric ID Description
INTF Interface name
TRGPT Total throughput of this interface (Rx + Tx) in kilobytes
RATE The activity of this network interface in kBps
RXED Amount of data (in kilobytes) received during last measurement interval
RXRATE Rate of received data through this interface in kBps
TXED Amount of data (in kilobytes) transmitted during last measurement interval
TXRATE Rate of data transmission through this interface in kBps
RXMCAST Number of multicast packets received on this interface during last measurement interval
RXDROP Number of data rx-packets dropped during last measurement interval
TXDROP Number of data packets dropped upon transmission during last measurement interval
DROPPED Number of dropped packets through this network interface because of running out of buffers during last measurement cycle
ERRS Total number of faults (Tx and Rx) on this interface
RXERRS The sum of receive errors rx-fifo errors and rx-frame errors
TXERRS The sum of transmit errors tx-fifo errors and carrier errors
FIFOERRS FIFO overrun errors on this interface caused by host being busy to serve the NIC hardware
CLLSNS Collisions detected on the transmission medium

There is definitely a lot more to explore in vimtop, but hopefully this provides a good reference point on quickly getting started. I have to say I really like a lot of the UI enhancements to vimtop, especially the ability to select and quickly watch a particular process. Hopefully some of these enhancements can make its way into esxtop to provide the same set of functionality in the future.

Updated VSAN 6.0 Nested ESXi OVF Templates for 64 Nodes, All-Flash Array & Fault Domain Testing

by // 22 Comments

During the development of vSphere & VSAN 6.0, I built several new OVF templates to be able to help quickly deploy different configurations of VSAN to help test and provide feedback to Engineering. I figure I would share out these new templates as it may also benefit others in the community who wish to quickly setup VSAN 6.0 and give it spin in their own home lab or development environment using Nested ESXi.

Disclaimer: Nested ESXi is not officially supported by VMware, please use at your own risk.

One thing to be aware of when running VSAN 6.0 using Nested ESXi is that you will need a minimum of 6GB of memory for each ESXi VM. If you wish to run additional workloads on top or create larger VSAN Clusters, you may need to add additional memory on each ESXi VM. This is not really a problem for real physical hardware but can be a problem for smaller lab environments when running Nested ESXi.

Here are the four new VSAN 6.0 Nested ESXi OVF Templates:

Note: If you decide to build a 64-Node Cluster, you will need run the following ESXCLI command (reboot is required) to go beyond 32 nodes up to 64 node:

esxcli system settings advanced set -o /VSAN/goto11 -i 1

Here is a few screenshots of running VSAN 6.0 using the above OVF Templates:

vsan-6.0-nested-esxi
vsan-6.0-fault-domain