WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Hardware Options
    • Hardware Reviews
    • Lab Deployment Scripts
    • Nested Virtualization
    • Homelab Podcasts
  • VMware Nostalgia
  • Apple

How to configure SMP-FT using Nested ESXi in vSphere 6?

03.06.2015 by William Lam // 1 Comment

Symmetric Multi-Processing Fault Tolerance (SMP-FT) has been a long-awaited feature by many VMware customers. With the release of vSphere 6.0, the SMP-FT capability is now finally available and if you want to try out this new feature and see how it works from a "functional" perspective, you can easily do so by running it in a Nested ESXi environment. SMP-FT no longer uses the "record/replay" capability like its younger brother Uniprocessing Fault Tolerance (UP-FT). Instead, SMP-FT now uses a new Fast Checkpointing technique which not only improves the overall performance of its predecessor but also greatly simplifies and reduces additional configurations when running in a Nested ESXi environment.

Disclaimer: Running SMP-FT in a Nested ESXi environment does not replace or substitute actual testing of physical hardware. For any type of performance testing, please test SMP-FT using real hardware.

Requirements:

  • pESXi host running either ESXi 5.5 or 6.0
  • vCenter Server 6.0
  • 2 x Nested ESXi VMs running ESXi 6.0 (vHW9+)
  • Shared storage for the Nested ESXi VMs

Instructions:

Step 1 - Created a Nested ESXi VM using guestOS type "ESXi 5.5/6.0 or later". You will need at least 2 vCPU or greater, 4GB of memory or greater for the installation of ESXi and most importantly, a VMXNET3 network adapter. The reason a VMXNET3 adapter is required is that SMP-FT has a requirement for 10Gbit network connection and the VMXNET3 driver can simulate a 10Gbit connection for a Nested ESXi VM. For further instructions on creating a Nested ESXi VM, please take a look at this article. If you are unable to add VMXNET3 adapter, you may need to first change the guestOS type to "Other 64-bit", add the adapter and then change the guestOS type back.

smp-ft-nested-esxi-0
Step 2 - Install ESXi 6.0 on the Nested ESXi VM and ensure you also have a vCenter Server 6.0 deployed if you have not done so already and add your Nested ESXi instances to a new vSphere Cluster which has vSphere HA enabled.

Step 3 - You will need to enable both vMotion and Fault Tolerance traffic type for the VMkernel interface that you wish to run FT traffic across.

smp-ft-nested-esxi-1
Step 4 - At this point, you can create a real or dummy VM and power it on. Once you have the powered on VM, you can now enable either UP-FT or SMP-FT by right clicking and selecting "Enable Fault Tolerance".

smp-ft-nested-esxi-2
As you can see from the screenshot above, I have successfully enabled FT on a VM with 4vCPU running inside of a Nested ESXi VM, how cool is that!? Hopefully this will help you get more familiar with the new SMP-FT feature when you are ready to give it a real spin on real hardware 🙂

Note: Intel Sandy Bridge is recommended when using SMP-FT (real physical hardware) but if you have older CPUs, you enable "Legacy FT" mode by adding the following VM Advanced Setting "vm.uselegacyft" to the VM you are enabling FT on.

Categories // ESXi, Nested Virtualization, vSphere 6.0 Tags // fault tolerance, nested ft, nested virtualization, smp-ft, vm.uselegacyft, vSphere 6.0

VMware Tools is now pre-installed with Nested ESXi 6.0

02.26.2015 by William Lam // 9 Comments

I just came across this super awesome little tidbit from Core Nested ESXi Engineer Jim Mattson, that ESXi 6.0 now comes per-installed with VMware Tools when running Nested ESXi. This means you no longer have to manually install the VMware Tools for Nested ESXi but ESXi will be able to automatically detect that it is running inside of a VM and automatically startup the vmtoolsd process.

Disclaimer: Nested ESXi is not officially supported by VMware, please use at your own risk.

Screen Shot 2015-02-26 at 9.25.12 AM
This new feature of Nested ESXi is agnostic to the underlying physical ESXi version as well as the virtual hardware version. The only requirement is that the Nested ESXi is running ESXi 6.0. Talk about ease of use, this just made Nested ESXi that much cooler as if it was not already! 🙂

If you need to directly call into the vmtoolsd process for extracting OVF properties/etc. make sure you have correct library paths setup before running the vmtoolsd command, else you will get an error. To do so, run the following two commands:

export LD_LIBRARY_PATH=/usr/lib/vmware/vmtools/lib:$LD_LIBRARY_PATH
export PATH=/usr/lib/vmware/vmtools/bin:$PATH

Categories // ESXi, Nested Virtualization, vSphere 6.0 Tags // nested virtualization, vmware tools, vSphere 6.0

How to configure an All-Flash VSAN 6.0 Configuration using Nested ESXi?

02.11.2015 by William Lam // 11 Comments

There has been a great deal of interest from customers and partners for an All-Flash VSAN configuration, especially as consumer grade SSDs (eMLC) continue to drop in price and the endurance levels of these devices lasting much longer than originally expected as mentioned in this article by Duncan Epping. In fact, last year at VMworld the folks over at Micron and SanDisk built and demoed an All-Flash VSAN configuration proving this was not only cost effective but also quite performant. You can read more about the details here and here. With the announcement of vSphere 6 this week and VMware Partner Exchange taking place the same week, there was a lot of excitement on what VSAN 6.0 might bring.

One of the coolest feature in VSAN 6.0 is the support for an All-Flash configuration. The folks over at Sandisk gave a sneak peak at VMware Partner Exchange couple weeks back on what they were able to accomplish with VSAN 6.0 using an All-Flash configuration. They achieved an impressive 2 Million IOPs, for more details take a look here. I am pretty sure there are going to be plenty more partner announcements as we get closer to the GA of vSphere 6 and there will be a list of supported vendors and devices on the VMware VSAN HCL, so stay tuned.

To easily demonstrate this new feature, I will be using Nested ESXi but the process to configure an All-Flash VSAN configuration is exactly the same for a real physical hardware setup. Nested ESXi is a great learning tool to understand and be able to walk through the exact process but should not be a substituted for actual hardware testing. You will need a minimum of 3 Nested ESXi hosts and they should be configured with at least 6GB of memory or more when working with VSAN 6.0.

Disclaimer: Nested ESXi is not officially supported by VMware, please use at your own risk.

In VSAN 1.0, an All-Flash configuration was not officially supported, the only way to get this working was by "tricking" ESXi into thinking the SSD's used for capacity tier are MD's by creating claimrules using ESXCLI. Though this method had worked, VSAN itself was assuming the capacity tier of storage are regular magnetic disks and hence the operations were not really optimized for anything but magnetic disks. With VSAN 6.0, this is now different and VSAN will optimize based on whether are you using using a hybrid or an All-Flash configuration. In VSAN 6.0, there is now a new property called IsCapacityFlash that is exposed and it allows a user to specify whether an SSD is used for the write buffer or for capacity purposes.

Screen Shot 2015-02-10 at 10.01.12 PM
Step 1 - We can easily view the IsCapacityFlash property by using our handy vdq VSAN utility which has now been enhanced to include a few more properties. Run the following command to view your disks:

vdq -q

all-flash-vsan-6
From the screenshot above, we can see we have two disks eligible for VSAN and that they both are SSDs. We can also see thew new IsCapacityFlash property which is currently set to 0 for both. We will want to select one of the disk(s) and set this property to 1 to enable it for capacity use within VSAN.

Step 2 - Identity the SSD device(s) you wish to use for your capacity tier, a very simple to do this is by using the following ESXCLI snippet:

esxcli storage core device list  | grep -iE '(   Display Name: |   Size: )'

all-flash-vsan-1
We can quickly get a list of the devices and their ID along with their disk capacity. In the example above, I will be using the 8GB device for SSD capacity

Step 3 - Once you have identified the device(s) from the previous step, we now need to add a new option called enable_capacity_flash to these device(s) using ESXCLI. There are actually three methods of assigning the capacity flash tag to a device and both provide the same end result. Personally, I would go with Option 2 as it is much simpler to remember than syntax for claimrules 🙂 If you have the ESXi hosts connected to your vCenter Server, then Option 3 would be great as you can perform this step from a single location.

Option 1: ESXCLI Claim Rules

Run the following two ESXCLI commands for each device you wish to mark for SSD capacity:

esxcli storage nmp satp rule add -s VMW_SATP_LOCAL -d naa.6000c295be1e7ac4370e6512a0003edf -o enable_capacity_flash
esxcli storage core claiming reclaim -d naa.6000c295be1e7ac4370e6512a0003edf

all-flash-vsan-2
Option 2: ESXCLI using new VSAN tagging command

esxcli vsan storage tag add -d naa.6000c295be1e7ac4370e6512a0003edf -t capacityFlash

Option 3: RVC using new vsan.host_claim_disks_differently command

vsan.host_claim_disks_differently --disk naa.6000c295be1e7ac4370e6512a0003edf --claim-type capacity_flash

Step 4 - To verify the changes took effect, we can re-run the vdq -q command and we should now see our device(s) marked for SSD capacity.

all-flash-vsan-3
Step 5 - You can now create your VSAN Cluster using the vSphere Web Client as you normally would and add the ESXi host into the cluster or you can bootstrap it using ESXCLI if you are trying to run vCenter Server on top of VSAN, for more details take a look here.

One thing that I found interesting is that in the vSphere Web Client when setting up an All-Flash VSAN configuration, the SSD(s) used for capacity will still show up as "HDD". I am not sure if this is what the final UI will look like before vSphere 6.0 GA's.

all-flash-vsan-4
If you want to check the actual device type, you can always go to a specific ESXi host under Manage->Storage->Storage Devices to see get more details. If we look at our NAA* device ID, we can see that both devices are in fact SSDs.

all-flash-vsan-5
Hopefully for those of you interested in an All-Flash VSAN configuration, you can now quickly get a feel for that running VSAN 6.0 in a Nested ESXi environment. I will be publishing updated OVF templates for various types of VSAN 6.0 testing in the coming weeks so stay tune.

Categories // ESXi, Nested Virtualization, VSAN, vSphere 6.0 Tags // enable_capacity_flash, esxcli, IsCapacityFlash, Virtual SAN, VSAN, vSphere 6.0

  • « Previous Page
  • 1
  • …
  • 17
  • 18
  • 19
  • 20
  • 21
  • …
  • 26
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Automating the vSAN Data Migration Pre-check using vSAN API 06/04/2025
  • VCF 9.0 Hardware Considerations 05/30/2025
  • VMware Flings is now available in Free Downloads of Broadcom Support Portal (BSP) 05/19/2025
  • VMUG Connect 2025 - Minimal VMware Cloud Foundation (VCF) 5.x in a Box  05/15/2025
  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...