WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

Does VSAN work with Free ESXi?

07.22.2014 by William Lam // 8 Comments

I recently had to re-provision one of my VSAN lab environments using my recently shared ESXi 5.5 VSAN Kickstart. I usually specify a license key within the Kickstart so I do not have to license the ESXi host later. This actually got me wondering on whether VSAN would in fact work with Free ESXi aka vSphere Hypevisor? Being a curious person, I of course had to test this in the lab 🙂

Needless to say, if you want to properly evaluate or use VSAN in production, you should go through the supported method of using vCenter Server as it provides a simple and intuitive management interface for VSAN. More importantly, having the ability to create individual VM Storage Policies that can be applied on a per VMDK basis based on SLA's for your given application or Virtual Machine.

Disclaimer: This is not officially supported by VMware and running ESXi without a VSAN license is against VMware's EULA.

Since we do not have a vCenter Server, we will need to be able to fully configure VSAN without it. Luckily, we know of a way to "bootstraping" VSAN onto an ESXi host without vCenter Server and I will be leveraging that blog post to test this scenario with Free ESXi.

Prerequisite:

  • 3 ESXi 5.5 hosts already installed and licensed with vSphere Hypervisor (Free ESXi) License
  • SSH Enabled

Step 1 - SSH to the first ESXi host and run the following ESXCLI command to create a VSAN Cluster:

esxcli vsan cluster join -u $(python -c 'import uuid; print str(uuid.uuid4());')

configure-vsan-for-free-esxi-0
Step 2 - Run the following ESXCLI command to make a note of the VSAN Cluster UUID (highlighted in green in the screenshot above) which will be needed later:

esxcli vsan cluster get

Step 3 - Enable VSAN Traffic for VMkernel interface you plan on using for VSAN traffic by running the following ESXCLI command:

esxcli vsan network ipv4 add -i vmk0

Step 4 - Run the following command to view a list of disks that are eligible for use with VSAN. You will need a minimum of 1xSSD and 1xMD

vdq -q

configure-vsan-for-free-esxi-1
Step 5 - Using the information from vdq, we will now create our VSAN Disk Group which will contain the SSD/MD's to be used for VSAN. Use the following ESXCLI command and substituting in the SSD/MD Names (please refer to the screenshot above for an example):

esxcli vsan storage add -s [SSD] -d [MD]

Step 6 - To ensure you have properly configured a VSAN Disk Group, you can run the following ESXCLI command to confirm:

esxcli vsan storage list

configure-vsan-for-free-esxi-2
At this point, we now have a single ESXi host configured with VSAN Datastore, we can also confirm this by running the following ESXCLI command:

esxcli storage filesystem list

configure-vsan-for-free-esxi-3
Step 7 - Repeat Steps 3-6 on the remainder two ESXi hosts

Step 8 - Finally, we now need to join the remainder ESXi hosts to the VSAN Cluster. We will need the VSAN Cluster UUID that we recorded earlier and specify that in the following ESXCLI command on each of the remainder ESXi hosts:

esxcli vsan cluster join -u [VSAN-CLUSTER-UUID]

If we now login to all of our ESXi hosts using the vSphere C# Client, we will see a common VSAN Datastore that is shared among the three ESXi hosts. To prove that that VSAN is in fact working, we can create a Virtual Machine and ensure we can power it on as seen in the screenshot below. By default, VSAN has a "Default" policy which defines FTT (Number of host failures to tolerate) set to 1 and assuming you have at least 3 ESXi hosts, all Virtual Machines will be protected by default.

configure-vsan-for-free-esxi-4
Even though you can run VSAN using Free ESXi and leveraging the default VM Storage Policy that is built into VSAN for protecting Virtual Machines, you are only exercising a tiny portion of the potential that VSAN can bring when consuming it through vCenter Server. As mentioned earlier, you will not have the ability to create specific VM Storage Policies and assign them based on the specific SLAs and be able to easily monitor their compliance and remediation. The management of VSAN Cluster for adding additional capacity or serviceability is also quite limited without vCenter Server, though it can be definitely be done it is much easier with just a couple of clicks in the vSphere Web Client or a simple API call.

Categories // ESXCLI, ESXi, VSAN, vSphere 5.5 Tags // ESXi 5.5, free esxi, VSAN, vsanDa, vSphere 5.5, vsphere hypervisor

ESXi 5.5 Kickstart script for setting up VSAN

07.21.2014 by William Lam // 12 Comments

In my lab, when I need to provision a new or rebuild an existing ESXi host, I still prefer to use the true and tried method of an unattended/scripted installation also known as Kickstart. Below is a handy ESXi 5.5 Kickstart that I have been using to setup a basic VSAN environment. I figure this might come in handy for anyone looking to automate their ESXi 5.5 deployment and include some of the VSAN configurations like creating a VSAN Disk Group or enabling VSAN Traffic type on a particular VMkernel interface. For more details about this Kickstart, refer to the bottom of the file where I break down the file in more detail.

# Sample kickstart for ESXi 5.5 configuring VSAN Disk Groups
# William Lam
# www.virtuallyghetto.com
#########################################

accepteula
install --firstdisk --overwritevmfs
rootpw vmware123
reboot

%include /tmp/networkconfig
%pre --interpreter=busybox

# extract network info from bootup
VMK_INT="vmk0"
VMK_LINE=$(localcli network ip interface ipv4 get | grep "${VMK_INT}")
IPADDR=$(echo "${VMK_LINE}" | awk '{print $2}')
NETMASK=$(echo "${VMK_LINE}" | awk '{print $3}')
GATEWAY=$(esxcfg-route | awk '{print $5}')
DNS="172.30.0.100"
HOSTNAME=$(nslookup "${IPADDR}" "${DNS}" | grep Address | grep "${IPADDR}" | awk '{print $4}')

echo "network --bootproto=static --addvmportgroup=true --device=vmnic0 --ip=${IPADDR} --netmask=${NETMASK} --gateway=${GATEWAY} --nameserver=${DNS} --hostname=${HOSTNAME}" > /tmp/networkconfig

%firstboot --interpreter=busybox

vsan_syslog_key = "VSAN-KS"

logger $vsan_syslog_key " Enabling & Starting SSH"
vim-cmd hostsvc/enable_ssh
vim-cmd hostsvc/start_ssh

logger $vsan_syslog_key " Enabling & Starting ESXi Shell"
vim-cmd hostsvc/enable_esx_shell
vim-cmd hostsvc/start_esx_shell

logger $vsan_syslog_key " Suppressing ESXi Shell Warning"
esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1

logger $vsan_syslog_key " Reconfiguring VSAN Default Policy"
esxcli vsan policy setdefault -c vdisk -p "((\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i1))"
esxcli vsan policy setdefault -c vmnamespace -p "((\"hostFailuresToTolerate\" i1) (\"forceProvisioning\" i1))"

logger $vsan_syslog_key "Enabling VSAN Traffic on vmk0"
esxcli vsan network ipv4 add -i vmk0

# assign license
vim-cmd vimsvc/license --set AAAAA-BBBBB-CCCCC-DDDDD-EEEEE

%firstboot --interpreter=python

import commands, os, uuid, syslog

vsan_syslog_key = "VSAN-KS"
debug = False

# Build VSAN Disk Group command based on vdq -q output
def createVsanDiskGroup():
	vdqoutput = eval(commands.getoutput("/sbin/vdq -q"))
	md = []
	ssd = ''
	for i in vdqoutput:
		if i['State'] == 'Eligible for use by VSAN':
			if i['Reason'] == 'Non-local disk':
				syslog.syslog(vsan_syslog_key + " Setting enable_local and reclaiming " + i['Name'])
				if debug == False:
					os.system("esxcli storage nmp satp rule add -s VMW_SATP_LOCAL -o enable_local -d " + i['Name'])
					os.system("esxcli storage core claiming reclaim -d " + i['Name'])
			if i['IsSSD'] == '1':
				ssd = i['Name']
			else:
				md.append(i['Name'])

	diskgroup_cmd = 'esxcli vsan storage add -s ' + ssd + ' -d ' + ' -d '.join(md)
	syslog.syslog(vsan_syslog_key + " Creating VSAN Disk Group using SSD: " + ssd +  " MD: " + ', '.join(md))
	if debug == False:
		os.system(diskgroup_cmd)

# Create VSAN Cluster (required only for first ESXi node)
def createVsanCluster():
	# generate UUID for VSAN Cluster
	vsan_uuid = str(uuid.uuid4())

	syslog.syslog(vsan_syslog_key + " Creating VSAN Cluster using UUID: " + vsan_uuid)
	if debug == False:
		os.system("esxcli vsan cluster join -u " + vsan_uuid)

createVsanDiskGroup()
createVsanCluster()

If you would like to see more details on creating ESXi Kickstart, make sure to check out my ESXi 4.x & 5.x examples here.

Line 6-9 This is generic Kickstart configurations specifying EULA, how to install, password, etc. You can refer to VMware's scripted install documentation.

Line 11-25 This extracts the DHCP IP Address (static allocation) and re-creates the network configuration in Line 25 for statically assigning the IP Address to the ESXi host

Line 27 This starts the firstboot script and assumes "Busybox" as the interpreter which means basic shell commands

Line 30 I create a custom key which will be logged in syslog for our installation

Line 32-41 Basic ESXi configurations leveraging vim-cmd and ESXCLI

Line 43-45 Configure the VSAN default storage policy, please refer to this article for more details.

Line 47-38 Configure the VSAN Traffic type on vmk0

Line 35 This starts a second firstboot script, but now using "Python"

Line 50-51 Assign a license to ESXi host

Line 53 Importing the appropriate libraries that will be used in the Python script

Line 58 Using the same custom key that I created earlier for logging to syslog

Line 61-81 A method for creating VSAN Disk Group by inspecting vdq CLI and marking disks as local

Line 83-90 A method for creating VSAN Cluster, please refer to this article for more details.

Line 92-93 Invoking the two Python methods. You can either create a custom Kickstart for your "first" ESXi node if you decide to bootstrap your VSAN Cluster onto a single ESXi host. You can also use custom boot options to specify whether the ESXi host being provisioned is the first or additional nodes. This topic is a bit advanced, but if you are interested, take a look at this article here.

Categories // Automation, ESXCLI, ESXi, VSAN, vSphere, vSphere 5.5 Tags // ESXi 5.5, kickstart, ks.cfg, VSAN, vSphere 5.5

Does reinstalling ESXi with an existing VSAN Datastore wipe your data?

07.11.2014 by William Lam // 16 Comments

The quick answer is no, reinstalling ESXi on a host that contained VSAN data will not be touched or destroyed. This, of course, assumes you do not touch any of the disks that contains your VSAN data. To further assist administrators, you can also see which disks are claimed by VSAN during the the disk selection of a reinstall/upgrade of ESXi, so the proper disk is selected.

reinstall-esxi-with-esxistig-vsan-datastore-0
This to me is a very important fact and probably a very conscious design decision by the VSAN Engineers to ensure that you do not accidentally wipe your data because ESXi needed to be reinstalled. The topic has been one I have been wanting to write about for awhile but just never found the time. The reason I am bringing this up is that I saw an interesting tweet the other day from fellow VMware colleague, Geordy Korte who works over in the NSBU that had the following request:

NEED HELP: Anyone have any clue on how to recreate a #VSAN using esxcli and not wiping the data. My Vcenter is on the VSAN. RT PLZ

— Geordy Korte (@gekort) July 2, 2014

Geordy needed to reinstall ESXi but he did not want to lose any of the VSAN data residing on the disks within the ESXi host which also contained his vCenter Server among other Virtual Machines. Fortunately, reinstalling ESXi is a very safe operation and it will not touch any of the VSAN partitions. In fact, after the installation, if ESXi detects there are partitions that contains VSAN data, it will automatically claim the devices for you. You may have even seen this as part of the ESXi boot process:

reinstall-esxi-with-esxistig-vsan-datastore-0.1
Going back to Geordy's original question, how do you re-create the VSAN Datastore after reinstalling ESXi? Well, the process is actually pretty straight forward and is quite similar to how you bootstrap vCenter Server onto a VSAN Datastore. To demonstrate this process, I decided to take a fully functional 3-Node VSAN setup that contained a Virtual Machine and reinstall ESXi on all three nodes. I will now take you through the process of restoring the VSAN Datastore and at the end I will also talk about an alternative approach depending on the situation you might be in.

Step 1 - You can easily check whether you have an existing VSAN Disk Group by running the following ESXCLI command:

esxcli vsan storage list

reinstall-esxi-with-esxistig-vsan-datastore-1
Step 2 - You will need to enable VSAN traffic type for the VMkernel interface you wish to run VSAN on by running the following command and specifying the interface (this will need to be done on ALL ESXi hosts):

esxcli vsan network ipv4 add -i vmk0

Step 3 - You will need to re-create a new VSAN Cluster and you can do so by running the following command which will generate a UUID and create a VSAN Cluster from that:

esxcli vsan cluster join -u $(python -c 'import uuid; print(str(uuid.uuid4()))')

reinstall-esxi-with-esxistig-vsan-datastore-2
Step 4 - Once the VSAN Cluster has been created, you will need to make a note of the UUID that was generated so you can join the remaining ESXi hosts to the same VSAN Cluster. To do so, you will need to run the following command:

esxcli vsan cluster get

You will need to look for "Sub-Cluster UUID" property as seen in the screenshot above highlighted in green.

Step 5 - On the remainder ESXi hosts, you can now join the the VSAN Cluster using the following command and specifying the UUID from the previous step:

esxcli vsan cluster join -u [UUID]

reinstall-esxi-with-esxistig-vsan-datastore-3
Step 6 - Once all ESXi hosts have re-joined the VSAN Cluster, you can now go into the VSAN Datastore and re-register all the Virtual Machines using either the vSphere C# Client or from the CLI on the ESXI Shell.

reinstall-esxi-with-esxistig-vsan-datastore-5

reinstall-esxi-with-esxistig-vsan-datastore-4
As you can see from the screenshot above, I was able to recover my Virtual Machine and there was no data loss!

If you recall earlier, I also mentioned there was an alternative approach that you could take depending on the situation. If you need to deploy a new vCenter Server for what ever reason, you can easily restore your existing VSAN Cluster by simply creating a new VSAN Cluster in the new vCenter Server. You would then add each ESXi hosts to the VSAN Cluster and the VSAN Datastore and its contents will automatically be restored without any issues. Once you have your VSAN Datastore, you can then re-create the missing VM Storage Policies and re-apply them to the respective Virtual Machines.

In this article I have described two potential scenarios when working with VSAN and in both cases you can safely recover your data without any problems. This is one of the things I have come to appreciate about VSAN is the amount of engineering effort to make it super simple to use but also very resilient!

Categories // ESXi, VSAN, vSphere 5.5 Tags // ESXi 5.5, VSAN, vsanDatastore, vSphere 5.5

  • « Previous Page
  • 1
  • …
  • 37
  • 38
  • 39
  • 40
  • 41
  • …
  • 53
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...