WilliamLam.com

How to bootstrap vCenter Server onto a single VSAN node Part 1?

by // 18 Comments

By now, I am sure you have heard about VMware Virtual SAN (VSAN) and you are probably anxious to give it a spin once the beta becomes publicly available in the very near future. I have been doing some testing in my lab with VSAN, not Nested VSAN, but on actual physical hardware. While getting started, I hit an interesting challenge given my physical hardware configuration and also this being a greenfield deployment.

Let me explain by what I mean by this. In my lab, I have three physical hosts and each contains a single SSD and single SATA drive. Each host has been provisioned with a small 5GB iSCSI boot LUN that is used to install ESXi (this could have also been another local disk or even USB/SD card). Though VSAN itself is built into the VMkernel, the management of the VSAN cluster, configurations and policies are all performed through vCenter Server. So for a greenfield deployment, you would need to first deploy a vCenter Server which would then require you to consume at least one of the local disks. This is the good ol chicken and egg problem!

In my environment, this was a problem because I only have a single SSD and SATA disk and I would not be able to setup a VSAN datastore for all three hosts at once. This meant I had to do the following steps:

  1. Create a local VMFS volume on the first ESXi host
  2. Deploy vCenter Server and then create a VSAN Cluster
  3. Add the two other ESXi host to the VSAN Cluster
  4. Storage vMotion the vCenter Server to the VSAN Datastore
  5. Destroy the local VMFS datastore on first ESXi host (existing VMFS partitions will not work with VSAN) & delete partitions
  6. Add the first ESXi host to VSAN Cluster

As you can see this can get a bit complicated and potentially error prone when needing to destroy VMFS volumes ...

I figured there had to be a better way and I was probably not going to be the only one hitting this scenario for a greenfield and even potentially for a brownfield deployments. In talking to Christian Dickmann, a Tech Lead for the VSAN project, I learned about a really cool feature of VSAN in which you can actually bootstrap vCenter Server onto a single VSAN node! This was possible due to the tight integration of VSAN within the VMkenel and best part about this solution is that it is fully SUPPORTED by VMware. From an operational perspective, this deployment workflow is much easier and intuitive than the process listed above. This also allows you to maximize the use of your hardware investment by running both your core infrastructure VMs as well as your regular workloads all on the VSAN datastore which is great for small or ROBO offices.

In my environment, I start out with a single ESXi 5.5 host which contains a single SSD and SATA disk and I create single VSAN node from that ESXi host and contribute its storage to the VSAN datastore. I then deploy a vCenter Server for which I am using the VCSA (vCenter Server Appliance) for quick and easy deployment. The default policy for VSAN is to automatically ensure there is at least one additional replica of the VM as new ESXi compute nodes join the VSAN cluster.

Once the vCenter Server is online, I can then create a vSphere Cluster and enable it with VSAN and add all three ESXi 5.5 hosts to the vSphere Cluster. This will then contribute all their storage to the VSAN datastore all while the vCenter Server is happily running. Once the other ESXi hosts join the VSAN cluster, we will automatically get replication between the other nodes to ensure our vCenter Server is replicated and of course you can change this policy.

As you can see this is much simpler setup than having to start out with an existing VMFS or even NFS datastore to initially store the vCenter Server and then create the VSAN datstore and migrate the vCenter Server. I also like how I can start deploying my infrastructure with a single ESXi host and then slowly bring in additional ESXi hosts (just make sure you do it in timely fashion as you have a SPOF until then). In part two of this article, I will go into more details on how to configure the single VSAN node and bootstrap vCenter Server. In the meantime, if you have not checked out these awesome articles by some of my VMware colleagues, I would highly recommend you give them a read, especially Cormac's awesome VSAN series!

Here is How to bootstrap vCenter Server onto a single VSAN node Part 2?

If you are interested in testing out VSAN, be sure to sign up for the beta here!

Cormac Hogan

Duncan Epping

Dave Hill

Quick Tip - Marking an HDD as SSD or SSD as HDD in ESXi

by // 9 Comments

This was a neat little trick that I picked up in one of our internal storage email distribution groups which I thought was quite interesting. Some of you may recall an article I wrote a few years back on how to trick ESXi 5 in seeing an SSD device which relied on adding an SATP rule for a particular storage device. The actual use case for this feature was that not all real SSD devices would automatically be detected by ESXi and this allowed a user to manually mark it as an SSD.

The other "non-official" use case for this feature allows a user to basically "simulate" an SSD by marking a regular HDD as an SSD and I this actually helped me test the new Host Cache (Swap-to-SSD) feature which was part of the vSphere 5 release. Recently there was a customer inquiry asking for the complete reverse, in which you could mark an SSD as an HDD. I am not sure what the use case was behind this request but I did learn it was actually possible using a similar method of adding a SATP rule to a device.

Note: If you are running Nested ESXi, a much simpler solution for simulating an SSD is to use the following trick noted here.

Before you begin, you will need to identify the storage device in which you wish to mark as an SSD or HDD. Use the following ESXCLI command to do so:

esxcli storage core device list

In the screenshot above, we can see for our device mpx.vmhba1.C0:T2:L0 shows "Is SSD" parameter as false. After running two commands below, we should then see that property change to true.

Marking HDD as SSD:

esxcli storage nmp satp rule add -s VMW_SATP_LOCAL -d mpx.vmhba1:C0:T2:L0 -o enable_ssd
esxcli storage core claiming reclaim -d mpx.vmhba1:C0:T2:L0

 

Marking SSD as HDD:

esxcli storage nmp satp rule add -s VMW_SATP_LOCAL -d mpx.vmhba1:C0:T1:L0 -o disable_ssd
esxcli storage core claiming reclaim -d mpx.vmhba1:C0:T1:L0

To perform the opposite, you simply just need to add the disable_ssd option. If you receive an error regarding a duplicate rule, you will need to first remove the SATP rule and then re-create with the appropriate option.

Another useful tidbit is that if you are running Nested Virtualization and the virtual disk of that VM is stored on an actual SSD, that virtual disk will automatically show up within the guestOS as an SSD so no additional changes are required.

Quick Tip - Correlating VMFS Datastore to Storage Device Using ESXCLI

by // 1 Comment

There was a question on Twitter this morning from AJ Kuftic on whether it is possible to display the mapping of a VMFS Datastore to its respective storage device using ESXCLI. Josh Coen beat me to the answer this morning, but yes it is possible using ESXCLI. I thought I still share this quick tip as it may not be obvious, especially when you need this information while performing a storage maintenance or troubleshooting with your storage administrator.

For those of you who are familiar with the legacy esxcfg-* commands, this information can be retrieved using the following command:

esxcfg-scsidevs -m

You can also retrieve this same information by using the following ESXCLI command (can be executed remotely as well):

esxcli storage vmfs extent list

As you can see from both screenshots, we can easily identify the name of the VMFS datastore and the specific storage device it is mapped along with other pieces of information. I prefer the ESXCLI method as it is nicely formatted along with the title header for each property.