Dave Hill recently wrote an article about running nested ESXi and a gotcha with EVC (Enhanced vMotion Compatibility). In vSphere 4.x, you could not join a nested ESXi host into a cluster with EVC enabled. With vSphere 5, there's actually a way to connect a nested ESXi 5 host to an EVC enabled cluster AND still power on 64bit nested guestOSes.
I have to thank my friend and partner in crime Tuan Duong for showing me this trick awhile back. Tuan was performing some tests using both nested and physical ESXi 5 hosts and discovered this method after a bit of tinkering. At the time, I was not sure if others would find this useful and I did not document the process.
Disclaimer: As usual, this is not officially supported by VMware, use at your own risk.
Here are the steps:
1. You must be running vSphere 5, create a nested ESXi 5 host using this article How to Enable Support for Nested 64bit & Hyper-V VMs in vSphere 5
2. Create an EVC enabled cluster or use an existing cluster with whatever baseline you would like and click on the "Current CPUID Details" in cluster settings.
3. Copy down the CPU mask flags for that particular EVC baseline, you will need this in the next step
4. Shutdown your nested ESXi 5 host and edit the VM's settings and under "Options" tab click on "CPUID Mask->Advanced". You will take the CPU mask from the above step and update the nested ESXi 5 VM to make it match
5. Go ahead and power on your nested ESXi 5 host and join it to the EVC enabled cluster you created earlier. You should not see any errors when connecting to the cluster and after that you can create a nested 64bit VM within that virtualized ESXi 5 host.
There you have it, running a nested ESXi 5 host and joined to an EVC enabled cluster! Isn't VMware technology awesome! 🙂
This is a great article. I just changed all my vESXi 5 systems that I use for vCD 1.5 over and enabled the Nehalem baseline. Worked a treat. Thanks so much for writing this William.
Michael, Glad the article was useful!
Was attempting this with EVC set to nehalem
With a 55xx and 56xx
I get an error....
An unexpected error was received from the ESX host while powering on VM vm-2803.
The processor does not support a required feature for this virtual machine. The virtual machine cannot be powered on.
Used these settings
Leaf 1 (used in Level 1 of CPU Id MASK)
eax 0000:0000:0000:0001:0000:0110:1010:0100
ebx
ecx 0000:0000:1001:1000:1110:0010:0011:1101
edx 1000:1111:1110:1011:1111:1011:1111:1111
Leaf 80000001 (Used in level 8000001 of CPU ID mask)
eax
ebx
ecx 0000:0000:0000:0000:0000:0000:0000:0001
edx 0010:1000:0001:0000:0000:1000:0000:0000
Leaf d
eax 0000:0000:0000:0000:0000:0000:0000:0000
ebx
ecx 0000:0000:0000:0000:0000:0000:0000:0000
edx 0000:0000:0000:0000:0000:0000:0000:0000
Really usefull tip, thanks william
This might be obvious to some, but I struggled until I set my EVC mode to one architecture lower. I have Sandy Bridge Xeons that the web client says support Sandy Bridg EVC, but I had to select Westmere to get this to work.
I was not able to get this to work - I have an Ivy Bridge CPU (i5-3750k). Is this supported with my CPU?
Hi William!
Sorry for coming back to such an old post, but I'm trying to setup a nested ESXi 6.5 deployment with EVC clusters and would like to know if the info in this post also applies to 6.x, not only to ESXi 5.
Could you confirm?
Thank you!