I noticed there were a few folks having some difficulties enabling Nested ESXi (VHV Virtual Hardware Virtualization) in the latest release of ESXi 5.1 and I thought I share some additional info and tips on troubleshooting your setup in case you are running into similar problems.
*** DISCLAIMER **** This is not officially supported by VMware, do not bother asking if it is supported or calling into VMware support for details or help.
If you wish to run nested ESXi or other hypervisors on ESXi 5.1 and run 32-bit nested virtual machines, you must meet the following hardware requirement:
- CPU supporting Intel VT-x or AMD-V
If you wish to run nested 64-bit virtual machines in your nested ESXi or other hypervisors, in addition to the requirement above, you must also meet the following hardware requirement:
- CPU supporting Intel EPT or AMD RVI
If you only meet the first criteria, you CAN still install nested ESXi or other hypervisors on ESXi 5.1, BUT you will only be able to run 32-bit nested virtual machines. When you create your virtual machine shell using the new vSphere Web Client, in the expanded CPU view, the "Hardware Virtualization" box will be grayed out. This is expected as you do not have full support for VHV, but you can still continue with your installation of ESXi or other hypervisors.
In ESXi 5.0, you may have been able to run 64-bit nested virtual machines without EPT/RVI support but performance was extremely poor. With ESXi 5.1, VHV now requires EPT/RVI.
Note: During the installation of ESXi, you may see the following message "No Hardware Virtualization Support", you can just ignore it.
If you are using sites such as Intel's ark.intel.com to check your CPU requirements, be aware that it is COMMON even for the hardware vendors to publish incorrect information about their websites. However, there is a quick way you can validate on your ESXi host whether you have full VHV support.
In vSphere 5.1, there is a new capability property called nestedHVSupported which specifies whether your physical ESXi 5.1 host has full VHV support. This property will only be true IF your CPU has both Intel-VT+EPT or AMD-V+RVI. A quick and easy way to validate this is using the vSphere MOB to retrieve the value.
To check nestedHVSupported property, please enter the following into a web browser (substitute the IP Address/hostname of your ESXi host):
https://himalaya.primp-industries.com/mob/?moid=ha-host&doPath=capability
After you login, search for the nestedHVSupported property on the page and you should see a value of either true or false. As mentioned earlier, if it is false, you might still be able to install nested ESXi or other hypervisors but you will not be able to run nested 64-bit virtual machines. I would also recommend taking a look at your system BIOS to ensure things like Intel-VT/EPT and AMD-V/RVI are enabled and sometimes it might just be as simple as a BIOS upgrade (you can always confirm by contacting the hardware vendor if you have further questions).
For proper networking connectivity, also ensure that either your standard vSwitch or Distributed Virtual Switch has both promiscuous mode and forged transmit enabled either globally on the portgroup or distributed portgroup your nested ESXi hosts are connected to.
Additional Resources:
- How to Enable Nested ESXi & Other Hypervisors in vSphere 5.1
- How to Enable Nested ESXi & Other Hypervisors in vCloud Director 5.1
Awesome post. My Xeon 5355 supposedly supports VT-x/EPT and I had no issue in 5.0, but the MOB stated false. Is EPT a new requirement, or was this also used in 5.0?
@Jason Ruiz,
EPT has always been a requirement to run nested 64-bit guest VMS. If you have VT-x, you can install nested ESXi and run nested 32-bit guests which is same behavior in 5.0
That's what I had thought, but it looks like ESXi 5.1 doesn't see this anymore, as it worked previously.
@Jason Ruiz,
Please re-read the article, if you only have VT-x, you CAN STILL install ESXi AND run nested 32-bit guest VMs (the check box for VHV is only required if you meet full requirements VT-x+EPT or AMD-V+RVI and you wish to enable nested 64-bit guest VMs).
Sorry if I didn't explain this clearly enough, but I could previously run 64bit nested VM's and the check box is grayed out. So if this had worked previously I should have a processor with both requirements.
@Jason Ruiz,
Ah gotcha. Yes, EPT is now required for VHV support in 5.1 and specifically to run nested 64-bit guest VMs. My understanding is this may have worked in 5.0 (EPT was not a requirement, just VT-x) but the performance was extremely poor w/o EPT. If the NestedHVSupport property shows false for your system, you most likely don't have EPT
Same thing here - 64bit nested worked fine with 5.0, not so with 5.1. It's a lab, so I can live with it, but it makes no sense. The option for both required elements (VT-x and EPT) are present and enabled in the BIOS for the two X5460 proc's.
@Anonymous
If the NestedHVSupport property shows false for your system, you most likely lack EPT as the necessary bits are auto-detected. In 5.1, EPT is required and though it may have worked in 5.0, w/o EPT, the performance was extremely poor. As you know this is a totally unsupported feature, if you need to nest 64-bit VMs, you can still run 5.0 on your physical ESXi host and then run vESXi 5.1 as a workaround
Struggling to get hyper-v installed / running now. I had a hyper-v 2008 vm on my previous 5.0 vcenter installation, which worked to the point of at least being able to power on a vm in hyper-v... Since I upgraded - actually lets just day updated instead - to 5.1, I have been unable to power on the guest vm in hyper-v. I changed the vhv.allow to enable on the host, and after I did that, the hyper-v vm itself will no longer power on, and every vm I tried to power on asked if I wanted to continue as hardware visualisation is not supported, or something... So I removed this line and enabled on a per-vm level... Still no joy, a combination of either being asked a question, or just point blank refusing to power on.. So called it quits, installed a hyper-v 2012 vm, which again booted ok, I installed the hyper-v feature, set up first vm and again could not power it on, was some error like, ' hypervisor not running'... Or similar... Tried various vmx editing to add / remove the vhv.enable line, as soon as I change it, I get asked the question, and now I've removed the line, the vm won't boot again...
So what do I need to do to make this work again?? I know I have vt-x, but not ept - server is power edge 1950 ii, cpus are xeon 5150... I know it should work, its just going bad somewhere! Any advice? Cheers
This worked fine, although, after install 2 nested esxi 5.1.0a server on top of esxi 5.1.0a it causes the physical host to freeze.
It seems to run for about 20 minutes - 2 hours and it just freezes
There is no purple screen it just locks up...I've checked log files but there's nothing in there...maybe i'm looking in the wrong ones?
This is a white box amd fx 8320 8 core cpu 32 gb ram.
Works fine when nested esxihost vms are not powered on.
There was also a problem where I was trying to kickstart some vm's within the nested ESXI hosts. They would contact the dhcp server get an dhcp lease assigned but the nested vm would never get the ip settings it would fail with : arp timeout several times then just sit there.
Although the nested esxi hosts would be able to talk to the network without problems.
Wierd...any suggestions ? 🙂
I'm planing to create a VMware vSphere 5.1 cluster with cheap hardware. If all the hardware is compatible would I be able to run nested vms under vSphere 5.1 with the
Intel G1610 CPU - http://ark.intel.com/products/71072/Intel-Celeron-Processor-G1610-2M-Cache-2_60-GHz?wapkw=intel+g1610
It has
Intel® VT-x with Extended Page Tables (EPT)
but no VT-d.
AFAIK the second one is only relevant if I want to include PCIe cards into VMS? Right?
Would I be fine with such a cheap processor if all other properties (mainboard, ...) were compatible?
Greetings and thanks
jb
hi
to deploy vCloud Director w'll need VMware DRS , VMware Distributed Switch and dvFilter licensed by VMware vSphere Enterprise License
but i didn't find a free vSphere Enterprise license, I find the free licensing of products which it's composed:VMware ESXi 5.1 Update 1 Installable,VMware vCenter Server 5.1 Update 1 and modules.....so can i deploy vCloud Director using these licenses ?
Thanks
where are you belong
This comment has been removed by the author.
This comment has been removed by the author.
William, thanks for all the great posts and resources!
I am running 5.5 on hardware that supports VT-x, but not EPT. I am able to run vESXi 5.5 with 32 bit nested VMs, but I am not able to get Hyper V to install on a Server 2008 VM.
On my pHost:
nestedHVSupported = FALSE (because I cannot support EPT)
“Expose Hardware assisted virtualization in the guest OS” is greyed out as well.
Has anyone been able to fool Windows into allowing me to install Hyper V, even though I would only be able to run 32 bit nested VMs?
-Dave