WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / Uncategorized / How to Enable Nested ESXi & Other Hypervisors in vSphere 5.1

How to Enable Nested ESXi & Other Hypervisors in vSphere 5.1

08.29.2012 by William Lam // 88 Comments

There are a ton of new features with the latest release of vSphere 5.1, but the one "unsupported" feature I always test first is "Nested Virtualization" (aka Nested ESXi) and with the latest release, it seems to have gotten even better. You will still need to have the same physical CPU prerequisites as you did in the past to run "Nested Virtualization" as well as nesting 64-bit VMs.

  • Intel VT-x or AMD-V is required for running "Nested Virtualization" which supports nested 32-bit VMs
  • Intel EPT or AMD RVI is required for running nested 64-bit VMs.

A quick way to verify whether your CPU truly supports both Intel-VT+EPT or AMD-V+RVI, you can paste the following into a browser:  

https://[your-esxi-host-ip-address]/mob/?moid=ha-host&doPath=capability

You will need to login with your root credentials and then look for the "nestedHVSupported" property and if it states false, it means you maybe able to install nested ESXi or other hypervisor, but you will not be able to run nested 64-bit VMs, only 32-bit VMs, assuming you have either Intel-VT or AMD-V support on your CPUs.

For more details, take a look at this article for troubleshooting: Having Difficulties Enabling Nested ESXi in vSphere 5.1?

Disclaimer: This is not officially supported by VMware, please use at your own risk.

There are some changes with Nested Virtualization in vSphere 5.1 also officially known as VHV (Virtual Hardware-Assisted Virtualization). If you are using vSphere 5.0 to run Nested ESXi or other nested Hypervisors, then please take a look at the instructions in this article. With vSphere 5.1, there have been a few minor changes to enable VHV.

  1. The new Virtual Hardware 9 compatibility will be required when creating your nested ESXi VM, Virtual Hardware 8 will not work if you are running ESXi 5.1 on your physical host. You will still need to enable promiscuous mode on the portgroup that will be used for your nested ESXi VM for network connectivity.
  2. vhv.allow = "true" is no longer valid for ESXi 5.1 to enable VHV. A new parameter has been introduced called vhv.enable = "true" that is now defined on a per VM basis to provide finer granularity of VHV support. This also allows for better portability between VMware's hosted products such as VMware Fusion and Workstation as they also support the vhv.enable parameter.
  3. You can now enable VHV on a per VM basis and using the new vSphere Web Client which basically adds the vhv.enable = "true" parameter to the VM's .VMX configuration file.

Note: You can run a nested ESXi 5.1 VM on top of a physical ESXi 5.0 host, just follow the instructions here.

Enabling VHV (Virtual Hardware-Assisted Virtualization)

Step 1 - Create a new Virtual Hardware 9 Virtual Machine using the new vSphere Web Client that's available with vCenter Server 5.1.

Step 2 - Select "Linux" as the guestOS Family and "Other Linux (64-bit)" as the guestOS Version.

Step 3 - During the customize hardware wizard, expand the "CPU" section and select "Hardware Virtualization" box to enable VHV.

Note: If this box is grayed out, it means that your physical CPU does not supported Intel VT-x + EPT or AMD-V + RVI which is required to run VHV OR that you are not using Virtual Hardware 9. If your CPU only supports Intel-VT or AMD-V, then you can still install nested ESXi, but you will only be able to run nested 32-bit VMs and not nested 64-bit VMs.

Step 4 - It is still recommended that you change the guestOS Version to VMware ESXi 5.x after you have created the VM shell, as there are some special settings that are applied automatically. Unfortunately with the new vSphere Web Client, you will not be able to modify the guestOS after creation, so you will need to use the C# Client OR manually go into the .VMX and update guestOS = "vmkernel5"

Now you are ready to install nested ESXi VMs as well as run nested 64-bit VMs within.

If you have followed my previous article about How to Enable Support for Nested 64bit & Hyper-V VMs in vSphere 5 you may recall a diagram about the levels of "Inception" that can be performed with nested ESXi. That is, the number of times you could nest ESXi and still have it be in a "functional" state. With vSphere 5.0, the limit that I was able to push was 2 levels of nested ESXi. With latest release of vSphere 5.1, I have been able to push that limit now to an extraordinary 3 levels of inception!

You might ask why would someone want to do this ... well I don't have a good answer other than ... because I can? 😉 VHV is one of the coolest "unsupported" feature in my books and I'm glad it is working beyond what it was designed for.

For proper networking connectivity, also ensure that either your standard vSwitch or Distributed Virtual Switch has both promiscuous mode and forged transmit enabled either globally on the portgroup or distributed portgroup your nested ESXi hosts are connected to.

Nesting "Other" Hypervisors

For those of you who feel inclined to run other hypervisors such as Hyper-V, you can do so with latest release of ESXi 5.1. The process if very straight forward just like running nested ESXi host.

Step 1 - Create a Virtual Hardware 9 VM and select the appropriate guestOS. In this example, I selected Windows Server 2012 (64-bit) as the guestOS version.

Step 2 - Enable VHV under the CPU section if you wish to create and run nested 64-bit VMs under Hyper-V

Step 3 - You will need to add one additional .vmx parameter which tells the underlying guestOS (Hyper-V) that it is not running as a virtual guest which in fact it really is. The parameter is hypervisor.cpuid.v0 = FALSE

More from my site

  • Having Difficulties Enabling Nested ESXi in vSphere 5.1?
  • How to Enable Nested ESXi & Other Hypervisors in vCloud Director 5.1
  • Nested Virtualization APIs For vSphere & vCloud Director 5.1
  • Nested Virtualization Resources
  • Nested ESXi 5.1 Supports VMXNET3 Network Adapter Type

Categories // Uncategorized Tags // esxi5.1, hyper-v, nested, vesxi, vhv, vSphere 5.1

Comments

  1. Unknown says

    09/04/2012 at 8:14 am

    Great post William as always

    Reply
  2. Unknown says

    09/10/2012 at 7:06 pm

    One thing to note, don't do the vhv.enable on the physical host if you plan on running VSE on it. It will complain that it doesn't support Virtual Hardware-Assisted Virtualization. Just enable it per VM.

    Reply
    • William says

      09/28/2012 at 8:41 pm

      It's actually best to enable VHV on a per VM basis, that way you won't hit this problem due to the virtual hardware version differences in the environment.

      Reply
  3. Ram says

    09/17/2012 at 5:53 pm

    Note: If this box is grayed out, it means that your physical CPU does not supported Intel VT-x or AMD-V which is required to run VHV OR that you are not using Virtual Hardware 9.

    I cannot understand above part.

    I have enabled Intel-VT in BIOS on PowerEdge 1950 server. I added vhv.enable = "TRUE" as well - yet, I cannot select "Hardware Virtualization" box to enable VHV when creating virtual machine using web client. Where am, I wrong. Appreciate your input.

    Reply
  4. Anonymous says

    09/19/2012 at 10:02 am

    Hi Ram. Same situation here. We have two VT-supported processors, one 5130 and one E5335. The latter works, the first doesn't. What processor do you have?

    Reply
    • William says

      09/28/2012 at 8:42 pm

      I've updated the article, take a look at the very top and search for "nestedHVSupported"

      Reply
  5. Anonymous says

    09/19/2012 at 4:39 pm

    Hey Ram,

    Same thing here Dell Precision T5400 with dual X5460, "hardware Virtualization" is grayed out, and when I start a VM I get a VM Message stating "Virtualized Intel VT-x/EPT is not supported on this platform".

    Output of esxcfg-info shows supported and enabled

    ~ # esxcfg-info | grep "HV Support"
    |----HV Support............................................3
    |----World Command Line.................................grep HV Support

    Worked fine with 4.1 and 5 -

    Reply
    • William says

      09/28/2012 at 8:42 pm

      I've updated the article, take a look at the very top and search for "nestedHVSupported"

      Reply
  6. whocarez says

    09/20/2012 at 4:47 pm

    Hi,

    Same thing here. Hp dl380g5 e5345 and dl380g6 l5520 both show

    ~ # esxcfg-info | grep "HV Support"
    |----HV Support............................................3
    |----World Command Line.................................grep HV Support

    But no luck getting it enabled.

    Reply
    • William says

      09/28/2012 at 8:42 pm

      I've updated the article, take a look at the very top and search for "nestedHVSupported"

      Reply
  7. Anonymous says

    09/22/2012 at 7:02 pm

    Same thing for me too!

    HP DL380 G5 with 1 of Dual-Core 5160 Intel CPU
    In Bios, "Intel Virtualisation" is enabled.
    But "Hardware Virtualisation" on vSphere Web Client 5.1 is greyed out for any newly created HW V.9 Virtual Machine.

    Reply
    • William says

      09/28/2012 at 8:42 pm

      I've updated the article, take a look at the very top and search for "nestedHVSupported"

      Reply
  8. Anonymous says

    09/23/2012 at 1:52 am

    Having the same problem on Dell 1950 -
    In Bios, "Intel Virtualisation" is enabled.
    But "Hardware Virtualisation" on vSphere Web Client 5.1 is greyed out for any newly created HW V.9 Virtual Machine.
    Used to work ok on vSphere 5.0.

    Reply
  9. SoAct says

    09/23/2012 at 6:53 pm

    Yes thank you
    You just need to create the VM in the webclient and then
    The option is not grayed out.
    If you create the vm in the viclient and then goto the webclient you can not select it.
    Still strange that there is a difference between the web and
    Windows client. But on the other hand probably going away from Ms windows dependency

    Regards Toine

    Reply
  10. Anonymous says

    09/24/2012 at 12:21 am

    Are you sure about last entry, just tried creating VM via Web Client, still greyed out. on Dell 1950 Host.
    There some inconsistency here. Works for some and not others.

    Reply
  11. Paul Braren says

    09/24/2012 at 9:11 am

    I put together some screenshots.
    Yes, ESXi 5.1 can run Microsoft Hyper-V Server 2012 VMs!
    http://tinkertry.com/esxi-5-1-running-hyper-v-server-2012

    Reply
  12. Ram says

    09/25/2012 at 6:39 pm

    I am also having same problem here. I was able to create virtual esxi5.1 - Created vm server 2012 through web client - yet Hardware Virtualization box is grayed out.

    I even checked intel site for xeon e5410 processor with regard to EPT - It is available but, I don't see an option to enable it in BIOS.

    Even checked esxcfg-info command to check the status of HV Support
    The output is HV Support...3
    World Command Line Grep HV Support

    I even ran coreinfo -v to check the status of EPT. It shows a DASH which means the processor does not support EPT. If you have * then the processor support EPT.

    Coreinfo -v output

    Intel(R) Xeon(R) CPU E5410 @ 2.33GHz
    Intel64 Family 6 Model 23 Stepping 6, GenuineIntel
    HYPERVISOR * Hypervisor is present
    VMX - Supports Intel hardware-assisted virtualization
    EPT - Supports Intel extended page tables (SLAT)

    Reply
    • Seymour Harding says

      09/27/2012 at 1:41 pm

      Same issues here:

      Dell PowerEdge 2950III with dual Xeon E5430's
      Dell Precision T7400 with dual Xeon E5420's

      both have latest BIOS revisions, all virtualisation switches enabled in BIOS, and Intel ARK clearly states that these CPU's support VT-X with EPT.

      Not a happy bunny at the moment 🙁

      Reply
    • William says

      09/27/2012 at 3:06 pm

      Ram/Seymour,

      Can you guys provide a bit more details on how you're configuring your nested ESXi VMs? Which method are you using? Please provide the step by step that you've walked through

      Reply
  13. Ram says

    09/28/2012 at 12:38 pm

    Completed steps on physical pe 1950 esxi 5.1

    1. Enabled vhv on physical esxi 5.1
    2. Enabled promiscuous mode to accept on physical esxi 5.1
    3. Created Virtual machine using web client 5.1 - Guest OS - Other Linux x64.
    4. At Customize hardware - Expanded CPU to select Hardware Virtualization - Greyed out.

    What is going on, I can't understand.

    Reply
    • William says

      09/28/2012 at 5:33 pm

      Can you open a browser and paste the following URL: https://[your-esxi-host-ip-address]/mob/?moid=ha-host&doPath=capability

      Login using the root credentials and then look for "nestedHVSupported" and see what it says. This basically lists the capabilities of the host and this should tell you whether vSphere believes you can run VHV.

      Also is your hardware on the HCL for ESXi 5.1?

      Reply
  14. JBF says

    09/28/2012 at 6:34 pm

    Same problem here. No option for Hardware Virtulisation. I'm running an E3-1240 which should have the extensions required.

    Reply
    • JBF says

      09/28/2012 at 10:59 pm

      This comment has been removed by the author.

      Reply
    • JBF says

      09/28/2012 at 11:02 pm

      It looks like my CPU does support nesting, so why can't I install a Hyper-V role on Windows Server 2012? I followed the instructions as above. No option for VHV when creating the VM via the web GUI.

      https://dl.dropbox.com/u/6104595/Screen%20Shot%202012-09-28%20at%2023.55.26.png

      Reply
    • William says

      09/29/2012 at 3:55 am

      Did you create a VM that has ESXi 5.1 compatibility (e.g. virtual hardware 9)? Please take a closer look at the steps on where to enable VHV.

      Reply
  15. Ram says

    09/28/2012 at 6:38 pm

    nestedHVSupported boolean false

    Now, I understand why, I cannot run nested vm's.

    By the way PowerEdge 1950 III is listed in HCL - VMware esxi 5.1

    Reply
    • William says

      09/28/2012 at 7:29 pm

      @Ram,

      Let me ping a few folks to see if there's anything else you could do. As you know this is all "not supported" and YMMV 😉

      Reply
    • William says

      09/28/2012 at 8:19 pm

      @Ram,

      It looks like the issue is that you don't have EPT support. I know you pointed to ark.intel.com but I was told that even their site is not always 100% accurate and these actual bits are detected and that is why you see nestedHVSupported = false.

      What this means is you won't be able to check the "Hardware Virtualization" box and you WILL be able to install nested ESXi or other hypervisors, just that you won't be able to run 64-bit guestOSes, only 32-bit just like previous releases.

      Reply
  16. Ram says

    09/29/2012 at 1:07 am

    @William - Thanks for all your effort.

    Yes coreinfo did point out that my pe1950 does not support EPT. Wrong info on intel site.

    My main purpose to run vesxi is for learning purpose only. I will play with my lab using 2 other physical pe 1950.

    Reply
    • William says

      09/29/2012 at 3:54 am

      @Ram

      If you CPU has Intel-VT support, you can still install nested ESXi. Please take a look at the note under Step 3

      Reply
  17. Ram says

    09/29/2012 at 1:06 pm

    @ William - I was able to install vESXi successfully. As pointed out b4 cannot run 64bit OS. That's OK!

    By the way, I am getting this error on DP - "the most common cause for this error is that the time on the vdp appliance and your sso server are not in sync" Any thoughts......

    Reply
  18. Ram says

    09/29/2012 at 5:15 pm

    Ignore DP issue, I was able to fix. Once again thanks for updated nested esxi 5.1 instructions.

    Reply
  19. Jeff Hunter says

    10/15/2012 at 5:48 pm

    @Ram - I am curious on what your resolution was for the VDP "most common cause for this error is that the times on the VDP appliance..." error message?

    Reply
  20. prezha says

    11/11/2012 at 9:20 pm

    Failed to install nested Red Hat Enterprise Virtualization Hypervisor v6.3 on ESXi v5.1:
    I enabled promiscuous mode on the portgroup and used Web Client to create a new VM HW v9 as Linux/Other Linux (64-bit) and successfully enabled Hardware Virtualization.
    When I boot VM from ISO and select "Install or Upgrade" option - it just stays frozen there.
    Any thoughts?
    Thank you!

    Reply
    • prezha says

      11/12/2012 at 1:10 am

      Found the workaround: hit to get further to boot: prompt, enter "install" and the installation will continue.
      Ended up with fully installed and configured RHEV Hypervisor and RHEV Manager VMs on ESXi 5.1, but inner-guest VMs (Win7 & WinXP) will not boot at all (staling with "Wait For Launch" message)...
      Any hint would be appreciated.
      prezha

      Reply
    • prezha says

      11/12/2012 at 2:17 pm

      Solved (by restart and some manual work).

      Bottom line: latest RHEV v6.3 *can* run virtualized on latest ESXi v5.1.

      Enjoy!

      Reply
    • Anonymous says

      11/24/2012 at 3:57 am

      Interesting!
      Can you share some tricks please?
      Thanks in advanced.

      Reply
    • Anonymous says

      04/04/2013 at 9:25 pm

      Hello all,

      I have to many problems put RHEV v6.3 run virtualized on latest ESXi v5.1.
      Can some one off you guys help me please?

      When I boot VM from ISO and select "Install or Upgrade" option (or any other option) it just stays frozen.

      @prezha can you tel me better wht you do on your workaround?

      tks in advance.

      Regards.

      Reply
    • Anonymous says

      08/29/2013 at 9:03 pm

      prezha was missing a VERY important word. You must press the "Escape" key at the install screen. That will present you with a "boot:" prompt at which point you must enter "install". Wizard continues from then on as normal.

      Reply
  21. ScruggsDL says

    11/30/2012 at 7:56 pm

    How do I enable hardware assisted virtualization without the web client, I only have the c# client (free version)? I have a Xeon E3-1230 Sandy bridge CPU.

    Reply
  22. VCP2004 says

    12/03/2012 at 5:37 pm

    The why is easy,

    Virtual Labs Today

    Using HPC FT for entire host workloads tomorrow
    (FT Multi CPU + Infiniband + RDMA + Fusion IO Mezanine Card + VMware Disk IO Caching and Replication).

    *protected email*

    Reply
  23. Jannie H says

    12/30/2012 at 2:00 pm

    Hopefully this helps someone at some point:

    Was able to turn on "Expose hardware assisted virtualisation", but when I powered on a VM, vCenter reported that "The target host does not support the virtual machine's current hardware requirements"

    This was odd, as my Intel x5650 is capable of EPT, and nestedHVSupported was listed as "true"...

    Turns out the EVC mode on my cluster was too low, hiding the functionality from the VMs. Increasing the EVC level solved the problem.

    Reply
  24. Venkat Kurela says

    12/30/2012 at 5:11 pm

    hi, i have installed server 2k8 r2 Hyper v on ESXi 5.1, i am able to create vm in server manager but when i tried to power on it i am getting the following error message "the virtual machine could not be started because the hypervisor is not running", checked the HyperV services are running. thanks in advance.

    Cheers
    Venkat Kurela

    Reply
  25. Iwan 'e1' Rahabok says

    01/02/2013 at 5:57 am

    Thanks William. Excellent as always.
    What's the best choice for the SCSI Controller and NIC adapter type?

    Thanks from Singapore
    e1

    Reply
    • William says

      01/02/2013 at 4:16 pm

      I use e1000 & LSI Logic

      Reply
  26. The Foodinator says

    01/16/2013 at 4:28 am

    Have you been able to create a filer at level 1 and present shared storage to subsequent levels? I cannot get this to work. I'd like to run iSCSI over vmxnet3 from level 2 to level 3 so that I don't lose as much disk I/O, ideally.

    Reply
  27. JC says

    01/27/2013 at 5:30 pm

    Hi William,

    Great article, as always. What base hardware specs would you suggest for a nested 5.1 host to be used for training purposes?

    Thanks!

    Reply
    • JC says

      01/27/2013 at 5:49 pm

      As a note, when I created teaching pods for 5.0, this is what I used:
      CPU: 1 socket, 1 core
      RAM: 4GB
      Net Card: E1000
      SCSI: LSI Logic SAS
      Disk: 40GB Thin, Independent, Persistant

      Reply
  28. blog says

    01/30/2013 at 9:14 pm

    Has this changed again? The advanced option setting "vhv.enable='TRUE'" for the VM does not persist from the vSphere Client. However, after tweaking the option in the Web Client, the exposed configuration parameter was "featMask.vm.hv.capable='Min:1'" and the nested feature works... (vSphere 5.1)

    Reply
    • blog says

      01/30/2013 at 9:47 pm

      Looks to hide the "vhv.enable" from vSphere Client while adding and exposing the "featMask.vm.hv.capable" setting. Hmmm...

      (prior to power-on)
      # diff vm03\ \(ESXi\ 4.0\).vmx /tmp/vm03\ \(ESXi\ 4.0\).vmx
      --- vm03 (ESXi 4.0).vmx
      +++ /tmp/vm03 (ESXi 4.0).vmx
      @@ -120,5 +120,4 @@
      config.readOnly = "FALSE"
      -softPowerOff = "FALSE"
      -featMask.vm.hv.capable = "Min:1"
      -vhv.enable = "TRUE"
      +softPowerOff = "TRUE"
      floppy0.present = "FALSE"

      Reply
    • Collin C. MacMillan says

      01/30/2013 at 10:13 pm

      Appears that when adding the setting "vhv.enable" from the vSphere Client, it will not show-up in subsequent listings and will not be persistent.

      However, if you remove it from inventory, add it via shell and re-inventory it, the setting sticks (but is still not listed in advanced settings) - this is verifiable in the Web Client.

      Have you worked-out the process for migrating a running vESXi VM from a 5.0 to a 5.1 host?

      Reply
    • Peter K says

      03/06/2014 at 6:23 pm

      I had the same issue - vhv.enable is not persisted when using vCenter Client with free ESXi 5.5

      I used this approach to manually edit .vmx configuration file:
      http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1017022

      Reply
  29. Anonymous says

    05/06/2013 at 8:39 pm

    Hi William,

    In my nested ESXi, everything is OK. I have a problem after build 50 ESXi VMs. 50 ESXi VMs can ping outside and anywhere but can't ping each other. I don't know the reason why, I use E1000 & E1000E Nic for testing but the result is the same. Can you help me?

    Thanks.
    Randy

    Reply
  30. Anonymous says

    06/22/2013 at 2:22 pm

    Nice post. Keep in way of good job! 🙂

    Reply
  31. Anonymous says

    09/04/2013 at 4:02 am

    Great informative post friend.
    I had trouble getting Nested HVH to work when nesting OracleVM 3.x cluster within ESXi 5.1 until I read this great post.

    Thanks

    --Tibbz

    Reply
  32. Sander van den Brom says

    12/26/2013 at 10:24 pm

    Wonderful post.
    Thank you very much!

    Reply
  33. Steijn van Essen says

    11/19/2014 at 8:34 am

    William, thank you for sharing this information. I'm a newbie on VMware and recently started playing around with ESXi 5.5 on two self built, i7 980X 6-core & i7 950 4-core based servers. I need virtualization for my network labs (exam prep). On the fly I try to pick up some knowledge of Windows Server, Linux and VMware.

    Looking for a way to share my hardware with others I stumbled upon your article about nested 64 bit on vSphere v5. After following all the leads and reading some more documentation I was able to install ESXi 5.5 as guest on my ESXi host. But I got the "HARDWARE_VIRTUALIZATION WARNING: Hardware Virtualization is not a feature of the CPU ..." message.

    When I looked further I ran into this post, for nested 64 bit on vSphere v5.1. The comments on this post gave me the last push. Here's what I do:

    - I create my guest based on VMware Hardware Version 9 in accordance with your articles
    - I do some tweaking in ESXi shell
    - I finalize my configuration with vSphere Client 5.5 (that comes with ESXi 5.5); this INDEED removes vhv.enable="TRUE" from my guest's VMX file
    - when I'm done with configuring I add vhv.enable="TRUE" through ESXi shell; as long as I refrain from further reconfiguration through vSphere Client it will stay there (I guess)
    - After bootup of my ESXi 5.5 guest, "https:///mob/?moid=ha-host&doPath=capability" shows me "nestedHVSupported boolean true" so I think I'm OK with this procedure

    But still: Would you know if there is a more elegant (permanent) solution to the disappearing vhv.enable="TRUE" line with ESXi 5.5 and vSphere 5.5?

    Thanx once again,

    Steijn van Essen
    Network Professional
    The Netherlands

    From i8088 to i7-980X in 25 years and still waiting ...

    Reply
  34. Steijn van Essen says

    11/19/2014 at 8:46 am

    Sorry, need to correct two omissions:
    - 2nd paragraph: ... I was able to install ESXi 5.5 as guest on my ESXi 5.5 host ...
    - 5th paragraph: ... disappearing vhv.enable=”TRUE” line with ESXi 5.5 and vSphere Client 5.5?

    Final note: I run ESXi 5.5 host and vSphere Client 5.5 with an evaluation license (restricted in functionality, but unlimited in time)

    Steijn van Essen

    From i8088 to i7-980X in 25 years and still waiting …

    Reply
  35. David Burns says

    05/19/2015 at 6:46 pm

    When following the instructions, I am able to get the nested hardware support - thank you for the efforts here.

    Now trying to script the same steps from a Packer build server and upload the image to ESX 5.1, the vhv.enable flag is being stripped from my .vmx! Everything looks good before the upload with ovftool 4.0, but when I check the contents on the ESX server the settings are gone.

    Any thoughts in how I might get the flag to pass through ovftool?

    Reply
  36. Bernardo says

    01/02/2016 at 1:58 am

    I'm trying to start a VM under Windows 10 Hyper-V, but the VM always fails to start. I'm trying to run the Windows Phone emulator inside the VM.

    I have an ESXi 5.5 Update 3 and I created a Windows 10 VM, following the 3 steps above for configuring an Hyper-V guest on ESXi. Windows 10 is at the latest build/update level. Then I enabled the Hyper-V feature on Windows 10 and created an Hyper-V VM. The Hyper-V VM is not able to start.

    If I replace, in the same VM, Windows 10 with Windows 8.1 and create an Hyper-V VM or start the Windows Phone Emulator, everything work fine.

    So it appears there is a problem with the Hyper-V that ships with Window 10. Has anyone been able to run an Hyper-V guest in windows 10?

    Reply
    • Michael Garcia says

      03/22/2016 at 2:48 pm

      I am having the same issue. Did you get this resolved?

      Reply
  37. vikrant pawar says

    01/15/2016 at 1:01 am

    I'm using Esxi 6.x and I want to enable KVM in one of our guest, web client is not working hence only option is vSphere Client, using earlier vmx files I set parameter for featMask.vm.hv.capable = "Min:1", however it's still not working.
    I there any thing else I need to setup.

    Reply
  38. Robi Rodriguez says

    01/31/2016 at 6:55 pm

    Hello everyone. I have been running nested Hyper-V inside of ESXi6 for sometime now as well as I also have a ESXi5.5 nested inside of my same ESXi6 environment. The question that I have is that I am able to install the Hyper-V role no problem and create VMs etc. The issue is that I have not been successful in trying to install the Windows Server RDS Virtualization Host role on a nested Windows Server 2016 Technical Preview 4 release VM in my ESXi6 environment. It keeps giving me an error that Hardware Virtualization is not present etc. Now keep in mind I have made all of the changes needed and this very server has Hyper-V installed and running VMs no problem at all. It is just the RDS VDI role that can not be installed. I have not been able to find any information pertaining to this being possible. If someone knows how to get this to work it would be great. Thanks

    Reply
  39. brenton says

    02/17/2016 at 11:38 pm

    THANK YOU!!! Without vSphere web client I really struggled with this but knowing how to edit the vmx by adding vhv.enable = "true" Was a life saver!

    Reply
  40. Rashmi says

    04/19/2017 at 9:09 am

    Hi, its nice article... I have my nested esxi. Now, if I want my nested esxi to have ip from dhcp server rather than 192.168.1.190, how can I do that? Thanks in advance

    Reply
  41. Neilson Henriques says

    04/25/2017 at 8:41 am

    William, just passing by to say how much your articles have helped me thru the years. Thanks a lot, mate. All the best to you and keep up the outstanding job !

    Reply
  42. shahnaz khan says

    10/06/2018 at 10:59 pm

    Hi Will, I need to setup a nested lab environment on esxi 6.7 , please send me required link.

    Reply

Trackbacks

  1. How To Enable Nested ESXi Using VXLAN In vSphere & vCloud Director | virtuallyGhetto says:
    02/28/2014 at 9:02 pm

    […] You are now done with the VXLAN configurations in the vCNS Manager and can proceed to the regular instructions for enabling Nested ESXi for vSphere. […]

    Reply
  2. Nested ESXi 5.1 Supports VMXNET3 Network Adapter Type | virtuallyGhetto says:
    03/01/2014 at 4:45 pm

    […] I decided to create a Nested ESXi 5.1 VM, but instead of selecting the e1000 driver which was the only network adapter type that would […]

    Reply
  3. How to Enable Nested ESXi & Other Hypervisors in vCloud Director 5.1 | virtuallyGhetto says:
    03/01/2014 at 4:46 pm

    […] very similar to the previous steps outlined for vCloud Director 1.5 release. The only change is how VHV (Virtual Hardware-Assisted Virtualization) aka "Nested Virtualization" is enabled in vCloud Director 5.1 and ESXi […]

    Reply
  4. TinkerTry IT @ home | VMware ESXi 5.1 can run Microsoft Hyper-V Server 2012 VMs, nice! says:
    03/22/2014 at 2:09 pm

    […] if you got VMware ESXi 5.1 working on your hardware, that means your BIOS and CPU are capable with Intel VT-x or AMD-V explained here, and you will very likely be able to get ESXi 5.1 to run Windows Server 2012 with […]

    Reply
  5. Server virtualization nested and tiered hypervisors - StorageIOblog says:
    04/09/2014 at 10:30 pm

    […] up to speed. One is a VMware knowledge base piece, two are from William Lam (@lamw) Virtual Ghetto (getting started here and VSAN here) and the other is from Duncan Epping @DuncanYB Yellow Bricks […]

    Reply
  6. 8-Core, 32GB RAM, 240GB Flash Home Lab Hardware Assembly - Virtxpert says:
    05/05/2014 at 12:19 pm

    […] CPU was whether it supported RVI so I could boot 64-bit OS VMs from a nested ESXi VM.  Thanks to this post from William Lam I was able to confirm that, in fact, the CPU does support this […]

    Reply
  7. AMD 8-Core Nested ESXi Setup (The hard way) - Virtxpert says:
    05/13/2014 at 7:18 pm

    […] can find more information thanks to William Lam's post here, including nesting […]

    Reply
  8. Ramdan Virtual : Episode 11: Nested VM – VM Man | VMware ,vsphere esxi ,VMware Course ,VCP Course ,Cloud ,VDI ,Hyperv ,Citrix ,شرح كورسات. says:
    07/09/2014 at 10:05 pm

    […] http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html […]

    Reply
  9. Ramdan Virtual : Episode 11: Nested VM | blog says:
    07/14/2014 at 7:28 am

    […] http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html […]

    Reply
  10. Nested ESXi Issues | Datability says:
    08/11/2014 at 7:03 am

    […] http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html […]

    Reply
  11. Home Lab Set-Up | VirtualEarthrunner says:
    10/15/2014 at 9:33 am

    […] http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html – by William Lam […]

    Reply
  12. Installing Nested ESXi Host | Caroline Wood : NET701 Modules says:
    10/16/2014 at 9:40 am

    […] How to Enable Nested ESXi & Other Hypervisors in vSphere 5.1 […]

    Reply
  13. Simplifying CloudStack Manager Build part 2 says:
    01/10/2015 at 5:04 pm

    […] This blog post here is one of the most useful regarding point 3. http://www.virtuallyghetto.com/2012/08/how-to-enable-nested-esxi-other.html […]

    Reply
  14. Building a Nested ESXi Home Lab | SIPALOOZA says:
    01/21/2015 at 4:21 pm

    […] be done on a per-VM basis by adding a new configuration option to a VM's vmx file (see this post by William Lam for details). You cannot do this through the vSphere Client, so stay at the command line and […]

    Reply
  15. vShiza says:
    02/01/2015 at 11:34 pm

    […] must be done on a per-VM basis by adding a new configuration option to a VM's vmx file (see this post by William Lam for details). You cannot do this through the vSphere Client, so stay at the command line and […]

    Reply
  16. vSphere 2015 : VMFS on USB - Hypervisor.fr says:
    03/30/2015 at 7:34 am

    […] ça, le support des VMXNET3, les vmtools pour ESXi, le fling Mac Learning dvFilter, le support du Virtual Hardware-Assisted Virtualization et le support d'Hyper-V en GuestOS, nous nous sommes laisser dire qu'on pourrait […]

    Reply
  17. Nested Hyper-V on vSphere ESXi | Cloud Jockey says:
    09/10/2015 at 8:40 pm

    […] VirtuallyGhetto […]

    Reply
  18. How to run a nested ESXi 6 on top of your vSphere 5.5 environment | Virtual Kim says:
    10/20/2015 at 4:37 am

    […] 6. open the vmx file of the host and add the folowing line in the vmx:  vhv.enable = "TRUE". This will aloow you to run 64 bit Guest OS's in your nested vSphere 6 environment build on top of your vSphere 5.x environment. This parameter is fully descirbed in William Lam's blog: Nested ESXi 5.5 […]

    Reply
  19. Nested Virtualization Resources | virtuallyGhetto says:
    12/15/2015 at 9:07 am

    […] How to Enable Nested ESXi & Other Hypervisors in vSphere 5.1 […]

    Reply
  20. vSphere 6.0 Update 2 hints at Nested ESXi support for Paravirtual SCSI (PVSCSI) in the future | virtuallyGhetto says:
    03/14/2016 at 10:06 am

    […] VMX file and even earlier versions required several VMX entries. VHV can now be easily enabled using either the vSphere Web Client or the vSphere […]

    Reply
  21. Automating ESXi 5.1 Kickstart Tips & Tricks | virtuallyGhetto says:
    06/16/2016 at 3:31 pm

    […] in ESXi 5.1, you need to to specify new vhv.enable parameter, please take a look at this article for more […]

    Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • Self-Contained & Automated VMware Cloud Foundation (VCF) deployment using new VLC Holodeck Toolkit 03/29/2023
  • ESXi configstorecli enhancement in vSphere 8.0 Update 1 03/28/2023
  • ESXi on Intel NUC 13 Pro (Arena Canyon) 03/27/2023
  • Quick Tip - Enabling ESXi Coredumps to be stored on USB 03/26/2023
  • How to disable the Efficiency Cores (E-cores) on an Intel NUC? 03/24/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023