WilliamLam.com

How to Enable Nested ESXi & Other Hypervisors in vSphere 5.1

by // 88 Comments

There are a ton of new features with the latest release of vSphere 5.1, but the one "unsupported" feature I always test first is "Nested Virtualization" (aka Nested ESXi) and with the latest release, it seems to have gotten even better. You will still need to have the same physical CPU prerequisites as you did in the past to run "Nested Virtualization" as well as nesting 64-bit VMs.

  • Intel VT-x or AMD-V is required for running "Nested Virtualization" which supports nested 32-bit VMs
  • Intel EPT or AMD RVI is required for running nested 64-bit VMs.

A quick way to verify whether your CPU truly supports both Intel-VT+EPT or AMD-V+RVI, you can paste the following into a browser:  

https://[your-esxi-host-ip-address]/mob/?moid=ha-host&doPath=capability

You will need to login with your root credentials and then look for the "nestedHVSupported" property and if it states false, it means you maybe able to install nested ESXi or other hypervisor, but you will not be able to run nested 64-bit VMs, only 32-bit VMs, assuming you have either Intel-VT or AMD-V support on your CPUs.

For more details, take a look at this article for troubleshooting: Having Difficulties Enabling Nested ESXi in vSphere 5.1?

Disclaimer: This is not officially supported by VMware, please use at your own risk.

There are some changes with Nested Virtualization in vSphere 5.1 also officially known as VHV (Virtual Hardware-Assisted Virtualization). If you are using vSphere 5.0 to run Nested ESXi or other nested Hypervisors, then please take a look at the instructions in this article. With vSphere 5.1, there have been a few minor changes to enable VHV.

  1. The new Virtual Hardware 9 compatibility will be required when creating your nested ESXi VM, Virtual Hardware 8 will not work if you are running ESXi 5.1 on your physical host. You will still need to enable promiscuous mode on the portgroup that will be used for your nested ESXi VM for network connectivity.
  2. vhv.allow = "true" is no longer valid for ESXi 5.1 to enable VHV. A new parameter has been introduced called vhv.enable = "true" that is now defined on a per VM basis to provide finer granularity of VHV support. This also allows for better portability between VMware's hosted products such as VMware Fusion and Workstation as they also support the vhv.enable parameter.
  3. You can now enable VHV on a per VM basis and using the new vSphere Web Client which basically adds the vhv.enable = "true" parameter to the VM's .VMX configuration file.

Note: You can run a nested ESXi 5.1 VM on top of a physical ESXi 5.0 host, just follow the instructions here.

Enabling VHV (Virtual Hardware-Assisted Virtualization)

Step 1 - Create a new Virtual Hardware 9 Virtual Machine using the new vSphere Web Client that's available with vCenter Server 5.1.

Step 2 - Select "Linux" as the guestOS Family and "Other Linux (64-bit)" as the guestOS Version.

Step 3 - During the customize hardware wizard, expand the "CPU" section and select "Hardware Virtualization" box to enable VHV.

Note: If this box is grayed out, it means that your physical CPU does not supported Intel VT-x + EPT or AMD-V + RVI which is required to run VHV OR that you are not using Virtual Hardware 9. If your CPU only supports Intel-VT or AMD-V, then you can still install nested ESXi, but you will only be able to run nested 32-bit VMs and not nested 64-bit VMs.

Step 4 - It is still recommended that you change the guestOS Version to VMware ESXi 5.x after you have created the VM shell, as there are some special settings that are applied automatically. Unfortunately with the new vSphere Web Client, you will not be able to modify the guestOS after creation, so you will need to use the C# Client OR manually go into the .VMX and update guestOS = "vmkernel5"

Now you are ready to install nested ESXi VMs as well as run nested 64-bit VMs within.

If you have followed my previous article about How to Enable Support for Nested 64bit & Hyper-V VMs in vSphere 5 you may recall a diagram about the levels of "Inception" that can be performed with nested ESXi. That is, the number of times you could nest ESXi and still have it be in a "functional" state. With vSphere 5.0, the limit that I was able to push was 2 levels of nested ESXi. With latest release of vSphere 5.1, I have been able to push that limit now to an extraordinary 3 levels of inception!

You might ask why would someone want to do this ... well I don't have a good answer other than ... because I can? 😉 VHV is one of the coolest "unsupported" feature in my books and I'm glad it is working beyond what it was designed for.

For proper networking connectivity, also ensure that either your standard vSwitch or Distributed Virtual Switch has both promiscuous mode and forged transmit enabled either globally on the portgroup or distributed portgroup your nested ESXi hosts are connected to.

Nesting "Other" Hypervisors

For those of you who feel inclined to run other hypervisors such as Hyper-V, you can do so with latest release of ESXi 5.1. The process if very straight forward just like running nested ESXi host.

Step 1 - Create a Virtual Hardware 9 VM and select the appropriate guestOS. In this example, I selected Windows Server 2012 (64-bit) as the guestOS version.

Step 2 - Enable VHV under the CPU section if you wish to create and run nested 64-bit VMs under Hyper-V

Step 3 - You will need to add one additional .vmx parameter which tells the underlying guestOS (Hyper-V) that it is not running as a virtual guest which in fact it really is. The parameter is hypervisor.cpuid.v0 = FALSE

How to Configure Nested ESXi 5 to Support EVC Clusters

by // 10 Comments

Dave Hill recently wrote an article about running nested ESXi and a gotcha with EVC (Enhanced vMotion Compatibility). In vSphere 4.x, you could not join a nested ESXi host into a cluster with EVC enabled. With vSphere 5, there's actually a way to connect a nested ESXi 5 host to an EVC enabled cluster AND still power on 64bit nested guestOSes.

I have to thank my friend and partner in crime Tuan Duong for showing me this trick awhile back. Tuan was performing some tests using both nested and physical ESXi 5 hosts and discovered this method after a bit of tinkering. At the time, I was not sure if others would find this useful and I did not document the process.

Disclaimer: As usual, this is not officially supported by VMware, use at your own risk. 

Here are the steps:

1. You must be running vSphere 5, create a nested ESXi 5 host using this article How to Enable Support for Nested 64bit & Hyper-V VMs in vSphere 5

2. Create an EVC enabled cluster or use an existing cluster with whatever baseline you would like and click on the "Current CPUID Details" in cluster settings.

3. Copy down the CPU mask flags for that particular EVC baseline, you will need this in the next step

4. Shutdown your nested ESXi 5 host and edit the VM's settings and under "Options" tab click on "CPUID Mask->Advanced". You will take the CPU mask from the above step and update the nested ESXi 5 VM to make it match

5. Go ahead and power on your nested ESXi 5 host and join it to the EVC enabled cluster you created earlier. You should not see any errors when connecting to the cluster and after that you can create a nested 64bit VM within that virtualized ESXi 5 host.

There you have it, running a nested ESXi 5 host and joined to an EVC enabled cluster! Isn't VMware technology awesome! 🙂

The Missing Piece In Creating Your Own Ghetto vSEL Cloud

by // 21 Comments

Awhile back I discovered an undocumented flag called "esxvm" in the SQL statements of the new vCloud Director 1.5 installer that suggested the possibility of deploying nested ESXi hosts in vCD. However, after further investigation the flag only enables the automated deployment of an ESXi 5 parameter (vhv.allow) which is required to run nested ESXi 4.x/5.x hosts as part of preparing a new ESXi 5 hosts in vCloud Director. There was still a missing piece to the puzzle to enable this functionality within vCloud Director user interface.

The answer eventually came from attending a recent session at VMworld 2011 in Las Vegas CIM1436 - Virtual SE Lab (vSEL) Building the VMware Hybrid Cloud by Ford Donald of VMware. I will not go into detail about what vSEL is, if you would like more information take a look at this blog post The Demo Cloud at VMworld Copenhagen or check out Ford's VMworld presentation online. In one of Ford's slides, he describes the necessary steps to enable nested ESXi called ESX_VM mode in vCloud Director which actually consists of two parts:

  • Enable nested virtualization and 64-bit vVM support in vSphere 5
  • Enable special mode in vCloud Director called ESX_VM to allow for vSphere 4 and 5 hosts as valid guestOS types

There are also some additional steps that are required after enabling ESX_VM mode:

  • Preparing or re-preparing ESXi 5 hosts
  • Allowing for Promiscuous Mode in vCD-NI or VLAN-backed Network Pool

********************* DISCLAIMER *********************
This is not a supported configuration by VMware and this can disappear at any time, use at your own risk 

********************* DISCLAIMER *********************

Note: I will assume the reader has a good understanding of how to install/configure vCloud Director and how it works. I will not be going into any details in configuring or installing vCD, you can find plenty of resources on the web including here, here, here and here. I will also assume you understand how to configure vCD-NI and VLAN-backed network pools in vCloud Director and how they work.

The first part is to enable nested virtualization (nested ESXi) support within the ESXi 5 hosts when they're being prepared by vCloud Director by updating the following SQL statement as noted in my earlier blog post Cool Undocumented Features in vCloud Director 1.5:

UPDATE config SET value='true' WHERE name='extension.esxvm.enabled';

The second part is to update the vCloud Director database to add support for both vSphere 4 and 5 hosts as valid guestOS types:

INSERT INTO guest_osfamily (family,family_id) VALUES ('VMware ESX/ESXi',6);

INSERT INTO guest_os_type (guestos_id,display_name, internal_name, family_id, is_supported, is_64bit, min_disk_gb, min_memory_mb, min_hw_version, supports_cpu_hotadd, supports_mem_hotadd, diskadapter_id, max_cpu_supported, is_personalization_enabled, is_personalization_auto, is_sysprep_supported, is_sysprep_os_packaged, cim_id, cim_version) VALUES (seq_config.NextVal,'ESXi 4.x', 'vmkernelGuest', 6, 1, 1, 8, 3072, 7,1, 1, 4, 8, 0, 0, 0, 0, 107, 40);

INSERT INTO guest_os_type (guestos_id,display_name, internal_name, family_id, is_supported, is_64bit, min_disk_gb, min_memory_mb, min_hw_version, supports_cpu_hotadd, supports_mem_hotadd, diskadapter_id, max_cpu_supported, is_personalization_enabled, is_personalization_auto, is_sysprep_supported, is_sysprep_os_packaged, cim_id, cim_version) VALUES (seq_config.NextVal, 'ESXi 5.x', 'vmkernel5Guest', 6, 1, 1, 8, 3072, 7,1, 1, 4, 8, 0, 0, 0, 0, 107, 50);

To apply these SQL statements to your vCloud Director 1.5 database, you will need to login to either to your Oracle or SQL Server database and manually execute these statements using the account that you originally created.

Here is an example of executing the SQL statements on an Oracle Express 11g database (Oracle Express is not officially supported by VMware):

As you can see, we need we first create a new guest_osfamily type called "VMware ESX/ESXi" and we need to also provide a unique family_id, which from a default installation of vCloud Director 1.5, the next available value will be 6. Next, we need to create the two new guestos_type "ESXi 4.x" and "ESXi 5.x" and again we need to provide a unique guestos_id which from a default installation of vCloud Director 1.5, the next available values will be 81 and 82. If any errors are thrown regarding a constraint being violated, then the ids may already have been used, you can always query to see what the next value is or select a new id.

Once you have executed the SQL statements, you will need to restart the vCloud Director Cell for the changes to take effect and if you already have prepared ESXi 5 hosts, you will need to re-prepare the hosts.

If you prefer not to manually do this, you can take a look at my blog post Automating vCloud Director 1.5 & Oracle DB Installation which has been updated to allow you to enable ESX_VM mode with your vCloud Director 1.5 installation. There is a new flag in the vcd.rsp file called ENABLE_NESTED_ESX which can be toggled to true/false which will automatically perform the SQL statements as part of the post-installation of vCloud Director 1.5 and restart the vCD Cell for you.

Here is a screenshot if you decide to enable this flag:

Finally, the last configuration tweak is to enable both promiscuous mode and forged transmit in either your vCD-NI or VLAN-backed Network Pool which is a requirement to run nested ESXi hosts. You locate the name of your network pool to identify distributed portgroup.

Next you can either use the vCD API or login to your vCenter Server and enable the promiscuous mode for that specific distributed portgroup.

UPDATE: Thanks to @DasNing - You can also enable promiscuous mode by executing the following SQL query: UPDATE network_pool SET promiscuous_mode='1' WHERE name=';

We are finally done with all the configurations!

If you successfully completed the above, when you go and create a new virtual machine in vCloud Director, you should now have a new Operation System Family called "VMware ESX/ESXi"

Within this new OS family, you can now provision a new ESXi 4.x or ESXi 5.x guestOS

Here is an example of my own vGhettoPod which includes vMA5 and vESXi 5 host which I can use to perform various types of testing in my home lab.

Now you can create your own ghetto vSEL cloud using VMware vSphere 5, vCloud Director 1.5 and vShield 5!