nested virtualization

VMworld vBrownBag Tech Talk : Nested Virtualization & Dev/Test/Home Lab Panel

08.12.2014 by William Lam // 4 Comments

VMworld is only a couple of weeks away and I can not believe this will be my 7th VMworld! My, how time has flown by so quickly. I have been pretty busy these last couple of months finishing up some internal projects as well as starting up a couple of new ones. I had been thinking about submitting a vBrownBag Tech Talk as I have done in past years, but there has just been too much going on. Giving it some more thought, I thought it would be cool to put together a panel of community folks to discuss some of my favorite topics like Nested Virtualization as well as Development/Test and Home Labs.

I am please to announce the VMworld vBrownBag Tech Talk : Nested Virtualization & Dev/Test/Home Lab Panel which will include Sean Crookston, Doug Baer, Nick Marshall and myself as the panelists. I was originally hoping to have a few more folks from the community, but due to the late submission, we ran into scheduling conflicts. I am very excited for this session which will take place on Wednesday, August 27th from 11:45am to 12:15pm (30minutes). I wanted to give a huge shout out to Sean Massey who was originally scheduled to present right after ours but decided to offer us his time slot as 15minutes was going to be tough for a panel discussion. Much appreciated Sean!

Due to the short amount of time, we really want to make the most out of this session and most importantly, make this as interactive as possible with the audience. We would like to collect any questions or topics that folks might be interested and we will pick a couple for the panelists to answer or discuss. We will also have topics that we may raise but it would be much more interesting to hear from you! Please leave a comment if you wish to ask a question and perhaps those that get selected, may even win a prize?

We hope to see you at the Tech Talk and lastly, this is going to be a MUST attend session ... that's all I can really say 🙂

BTW - I also would like to give a shout out to Doug Baer who will be running a VMware Knowledge Expert discussion related to the HOL Lab Environment on Tuesday, August 26 at 1pm PST. Though his focus will primarily be HOL, but as many of you know the underlying technology is Nested Virtualization. A couple of us will also be attending that session, so if there are any questions you would like to ask but did not get a chance to during the Tech Talk, you can also find us there.

How to run Nested Mac OS X guest on ESXi VM on top VMware Fusion?

08.08.2014 by William Lam // 1 Comment

You might be asking, why would anyone want to do this? Well, luckily this is not a "because you can" type of answer but was it was an interesting solution that one of our VMware Engineers (Darius) had shared with me after helping out on this VMTN Community forum thread.

The user was running VMware Fusion on his physical Mac OS X system and wanted to be able to test OS X Mavericks under ESXi. Not having a physical ESXi host to test with, the next best thing was to run a ESXi VM under VMware Fusion and then run the Mavericks guest on top of that.

Here is a quick diagram of the user setup:

The issue with just simply doing this is that for a Mac OS X guest to properly run on ESXi, the underlying hardware must be Apple Hardware. The reason for this is not a technical challenges, but rather a legal one per Apple's EULA. The way in which ESXi detects that the underlying hardware is Apple is by checking whether Apple's SMC (System Management Controller) is available.

In the scenario above, the Nested ESXi VM is not automatically passing through the SMC from the physical Mac OS X system and hence the Mac OS X VM at the very top of the stack will not properly function. The solution that Darius found was to add the following two Advanced VM Settings (VMX) entries to the ESXi VM:

smc.present = "TRUE"
smbios.reflectHost = "TRUE"

This will allow the passing of the underlying SMC up into the Nested ESXi VM which will then allow Mac OS X guest VMs to properly function. We can also confirm this by check the Nested ESXi MOB by pointing a browser to the following URL: https://[ESXI-IP]/mob/?moid=ha-host&doPath=hardware

If you did not add the two entries above, then the smcPresent property would show up as false. In our case, we did add the following two entries and we now run our Mac OS X Guest. Here are a couple of screenshots of performing this on my iMac at home running the same exact configuration:

Thanks Darius for sharing this with me and the community! I am sure this will come in handy for anyone wanting to test Mac OS X guests under ESXi but do not have a physical ESXi host and can easily substitute using VMware Fusion.

Do I need additional licenses for Nested ESXi?

05.21.2014 by William Lam // Leave a Comment

The topic of licensing is usually something I try to stay far away from because I have no interest in it and it is usually complex, which ends up hurting my brain. Having said that, one question that I receive every so often from customers who run large number of Nested ESXi instances in their environment is whether or not they need an additional license for a Nested ESXi instance?

The answer to that question can be found in a VMware KB 2009916 that was published last year, here is a quick excerpt:

Customers running nested ESXi/ESX will need to obtain additional licenses for the nested ESXi/ESX. This includes, but is not limited to:

VMware ESXi/ESX running in VMware Workstation or VMware Fusion

VMware ESXi/ESX running in VMware ESXi/ESX

VMware ESXi/ESX running in other third-party hypervisor solutions

I know this might be a bit disappointing to hear, but you should remember that Nested Virtualization and specifically Nested ESXi is not officially supported by VMware. As such, from a EULA point of view, this is just seen as another instance of ESXi being deployed which will require a license. For customers who have an ELA (Enterprise Licensing Agreement) with VMware with an "all you can eat" license agreement, this is not an issue, but for others who do not have such an agreement, here are a couple of tips that may help.

Disclaimer: The information listed below are those of my own personal opinion. For any concerns or clarifications regarding licensing, please reach out to your local VMware account team.

Run ESXi in evaluation mode - Depending on your use case, you may only need your Nested ESXi instance for a short period of time and you can always re-install using the 60 day evaluation
Configure Nested ESXi with single vCPU - Since ESXi is licensed on a per socket basis, you can assign a single vCPU with 2-cores to minimize the licensing impact. For customers who are using vCloud Director and is not on the latest VCD 5.5 release, you may want to import your Nested ESXi VM through vCenter Server as VCD does not support per core allocation in previous releases

If you feel that Nested ESXi should be supported or that there should be a specific SKU or licensed feature for Nested ESXi, be sure to provide this feedback to your local account team with your use cases so it can be fed back to product management.