WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / ESXi / How to run Nested ESXi on the vCloud Hybrid Service?

How to run Nested ESXi on the vCloud Hybrid Service?

05.02.2014 by William Lam // 7 Comments

nested-esxi-on-vchsToday I was granted access to VMware's vCloud Hybrid Service and the first order of business for me of course, was to provision a Nested ESXi VM! After going through the vCHS UI (which is very slick and easy to use by the way) and the vCloud Director UI, I realized the ESXi guestOS type has not been enabled on the backend of the vCloud Director Database. This totally makes sense, as vCHS is a production ready service and they definitely would not want to run anything that is not officially supported.

Having said that, I can see the benefits to customers who would like build out a Nested ESXi environment on vCHS for lab purposes instead of having to manage their own. Some customers even leverage Nested ESXi as part of their development and testing of software and it can be challenging at times to quickly spin up a brand new environment. Instead, they go to vCHS and with just a couple of of clicks in the UI or automatically using the vCloud APIs, provision a couple of Nested ESXi instances for testing. You can easily discard the resources once you are done or keep them running a bit longer.

Having worked with vCloud Director in the past, I knew that you could import an OVF/OVA and I thought maybe I could just import the Nested ESXi OVF templates that I built and potentially workaround vCHS "limitation" 🙂

Disclaimer: Nested ESXi and Nested Virtualization is not officially supported by VMware nor is it supported on vCHS

I tried to upload one of the OVF templates that I built, but it turns out vCloud Director does not supported the Dynamic Disks feature, so I had to perform two additional steps.

Step 1 - Download one of the following Nested ESXi OVF templates

  • Single Nested ESXi VM Template
  • 3-Node VSAN Nested ESXi VM Template
  • 32-Node VSAN Nested ESXi VM Template

Step 2 - Import the OVF template in an existing vSphere environment and ensure you are doing so using the vSphere Web Client, as some of the properties may not be imported properly

Step 3 - Once deployed, go ahead and re-export the image to an OVF/OVA (I choose OVA as it is a single file) and this will generate the empty VMDKs for you so the image should still be very small (< 1MB)

Step 4 - Login into to your vCHS account and  click on your Virtual Datacenter. Select Virtual Machines and then click on Manage in vCloud Director. Import the OVF/OVA that you have just exported

Step 5 - Once the import has been completed, you now have a Virtual Machine that has been configured with the correct guestOS type which should be VMware ESXi 5.x as seen in the screenshot below

nested-esxi-on-vchs-2
Step 6 - At this point, you can either mount an ESXi ISO over your browser or upload it into the vCloud Director Catalog so you can mount it locally and begin your installation of ESXi. Below is a screenshot of 3 Nested ESXi VMs running on vCHS

nested-esxi-on-vchs-3
Note: It looks like some of the advanced VM settings that are part of my OVF template are ignored as part of the vCloud Director import. This means that if you would like to run a Nested VSAN environment on vCHS, you will not be able to rely on the SSD emulation setting but instead, you will need to run through the ESXCLI claim rules to mark particular disks as "SSD" devices. It would have been really nice if vCloud Director would preserve all the advanced VM settings but at least you can still run a Nested VSAN environment.

So there you have it, Nested ESXi running on vCHS! I am kind of curious if this is the first instance of a Nested ESXi VM running on vCHS without having admin access on the backend system?

Note: One limitation to be aware of is that since the backend of vCloud Director is not properly enabled for Nested Virtualization support, this means you will NOT be able to run Nested VMs on top of the Nested ESXi instances. This is due to the lack of having Network Pool which has both Promiscuous & Forge Transmits enabled which is a requirement for proper Nested VMs connectivity. I wonder if vCHS should provide Nested Virtualization capabilities? I know I definitely would like to see it, what do you think? Leave a comment if you have some thoughts on this topic.

UPDATE (05/4/14) - If you wish to run a Nested VSAN environment on vCHS, you will need to take a look at this blog post here on how to "fake" an SSD on one of the devices by using ESXCLI claim rules. The rason for this is that you will not be able to leverage the other method of emulating an SSD device via advanced setting as that requires access to the underlying vSphere environment which you will not have in vCHS.

More from my site

  • VM serial logging to the rescue for capturing Nested ESXi PSOD
  • vSphere 6.0 Update 2 hints at Nested ESXi support for Paravirtual SCSI (PVSCSI) in the future
  • Deploying Nested ESXi is even easier now with the ESXi Virtual Appliance
  • VMware Tools for Nested ESXi updated to v1.2
  • How to run Qemu & KVM on ESXi?

Categories // ESXi, Nested Virtualization, VSAN, vSphere Tags // esxi, nested, nested virtualization, ssd, vCHS, vcloud hybrid service, VSAN

Comments

  1. vinf.net says

    05/02/2014 at 3:33 pm

    It should support it. just think, uber vTARDIS 😉

    Reply
  2. Travis Lawrence says

    06/09/2014 at 4:51 pm

    Running nested VMs in nested ESXi in vCHS would be huge for our Solutions Lab environment. It would greatly help the resellers we work with.

    Reply
  3. Matt Vaughan says

    06/27/2014 at 2:00 am

    Yes, would really love nested VM support. That would be most advantageous for DR purposes using a 3rd party solution such as Veeam.

    Reply
  4. Patrice says

    07/12/2014 at 4:13 pm

    A nested ESXi support on the vCloud Hybrid Services, would be very nice if it were officially supported. It could be used as reserve resource, in a VCAC managed Cloud environment, with the advantage that no additional blueprints have to be created to provision a service to vCHS. The control could be made ​​through the reservations and their priority

    Reply
  5. daunce says

    10/22/2015 at 4:57 am

    For Step 3, why can't you just make that .ova available for download, and save us from having to import then export your Template?

    Reply
    • William Lam says

      10/22/2015 at 5:50 am

      I would love to but I'm unable to redistribute our official software

      Reply
  6. Jeremy Forguson says

    11/26/2015 at 9:21 pm

    It would be awesome of you could create trunked networks within vCD to which you could map vSphere hosts. This would open up the door for the ability to run guest VMs inside of nested vSphere hosts. Totally not supported for Production environments, but for educational purposes and making oneself familiar with products such as vRA and View, this would be huge.

    Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • How to disable the Efficiency Cores (E-cores) on an Intel NUC? 03/24/2023
  • Changing the default HTTP(s) Reverse Proxy Ports on ESXi 8.0 03/22/2023
  • NFS Multi-Connections in vSphere 8.0 Update 1 03/20/2023
  • Quick Tip - How to download ESXi ISO image for all releases including patch updates? 03/15/2023
  • SSD with multiple NVMe namespaces for VMware Homelab 03/14/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...