WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / Automation / How Wrecking Crew Inc. leveraged vSphere's Instant Clone to instantly provision hundreds of VMs

How Wrecking Crew Inc. leveraged vSphere's Instant Clone to instantly provision hundreds of VMs

09.18.2015 by William Lam // 9 Comments

Okay, Wrecking Crew Inc. is not a real company but a fictitious one that was used during Alan Renouf and Li Zheng's VMworld session, #INF5803 - Deploy Hundreds of VMs Instantly via Forking (aka vSphere Instant Clone). During the session, Wrecking Crew Inc. was described as a modern online gaming company who was about to launch a new product. They found out that their game was going to be more popular than they had originally anticipated and needed to be able to quickly spin up a large number of instances of their game to handle the load.

Though this was a fictional company to help set the stage for the demo, the underlying use case is an actual challenge for many of our customers. One of the examples that Alan used was the traditional Black Friday sale which occurs on the Friday after Thanksgiving in US. As you can imagine, being able to quickly scale up your online application based on customer demand can really give retailers an real edge over their competition. In todays world, waiting a few additional seconds for a site to load can mean the difference between a sale and customer going else where for their business. Online retail is just one of the many industries and verticals that can easily benefit from the Instant Clone capability.

To help further demonstrate the use case, a recorded demo was shown utilizing the new PowerCLI Extensions which provides access to the Instant Clone feature from an Automation standpoint. I actually built the demo for Alan and Li's session and I thought I would share some more details along with the video in case you were not able to attend the session in person, which I was not able to do.

Below are the set of technologies that were used in the demo:

  • VMware vSphere + Virtual SAN
  • Intel Hardware
  • VMware Photon
  • HashiCorp Consul
  • Nginx
  • Registrator
  • Docker
  • Node.js

wrecking-crew-inc-using-instant-clone-to-provisioning-hundreds-to-thousands-of-vms-0
I wanted to also give a quick shoutout to my buddy Rawlinson Rivera for his help with Intel and getting us access to their 64 Node NVMe VSAN Cluster. Our friends over at Intel were kind enough to allow us to quickly record a demo before they had to pack and ship the hardware to San Francisco for VMworld.

To provide some additional context on what you will see in the video, a diagram is provided below. The environment initially consists of two Photon VMs which runs the Consul Cluster which provides service discovery and configuration capabilities and the Nginx Load Balancer which is highlighted in the purple and blue box. Each of those VMs run their respective application which runs inside of a Docker Container and is highlighted in the smaller boxes. On each ESXi host, we have Photon VM which contains our Node.js application that we will be Instant Cloning from. This Photon VM is also referred to as the Parent VM and where all the new forked children VMs will be spawn off of.

This Photon VM also runs an instance of Consul, Registrator and the Node.js application inside of a Docker Container. As new instances are brought online which is highlighted in the black box, they will automatically register themselves with the Consul Cluster. As the Docker Containers are started up, Registrator will be notified to automatically add the new instances to the Nginx Load Balancer, making our application servers available for use immediately. When our application server powers down, Registrator will automatically detect that the Docker Container for our Node.js application is no longer running and automatically unregister it from the Nginx Load Balancer. To simply scale up or down the application, it is simply Instant Cloning our Photon (Parent) VM and powering it on or off, there are no additional steps required.

wrecking-crew-inc-using-instant-clone-to-provisioning-hundreds-to-thousands-of-vms-1

Without further ado, here is the video of the demo. If you want additional commentary and you attended VMworld US but could not make it to Alan and Li's session, you can now watch it online here. This session will also be repeated in VMworld EMEA for those attending in a couple of weeks. I would highly recommend you check out the session as there is a lot of awesomeness in the session along with technical deep dive of the Instant Clone technology.

VMworld 2015 Demo of how Wrecking Crew Inc. leverages vSphere's Instant Clone feature from lamw on Vimeo.

For more details about Instant Clone, be sure to check out these resources below which includes a Instant Clone script repository for several GuestOSes, including Photon which was used in the demo:

  • Project Fargo aka VMFork – What is it?
  • Project Fargo aka VMFork and TPS?
  • Instant Clone PowerCLI cmdlets Best Practices & Troubleshooting
  • Instant Clone community customization script repository
  • How to VMFork aka Instant Clone Nested ESXi?
  • VMware Instant Clone is now at your fingertips with the updated PowerCLI Extensions fling!
  • Using VMware Instant Clone via PowerCLI Extensions Fling

More from my site

  • Instant Clone community customization script repository
  • Leveraging Instant Clone in vSphere 6.7 for extremely fast Nested ESXi provisioning
  • Translating vSAN VM Object IDs (UUID to VM and VM to UUID)
  • How to convert vSAN RVC commands into PowerCLI and/or other vSphere SDKs?
  • Project USB to SDDC - Part 3

Categories // Automation, Cloud Native, Docker, VSAN, vSphere 6.0 Tags // consul, Docker, instant clone, nginx, node.js, Photon, PowerCLI, registrator, Virtual SAN, vmfork, VSAN, vSphere 6.0

Comments

  1. Marce says

    09/18/2015 at 6:31 pm

    Woah! Amazing!

    Reply
  2. Eddyc says

    09/21/2015 at 3:09 am

    Hi William,

    Very great demo. But the link provided to watch the demo video online redirects me to the link below which i could not find any video in that page.
    https://vmworld2015.lanyonevents.com/connect/sessionDetail.ww?SESSION_ID=5803&tclass=popup

    Reply
    • William Lam says

      09/21/2015 at 1:53 pm

      The link you're referring to is the VMworld session itself in case you wanted additional commentary and you had to be a VMworld attendee to view.

      However, the video of my demo is available for everyone and you can see it by scrolling towards the bottom of the article 🙂

      Reply
      • Kevin says

        03/14/2016 at 8:29 am

        This seems like great technology, have you heard any word on a WebServices API support though? Seems like a lot of people working at this scale wouldn't be using powercli.

        Reply
        • William Lam says

          03/14/2016 at 8:31 am

          Hi Kevin,

          Yes, having an API for Instant Clone would indeed be quite powerful for those looking from a pure Automation standpoint. This is something I've mentioned to the PM as well and from what I understand, this is being worked on.

          Reply
          • Kevin says

            03/22/2016 at 11:00 am

            Very cool thanks for the info, hopefully we'll at least get an experimental api in the near future, I think there's a ton of potential for this functionality.

          • Kevin says

            05/06/2016 at 11:09 am

            For those who get to this comment via google, there's a bit of a thread at https://communities.vmware.com/message/2556202. It actually appears that some of the API exists and is public except for the method that accepts CreateChildSpec (VirtualMachine.createChild?). Is a bit of a shame for those of us in highly automated environments, since it's soooo close.

          • Kevin says

            06/21/2017 at 6:19 am

            Is there a reason that VMWare is reticent to publicize functionality of the vsphere ws apis? It seems like VMWare would rather make a new api (sometimes that calls webservices... eg pyvmomi) than to document things in web services? Tagging seems to be another case of this, we've got lots of other api's that do this (and probably call webservices internally) but we don't have the webservices documentation...

            Anyways great blog.

  3. Alexey Vekshin says

    10/01/2015 at 10:34 am

    Hi William,

    Speaking of cloning, mass depoyment and image customization: I've implemented some script for Linux customization from vApp/OVF properties based on your old blog posts about OVF environment. It is rather simple, but works quite OK for me and greatly simplifies VM cloning process. Sources and some documentation are on github: https://github.com/veksh/ovfconf

    Thank you for your blog, it is very informative and inspiring.

    Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • How to disable the Efficiency Cores (E-cores) on an Intel NUC? 03/24/2023
  • Changing the default HTTP(s) Reverse Proxy Ports on ESXi 8.0 03/22/2023
  • NFS Multi-Connections in vSphere 8.0 Update 1 03/20/2023
  • Quick Tip - How to download ESXi ISO image for all releases including patch updates? 03/15/2023
  • SSD with multiple NVMe namespaces for VMware Homelab 03/14/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...