Okay, Wrecking Crew Inc. is not a real company but a fictitious one that was used during Alan Renouf and Li Zheng's VMworld session, #INF5803 - Deploy Hundreds of VMs Instantly via Forking (aka vSphere Instant Clone). During the session, Wrecking Crew Inc. was described as a modern online gaming company who was about to launch a new product. They found out that their game was going to be more popular than they had originally anticipated and needed to be able to quickly spin up a large number of instances of their game to handle the load.
Though this was a fictional company to help set the stage for the demo, the underlying use case is an actual challenge for many of our customers. One of the examples that Alan used was the traditional Black Friday sale which occurs on the Friday after Thanksgiving in US. As you can imagine, being able to quickly scale up your online application based on customer demand can really give retailers an real edge over their competition. In todays world, waiting a few additional seconds for a site to load can mean the difference between a sale and customer going else where for their business. Online retail is just one of the many industries and verticals that can easily benefit from the Instant Clone capability.
To help further demonstrate the use case, a recorded demo was shown utilizing the new PowerCLI Extensions which provides access to the Instant Clone feature from an Automation standpoint. I actually built the demo for Alan and Li's session and I thought I would share some more details along with the video in case you were not able to attend the session in person, which I was not able to do.
Below are the set of technologies that were used in the demo:
- VMware vSphere + Virtual SAN
- Intel Hardware
- VMware Photon
- HashiCorp Consul
I wanted to also give a quick shoutout to my buddy Rawlinson Rivera for his help with Intel and getting us access to their 64 Node NVMe VSAN Cluster. Our friends over at Intel were kind enough to allow us to quickly record a demo before they had to pack and ship the hardware to San Francisco for VMworld.
To provide some additional context on what you will see in the video, a diagram is provided below. The environment initially consists of two Photon VMs which runs the Consul Cluster which provides service discovery and configuration capabilities and the Nginx Load Balancer which is highlighted in the purple and blue box. Each of those VMs run their respective application which runs inside of a Docker Container and is highlighted in the smaller boxes. On each ESXi host, we have Photon VM which contains our Node.js application that we will be Instant Cloning from. This Photon VM is also referred to as the Parent VM and where all the new forked children VMs will be spawn off of.
This Photon VM also runs an instance of Consul, Registrator and the Node.js application inside of a Docker Container. As new instances are brought online which is highlighted in the black box, they will automatically register themselves with the Consul Cluster. As the Docker Containers are started up, Registrator will be notified to automatically add the new instances to the Nginx Load Balancer, making our application servers available for use immediately. When our application server powers down, Registrator will automatically detect that the Docker Container for our Node.js application is no longer running and automatically unregister it from the Nginx Load Balancer. To simply scale up or down the application, it is simply Instant Cloning our Photon (Parent) VM and powering it on or off, there are no additional steps required.
Without further ado, here is the video of the demo. If you want additional commentary and you attended VMworld US but could not make it to Alan and Li's session, you can now watch it online here. This session will also be repeated in VMworld EMEA for those attending in a couple of weeks. I would highly recommend you check out the session as there is a lot of awesomeness in the session along with technical deep dive of the Instant Clone technology.
VMworld 2015 Demo of how Wrecking Crew Inc. leverages vSphere's Instant Clone feature from lamw on Vimeo.
For more details about Instant Clone, be sure to check out these resources below which includes a Instant Clone script repository for several GuestOSes, including Photon which was used in the demo:
- Project Fargo aka VMFork – What is it?
- Project Fargo aka VMFork and TPS?
- Instant Clone PowerCLI cmdlets Best Practices & Troubleshooting
- Instant Clone community customization script repository
- How to VMFork aka Instant Clone Nested ESXi?
- VMware Instant Clone is now at your fingertips with the updated PowerCLI Extensions fling!
- Using VMware Instant Clone via PowerCLI Extensions Fling
Very great demo. But the link provided to watch the demo video online redirects me to the link below which i could not find any video in that page.
William Lam says
The link you're referring to is the VMworld session itself in case you wanted additional commentary and you had to be a VMworld attendee to view.
However, the video of my demo is available for everyone and you can see it by scrolling towards the bottom of the article 🙂
This seems like great technology, have you heard any word on a WebServices API support though? Seems like a lot of people working at this scale wouldn't be using powercli.
William Lam says
Yes, having an API for Instant Clone would indeed be quite powerful for those looking from a pure Automation standpoint. This is something I've mentioned to the PM as well and from what I understand, this is being worked on.
Very cool thanks for the info, hopefully we'll at least get an experimental api in the near future, I think there's a ton of potential for this functionality.
For those who get to this comment via google, there's a bit of a thread at https://communities.vmware.com/message/2556202. It actually appears that some of the API exists and is public except for the method that accepts CreateChildSpec (VirtualMachine.createChild?). Is a bit of a shame for those of us in highly automated environments, since it's soooo close.
Is there a reason that VMWare is reticent to publicize functionality of the vsphere ws apis? It seems like VMWare would rather make a new api (sometimes that calls webservices... eg pyvmomi) than to document things in web services? Tagging seems to be another case of this, we've got lots of other api's that do this (and probably call webservices internally) but we don't have the webservices documentation...
Anyways great blog.
Alexey Vekshin says
Speaking of cloning, mass depoyment and image customization: I've implemented some script for Linux customization from vApp/OVF properties based on your old blog posts about OVF environment. It is rather simple, but works quite OK for me and greatly simplifies VM cloning process. Sources and some documentation are on github: https://github.com/veksh/ovfconf
Thank you for your blog, it is very informative and inspiring.