WilliamLam.com

How To Enable Nested ESXi Using VXLAN In vSphere & vCloud Director

by // 9 Comments

Recently I had received several inquiries asking on how to configure nested ESXi (Nested Virtualization) to function in a VXLAN environment. I have written several articles in the past on configuring nested ESXi in a regular vSphere and vCloud Director environment, but with the use of a VXLAN backed network, there are a few additional steps that are required. These steps include additional configurations of the vCloud Network & Security Manager (previously known as vShield Manager) which ensures that both the required promiscuous mode and forged transmits are automatically enabled for the VXLAN virtual wires (vWires) as they are managed exclusive by the vCNS Manager.

In this article, I will walk you through the configurations that is required when using VXLAN in both a vSphere only environment as well as a vCloud Director environment. If you would like to learn more about how VXLAN works, be sure to check out the multi-part VXLAN series (Part 1/Part 2) by Venky Deshpande.

Disclaimer: This is not officially supported by VMware, please use at your own risk.

Configurations for VXLAN in vSphere Environment

Step 1 - Deploy vCNS Manager and configure it to point to your vCenter Server (do not enable or prepare VXLAN, this must be done after the configurations)

Step 2 - You will need to identify the VDS MoRef ID in your vCenter Server which will be used in the next step. Since the configuration is applied at the VDS level, you may want to consider having a separate VDS serving Nested Virtualization traffic since both promiscuous mode & forged transmits will automatically be enabled for all vWires. To locate the VDS MoRef ID, login to the vSphere Web Client and select the summary view for the VDS.

The VDS MoRef ID will be towards the end of the URL link and it should start with dvs-X where X is some arbitrary number. Record this value down for the next step

Step 3 - Download the enablePromForVDS.sh shell script which will be used to prepare the VDS within the vCNS Manager. The script basically performs a POST to the REST API to the vCNS Manager using cURL and it accepts three input parameters: vCNS Manager IP Address/Hostname, VDS MoRef ID and VDS MTU. The username/password is hard coded in the script to use the default which is admin/default. If you have modified the default password like any good admin, you will want to change the password before running the script. If you take a look at the request body, you will notice only promiscuous mode is enabled to true, but this will also automatically enable forged transmits as well.

In my lab enviroment, I have the vCNS Manager IP to be 172.30.0.196, VDS MoRef ID to be dvs-13 and VDS MTU to be 9000. So the syntax to run the script would be:

./enablePromForVDS.sh 172.30.0.196 dvs-13 9000

Here is a screenshot of executing the script, you should see a response back with 200 to indicate successful execution of the script.

Step 4 - Now, we will proceed with the VXLAN preparation. Start off by logging into the vCNS Manager and selecting the vSphere Datacenter which you wish to enable VXLAN. On the right you should see a tab called "Network Virtualization" go ahead and click on that and then click on the sub-tab called "Preparation". Click on edit and then select the vSphere Cluster and proceed through the wizard based on your environment configuration.

Step 5 - Once the VXLAN preparation has completed, click on the "Segment ID" and configure that based on your environment.

Step 6 - Next, click on "Network Scopes" and you will create a network scope and specify the set of vSphere Clusters the VXLAN network will span.

Step 7 - Lastly, click on "Networks" and this is where you will create your vWires and ensure it the proper network scope is selected.

Step 8 - To confirm that everything has been configured properly. We now log back into our vSphere Web Client and heading over to the VDS settings page. You should now see a new vWire portgroup that is created, if we take a look at it's settings we should see that both promiscuous mode and forged transmits is enabled.

You are now done with the VXLAN configurations in the vCNS Manager and can proceed to the regular instructions for enabling Nested ESXi for vSphere.

Note: If you have already prepared VXLAN in your environment, you can still configure the above without having to un-prepare your VXLAN configurations. You just need to login to the vCNS Manager via the REST API and perform a DELETE on the VDS switch (Please refer to page 153 of the vCNS API Programming Guide) which will just delete the mapping from vCNS but will not destroy any of your VDS configuration. Once that is done, you will be able to use the script to configure the VDS with the proper settings.

Configurations for VXLAN in vCloud Director Environment

A VXLAN network pool is automatically created for you when using vCloud Director 5.1, so the steps for preparing Nested Virtualization for vCloud Director is extremely simple compared to the vSphere only environment.

Note: VXLAN is only supported in vCloud Director 5.1, for previous versions you have the choice of using a VCD-NI or vSphere backed network and the configurations for that can be found here.

Step 1 - Please follow the steps 1-5 from above in the vSphere only environment and then you are done. If you would like a more detailed walk through for configuring VXLAN for a vCloud Director environment, check out this article by Rawlinson Rivera who takes you through the process step by step.

Step 2 - Proceed to the regular instructions for enabling Nested ESXi for vCloud Director.

Step 3 - Lastly, you will go through the vCloud Director setup which is to attach your vCenter Server & vCNS Manager, create a Provider VDC, create an Organization and assign resources to your Organization VDC and ensure that the OrgVDC is consuming the VXLAN network pool that is automatically created for you when you create the Provider VDC. Once that is done, when you deploy your vApp, you will see a vWire that automatically created for you. If we login to the vSphere Web Client and go to the VDS settings, you will see the vWire has both promiscuous mode and forged transmits automatically enabled.

Additional Resources:

How To Quickly Get Started With The New VMware Puppet Modules

by // 5 Comments

Yesterday, VMware Automation gurus Nick Weaver and Nan Liu just announced the release of four awesome new VMware Puppet modules that can help you manage and configure vCenter Server (including ESXi) and vCloud Networking & Security (vShield). You can read all about the details here and here and if you were lucky enough to have attended PEX (Partner Exchange) you might even have caught the demo given by Nick in his session.

I have used Puppet in the past, but it was pretty limited and specifically in How to Deploy ESXi 5 Using Razor & Puppet. I thought this might be a good time to revisit Puppet and try out the new VMware Puppet modules. I took a look at some of the examples provided by Nan on his blog but for new users to Puppet, it may not provide enough details to quickly get started (including myself). I thought I document the minimal steps I took to quickly get started (I also ran into a few bugs which Nan has fixed).

Step 1 - Install Ubuntu Precise (Ubuntu Server 12.04 LTS - See more at: http://www.virtuallyghetto.com/2012/05/how-to-deploy-esxi-5-using-razor-puppet.html

Step 1 - Install Ubuntu Precise (Ubuntu Server 12.04 LTS). You can use other distros, I just choose Ubuntu as I had the image lying around.

Step 2 - Download Puppet Labs package repository by running the following commands:

wget http://apt.puppetlabs.com/puppetlabs-release-$(lsb_release -c | cut -f 2).deb
dpkg -i puppetlabs-release-$(lsb_release -c | cut -f 2).deb
apt-get update

Step 3 - Install all the necessary packages such as Ruby, Ruby Gems, Puppet, etc. by running the following commands:

apt-get install -y libxslt-dev libxml2-dev ruby rubygems puppet
gem install nokogiri
gem install net-ssh

Step 4 - Install the VMware Puppet modules by running the following command:

puppet module install vmware/vcsa
puppet module install vmware/vcenter
puppet module install vmware/vshield

To start using the VMware Puppet modules, you will need to create what's known as a manifest file that contains the resources which maps to the actions you wish to perform (e.g. configure a newly deployed VCSA appliance or create a Cluster in vCenter Server and add an ESXi host to that cluster). You can find a bunch of example manifest files in each of the Puppet modules, here is the path to each:

/etc/puppet/modules/vcsa/tests/
/etc/puppet/modules/vcenter/tests/
/etc/puppet/modules/vshield/tests/

You will see in some of the examples, they import a file in each directory called data.pp which contains the actual definitions of your VCSA, vCNS and ESXi hosts but you can also just specify that in the main manifest file as well for simplicity. The latter option provides more flexibility as you can easily reference various configurations for different environments. For your convenience, I have created the following manifest files that you can use and you just need to modify them to fit your environment.

Here is what my lab environment looks like and their respective IP Addresses for your reference (these must already be deployed and vCenter & vCNS does not need to be configured but just accessible over network):

vCenter Server = 172.30.0.135
vCloud Networking and Security = 172.30.0.136
ESXi Host = 172.30.0.137

Step 5 - As mentioned by Nan, a custom Rbvmomi was used and we will need to ensure our Puppet management host (Ubuntu system we are on) includes it. To ensure all the necessary packages are downloaded for us, we will use the rbvmomi.pp manifest file for our host and use Puppet to apply the policy. Replace management_server in rbvmomi.pp with the hostname or IP Address of your Ubuntu host and then run the following command:

puppet apply rbvmomi.pp

Note:  You can safely ignore the red warnings, it must not have liked something in my environment.

Step 6 - We will start off by configuring the VCSA so we can then perform operations such as adding in Datacenters, Clusters, ESXi hosts, etc. We will use the configure-vcsa.pp manifest file by running the following command:

puppet apply configure-vcsa.pp

Step 7 - Next we will create a Datacenter, Cluster and add our ESXi host by using the setup-vcenter.pp manifest file by running the following command:

puppet apply setup-vcenter.pp

Step 8 - We are now onto configuring vCloud Networking and Security and we will also associate it with our vCenter Server by using the configure-vcns.pp manifest file and running the following command:

puppet apply configure-vcns.pp

Step 9 - After configuring vCloud Networking and Security, we can now deploy a vCloud Networking and Security Edge Gateway to provide various networking services to our vSphere environment using the deploy-edge.pp and by running the following command:

puppet apply deploy-edge.pp

In about 5-10 minutes, you will have a fully configured vSphere environment that contains your vCenter Server, vCloud Networking and Security Manager and Edge Gateway and ESXi hosts all ready to start providing compute and networking services for your virtual machines and applications! I want to stress the above is a very simplistic example of what you can do with the new VMware Puppet modules. There are definitely more advanced capabilities provided in the modules and I would recommend you take a look in the samples directory of each module for more details.

Overall, I was pretty impressed with the VMware integration that Nick, Nan and team built with Puppet. This was a great learning experience for myself, I learned quite a bit with just trying out these modules and I think I might have found a reason to dive more into Puppet! 🙂

Big thanks to Nan for helping me out with some of my Puppet questions!

Using the New vCloud Networking and Security to Load Balance Multiple vSphere Web Client Servers

by // 1 Comment

If you followed my previous two articles Configuring Additional Windows vSphere Web Client 5.1 Servers & Configuring Additional VCSA 5.1 as vSphere Web Client Servers I demonstrated how easy it is to add additional vSphere Web Client Servers to provide additional connection load balancing as well as redundancy in accessing your vSphere 5.1 environment. The next logical step is to actually place a load balancer in front of all the vSphere Client Servers to provide a single entry point for users to connect to.

I did a quick search for some load balancer solutions for testing, but found that they were quite complex and non-intuitive to setup. Since this was for my home lab, I decided to try out the new vCloud Networking & Security 5.1 (previously known as vShield). This being my first time setting up a load balancer (usually a task given to Network Engineers), I was pleasantly surprised and shocked at how easy it was to deploy and configure a load balancer using the new vCloud Networking and Security. Instead of going through this step by step, I thought I record a quick video.

The video below shows how you can leverage the new vCloud Networking & Security 5.1 to deploy and configure a Load Balancer to load balance multiple vSphere Web Client Servers.

If you have not tried out the new vCloud Networking and Security, I highly recommend you give it a try, especially with all the new network and security features in the latest 5.1 release.