WilliamLam.com

VMware Labs releases Proactive DRS Fling from last years Fling Contest

by // 1 Comment

Last year, the VMware Labs team ran an Open Innovation contest where users could submit ideas for Flings that they would like to see get built. In addition to that, one lucky winner will be selected and their idea will actually be sponsored internally within VMware R&D and VMware Engineers would then implement that idea. The winner of that contest was Mike Preston on an idea called Proactive DRS:

“What I would like to see is some sort of appliance/script that can hook into both the vCenter Operations API’s as well as the vSphere APIs and merge these two technologies together. By interpreting vCenter Operations predictions for what is going to happen within your environment and then leveraging vMotion/DRS/DPM to prepare for this before it happens we could be left with a more proactive approach. (I.E. Historically VM1 will utilize 100% CPU at 4am in the morning, let’s be sure these resources are available on the host at 3:45 by migrating other VMs off, rather than waiting for DRS to kick in at 4:05 – At 6am everyday my workload normally increases to the point where DPM kicks in and turns on some hosts, let’s turn these on at based on the vCenter Operations stats rather than having to manually configure a setting to do so).”

Today at the VMware R&D Innovation booth at VMworld, the VMware Labs team has just announced that they will be releasing the Proactive DRS Fling (download coming very soon!). I think this is pretty awesome, I mean how cool is that, an idea that you came up with was implemented by a group of VMware Engineers? Congrats on the awesome idea Mike!

Here is a bit more details on how Proactive DRS works:  

It is a way for DRS to react to changes in the virtual cluster, and to act on predicted changes in resource demands before hosts become stressed. For example, if you have a VM that historically uses 100% CPU at 8am every morning, ProactiveDRS  makes  sure that the CPU resources will be available for that VM before 8am. These actions ensure that your cluster runs smoother and reduces the amount of reactive VM rebalances that occur.

Features

  • Given an advance forecast/prediction of each VM’s resource demands, e.g., CPU and Memory, ProactiveDRS seeks to reorganize the placement of VMs to best balance the current resource demands while taking early actions to accommodate the future/predicted resource demands.
  • Sample actions include, but are not limited to, proactive vMotions (migrations) and proactively powering on a new physical host to accept VMs whose resource demands are expected to spike beyond the available resources of their current physical host.

The VMware Labs folks did not stop there, they are also running a new 2013 Fling Contest and if you are at VMworld, you can drop by the VMware R&D Innovation booth which is behind the main VMware booth to submit your ideas. Another lucky winner will be selected and their idea will also be built by VMware Engineers which includes a free ticket to VMworld 2014. I recommend you either drop by the booth if you are at VMworld or go online to submit an idea and you could be the next person with their very own Fling!

VMware Software Defined Kit

by // 10 Comments

A hobby of mine when I am not playing with VMware technologies and such is riding on my road bike. I was pleasently surprise to find out this week that my VMware cycling kit has finally arrived! The cycling kit was designed by small group of VMware employees in their free time and Scott Jobe has been managing all the orders and logistics for the past few years. When I had first joined VMware and heard about this, I had just missed the deadline for submitting my order. This year, I made sure to keep an eye out once the new kits were available for order. With a holiday this week, I took my new kit out for a spin and it fits and looks great. I even dropped by our office for a quick picture 🙂

I would like to thank Scott and the team for putting all of this together, I know it took a lot of time including the re-makes, but I am sure everyone is enjoying their new VMware kits! Thank you!

Here are some additional pictures of the items I had purchased. All the clothing were custom printed by Louis Garneau.

Jersey (front & back)

Vest (front & back)

  Bib (front & back)

Arm Warmers & Gloves

How to Quickly Get Started with VMware vSphere & OpenStack?

by // 20 Comments

Kenneth Hui recently published a number of interesting articles diving into the latest VMware vSphere integration with the OpenStack Grizzly release called OpenStack For VMware Admins: Nova Compute With vSphere Part1 and Part2. There has definitely been a lot of chatter around OpenStack lately and I agree with Kenneth, there is also a lot of confusion around the topic in general. Although I have not used OpenStack personally, one very important concept to understand is that OpenStack is really just a framework that allows you to build a Cloud solution that is comprised of the best of breed products that can then be plugged into the underlying compute, network, storage and management infrastructure.

One example of this is OpenStack's Nova compute component which supports a variety of Hypervisor solutions including KVM, XEN and now also VMware vSphere. Another example is OpenStack's Neutron (formally Quantum) networking component which also supports a variety of networking platforms including the leader in this space which is VMware's Nicira NVP (Networking Virtualization Platform).

Having said all that, since I have never worked with OpenStack before, I thought this would be a great opportunity to give OpenStack a test run with my vSphere home lab environment. With a quick Google search, I found an OpenStack Wiki guide for setting up VMware's Nova integration and I thought I should be able to just follow that. As it turns out, some of the commands no longer function due to some recent code changes in OpenStack and the instructions were also incomplete for a few steps. With the assistance of the OpenStack development team at VMware, I was able to get everything working and I wanted to share the details while the Wiki gets corrected.

Here is a diagram of what a vSphere and OpenStack solution could look like and we will be primarily focusing on the Nova component:

Pre-requesites:

  • vSphere ready environment with vCenter Server and at least one ESXi host (I recommend using the vCenter Server Appliance for quick setup)
  • vanilla installation of Ubuntu 12.04 LTS (you can find more details here)

Here is what my vSphere inventory looks like and the nice about this is you can use an existing vSphere environment. As you can see I have my Apple Mac Mini running ESXi, which is also hosting my vCenter Server along with my OpenStack virtual machine.

Installation:

Step 1 - Install git and we will be using that to clone out the latest DevStack which is basically a huge shell script that helps you quickly stand up an OpenStack instance for testing/development as it is not a trivial task to install OpenStack. Run the following commands on your Ubuntu OpenStack host:

sudo apt-get -y install git
git clone http://github.com/openstack-dev/devstack.git
cd devstack

Step 2 - Next we will need to setup a Tun/Tap interface which can do userspace networking and this helps ensure we do not mess with our primary interface (eth0) that is used to connect to the OpenStack VM. Run the following commands:

sudo ip tuntap add dev tapfoo mode tap
sudo ifconfig tapfoo 172.30.0.1 up

Note: You can select any IP Address that is not being used, I chose 172.30.0.1

To confirm the software interface was created correctly, you can run the ifconfig command and you should see a "tapfoo" interface with the IP Address that you had specified from above.

Step 3 - Now we need to create a file called localrc in the devstack directory with the following configurations listed below which will be used by DevStack to build and configure our OpenStack instance.

ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-cpu,n-net,n-cond,n-sch,rabbit,mysql,horizon
VIRT_DRIVER=vsphere
VMWAREAPI_IP=192.168.1.127
VMWAREAPI_USER=root
VMWAREAPI_PASSWORD=vmware
VMWAREAPI_CLUSTER=Cluster
DATABASE_PASSWORD=nova
RABBIT_PASSWORD=nova
SERVICE_TOKEN=nova
SERVICE_PASSWORD=nova
ADMIN_PASSWORD=nova
FLAT_INTERFACE=tapfoo
HOST_IP=192.168.1.143
SCREEN_LOGDIR=/home/primp/devstack-logs
SYSLOG=True
SYSLOG_HOST=192.168.1.104
SYSLOG_PORT=514

The configurations in BLACK are required, where as the ones in GREEN are optional and I will explain those in a bit.

VMWAREAPI_IP is the IP Address of your vCenter Server
VMWAREAPI_PASSWORD is the password of your vCenter Server
VMWAREAPI_CLUSTER is the name of the vSphere Cluster if you have one, else you can leave it blank
HOST_IP is the IP Address of your OpenStack Ubuntu host

Optional configurations:

SCREEN_LOGDIR will log all the OpenStack logs to a directory of your choice. By default, DevStack will log to standard out and only visible through the Screen sessions of each component which is not very user friendly nor easy for troubleshooting.

If you wish to forward OpenStack logs to a remote syslog host, you can also enable the following three configurations which should be pretty straight forward:

SYSLOG=True
SYSLOG_HOST is the IP Address of your remote syslog host (more details on this towards the bottom)
SYSLOG_PORT is the port of your remote syslog host, default will be 514

Note: If you want to learn about other DevStack localrc options, take a look a the documentation here

Step 4 - We are now ready to build and deploy our OpenStack instance. To start, just run the following command:

./stack.sh

This process will take a few minutes depending on how fast your system is and the connection to download all the necessary packages. If everything was successful, you should see a summary about logging into your OpenStack instance and the URL for the Horizon UI as shown in the screenshot below.

Step 5 - Go ahead and confirm you can access the Horizon UI by opening up a browser and pointing it to the IP Address of your OpenStack instance.

Step 6 - To start using OpenStack, we will need to first upload a virtual machine disk to OpenStack's Glance component which handles VM images. There is a sample Debian VMDK that is available on the OpenStack Wiki that we will be downloading to our OpenStack instance. To do so, we will set our credentials on the command-line for the next step and perform a wget to download the VMDK by running the following commands:

source openrc demo demo
wget https://www.dropbox.com/s/utvri5bw3zztty6/Debian-flat.vmdk

Step 7 - We will now use the following Glance CLI to upload our virtual disk and we can also list it once it is uploaded:

glance image-create --name Debian --is-public=True --container-format=bare --disk-format=vmdk --property vmware-disktype="preallocated" < Debian-flat.vmdk
glance image-list

Step 8 - To deploy a new instance of the image we have just uploaded, we will switch over to the Nova CLI and specify the Image ID from the previous step and run the following command which will deploy to our vSphere environment.

nova boot --image --flavor 1 my-first-openstack-vm
nova list

Step 9 - We can continue to run "nova list" to view the status, but it would be more interesting to see this from the Horizon UI. You can head over to the OpenStack UI and see the progress under the Instances tab.
Once the VM is ready, we should see an IP Address assignment and the status set to ready and the VM should show powered on.
To confirm that we have actually provisioned the VM onto our vSphere compute cluster, we can login to either the vSphere Web Client or vSphere C# Client and we should see our newly deployed VM running.
If you wish to deploy using the Horizon UI, you can go to Project -> Instances -> Launch Instance and go through the wizard selecting the image, specifying a name and configuration flavor and then click on Launch once you are ready to deploy.
Step 10 - Once you are finished, you can run the ./unstack.sh command which will reset and clean up your environment and delete any images that were uploaded. Again, DevStack is not meant for running production workloads, but can be used for quickly testing or developing against OpenStack. You can also view the consoles of each of the OpenStack components by using screen -x stack.
Using DevStack, you can quickly get a basic OpenStack instance up and running without too much hassle but this is not to say that OpenStack is easy or trivial to install. If this is your first time, I would highly recommend configuring your localrc to store the logs in a directory so you can either go through them if you run into any issues or more likely forward it over to an OpenStack expert to help you decipher. I personally had ran into a few issues and it was partially due to some errors in the Wiki, but troubleshooting can be like search for a needle in a haystack.

DevStack Syslog Configuration

If you recall earlier in the localrc configuration, there is a section that specifies remote syslog configurations for the OpenStack instance. Since I am a fan of the new vCenter Log Insight product that was just released as a beta from VMware, I thought it would be neat to forward the OpenStack logs to it. After a bit of trial and error, it turns out that DevStack configures rsyslog (which is the syslog daemon) running on the Ubuntu host to forward logs using RELP format which is not supported by vCenter Log Insight. If you want to get this working, you will need to disable RELP format by tweaking the rsyslog configuration in /etc/rsyslog.d/90-stack-s.conf
You will need to replace :omrelp:

*.*             :omrelp:192.168.1.104:514

to just @@:

*.*             @@192.168.1.104:514

Finally, you need to restart the rsyslog service for the changes to take effect by running the following command:

service rsyslog restart

If we login to our vCenter Log Insight UI, we should now see our OpenStack instance logging remotely. Once you unstack and run stack, the configurations will default back to the original.

Additional Resources: