WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple
You are here: Home / Uncategorized / How to Quickly Get Started with VMware vSphere & OpenStack?

How to Quickly Get Started with VMware vSphere & OpenStack?

07.01.2013 by William Lam // 20 Comments

Kenneth Hui recently published a number of interesting articles diving into the latest VMware vSphere integration with the OpenStack Grizzly release called OpenStack For VMware Admins: Nova Compute With vSphere Part1 and Part2. There has definitely been a lot of chatter around OpenStack lately and I agree with Kenneth, there is also a lot of confusion around the topic in general. Although I have not used OpenStack personally, one very important concept to understand is that OpenStack is really just a framework that allows you to build a Cloud solution that is comprised of the best of breed products that can then be plugged into the underlying compute, network, storage and management infrastructure.

One example of this is OpenStack's Nova compute component which supports a variety of Hypervisor solutions including KVM, XEN and now also VMware vSphere. Another example is OpenStack's Neutron (formally Quantum) networking component which also supports a variety of networking platforms including the leader in this space which is VMware's Nicira NVP (Networking Virtualization Platform).

Having said all that, since I have never worked with OpenStack before, I thought this would be a great opportunity to give OpenStack a test run with my vSphere home lab environment. With a quick Google search, I found an OpenStack Wiki guide for setting up VMware's Nova integration and I thought I should be able to just follow that. As it turns out, some of the commands no longer function due to some recent code changes in OpenStack and the instructions were also incomplete for a few steps. With the assistance of the OpenStack development team at VMware, I was able to get everything working and I wanted to share the details while the Wiki gets corrected.

Here is a diagram of what a vSphere and OpenStack solution could look like and we will be primarily focusing on the Nova component:

Pre-requesites:

  • vSphere ready environment with vCenter Server and at least one ESXi host (I recommend using the vCenter Server Appliance for quick setup)
  • vanilla installation of Ubuntu 12.04 LTS (you can find more details here)

Here is what my vSphere inventory looks like and the nice about this is you can use an existing vSphere environment. As you can see I have my Apple Mac Mini running ESXi, which is also hosting my vCenter Server along with my OpenStack virtual machine.

Installation:

Step 1 - Install git and we will be using that to clone out the latest DevStack which is basically a huge shell script that helps you quickly stand up an OpenStack instance for testing/development as it is not a trivial task to install OpenStack. Run the following commands on your Ubuntu OpenStack host:

sudo apt-get -y install git
git clone http://github.com/openstack-dev/devstack.git
cd devstack

Step 2 - Next we will need to setup a Tun/Tap interface which can do userspace networking and this helps ensure we do not mess with our primary interface (eth0) that is used to connect to the OpenStack VM. Run the following commands:

sudo ip tuntap add dev tapfoo mode tap
sudo ifconfig tapfoo 172.30.0.1 up

Note: You can select any IP Address that is not being used, I chose 172.30.0.1

To confirm the software interface was created correctly, you can run the ifconfig command and you should see a "tapfoo" interface with the IP Address that you had specified from above.

Step 3 - Now we need to create a file called localrc in the devstack directory with the following configurations listed below which will be used by DevStack to build and configure our OpenStack instance.

ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-cpu,n-net,n-cond,n-sch,rabbit,mysql,horizon
VIRT_DRIVER=vsphere
VMWAREAPI_IP=192.168.1.127
VMWAREAPI_USER=root
VMWAREAPI_PASSWORD=vmware
VMWAREAPI_CLUSTER=Cluster
DATABASE_PASSWORD=nova
RABBIT_PASSWORD=nova
SERVICE_TOKEN=nova
SERVICE_PASSWORD=nova
ADMIN_PASSWORD=nova
FLAT_INTERFACE=tapfoo
HOST_IP=192.168.1.143
SCREEN_LOGDIR=/home/primp/devstack-logs
SYSLOG=True
SYSLOG_HOST=192.168.1.104
SYSLOG_PORT=514

The configurations in BLACK are required, where as the ones in GREEN are optional and I will explain those in a bit.

VMWAREAPI_IP is the IP Address of your vCenter Server
VMWAREAPI_PASSWORD is the password of your vCenter Server
VMWAREAPI_CLUSTER is the name of the vSphere Cluster if you have one, else you can leave it blank
HOST_IP is the IP Address of your OpenStack Ubuntu host

Optional configurations:

SCREEN_LOGDIR will log all the OpenStack logs to a directory of your choice. By default, DevStack will log to standard out and only visible through the Screen sessions of each component which is not very user friendly nor easy for troubleshooting.

If you wish to forward OpenStack logs to a remote syslog host, you can also enable the following three configurations which should be pretty straight forward:

SYSLOG=True
SYSLOG_HOST is the IP Address of your remote syslog host (more details on this towards the bottom)
SYSLOG_PORT is the port of your remote syslog host, default will be 514

Note: If you want to learn about other DevStack localrc options, take a look a the documentation here

Step 4 - We are now ready to build and deploy our OpenStack instance. To start, just run the following command:

./stack.sh

This process will take a few minutes depending on how fast your system is and the connection to download all the necessary packages. If everything was successful, you should see a summary about logging into your OpenStack instance and the URL for the Horizon UI as shown in the screenshot below.

Step 5 - Go ahead and confirm you can access the Horizon UI by opening up a browser and pointing it to the IP Address of your OpenStack instance.

Step 6 - To start using OpenStack, we will need to first upload a virtual machine disk to OpenStack's Glance component which handles VM images. There is a sample Debian VMDK that is available on the OpenStack Wiki that we will be downloading to our OpenStack instance. To do so, we will set our credentials on the command-line for the next step and perform a wget to download the VMDK by running the following commands:

source openrc demo demo
wget https://www.dropbox.com/s/utvri5bw3zztty6/Debian-flat.vmdk

Step 7 - We will now use the following Glance CLI to upload our virtual disk and we can also list it once it is uploaded:

glance image-create --name Debian --is-public=True --container-format=bare --disk-format=vmdk --property vmware-disktype="preallocated" < Debian-flat.vmdk
glance image-list

Step 8 - To deploy a new instance of the image we have just uploaded, we will switch over to the Nova CLI and specify the Image ID from the previous step and run the following command which will deploy to our vSphere environment.

nova boot --image --flavor 1 my-first-openstack-vm
nova list

 
Step 9 - We can continue to run "nova list" to view the status, but it would be more interesting to see this from the Horizon UI. You can head over to the OpenStack UI and see the progress under the Instances tab.
Once the VM is ready, we should see an IP Address assignment and the status set to ready and the VM should show powered on.
To confirm that we have actually provisioned the VM onto our vSphere compute cluster, we can login to either the vSphere Web Client or vSphere C# Client and we should see our newly deployed VM running.
If you wish to deploy using the Horizon UI, you can go to Project -> Instances -> Launch Instance and go through the wizard selecting the image, specifying a name and configuration flavor and then click on Launch once you are ready to deploy.
Step 10 - Once you are finished, you can run the ./unstack.sh command which will reset and clean up your environment and delete any images that were uploaded. Again, DevStack is not meant for running production workloads, but can be used for quickly testing or developing against OpenStack. You can also view the consoles of each of the OpenStack components by using screen -x stack.
Using DevStack, you can quickly get a basic OpenStack instance up and running without too much hassle but this is not to say that OpenStack is easy or trivial to install. If this is your first time, I would highly recommend configuring your localrc to store the logs in a directory so you can either go through them if you run into any issues or more likely forward it over to an OpenStack expert to help you decipher. I personally had ran into a few issues and it was partially due to some errors in the Wiki, but troubleshooting can be like search for a needle in a haystack.

DevStack Syslog Configuration

If you recall earlier in the localrc configuration, there is a section that specifies remote syslog configurations for the OpenStack instance. Since I am a fan of the new vCenter Log Insight product that was just released as a beta from VMware, I thought it would be neat to forward the OpenStack logs to it. After a bit of trial and error, it turns out that DevStack configures rsyslog (which is the syslog daemon) running on the Ubuntu host to forward logs using RELP format which is not supported by vCenter Log Insight. If you want to get this working, you will need to disable RELP format by tweaking the rsyslog configuration in /etc/rsyslog.d/90-stack-s.conf
You will need to replace :omrelp:

*.*             :omrelp:192.168.1.104:514

to just @@:

*.*             @@192.168.1.104:514

Finally, you need to restart the rsyslog service for the changes to take effect by running the following command:

service rsyslog restart

If we login to our vCenter Log Insight UI, we should now see our OpenStack instance logging remotely. Once you unstack and run stack, the configurations will default back to the original.

Additional Resources:

  • vSphere + OpenStack Nova wiki guide
  • OpenStack CLI reference
  • Screen command reference

More from my site

  • Community stories of VMware & Apple OS X in Production: Part 2
  • Community stories of VMware & Apple OS X in Production: Part 1
  • A Hidden vSphere 5.1 Gem - Forwarding Virtual Machine Logs (vmware.log) to Syslog Part 2
  • A Hidden vSphere 5.1 Gem - Forwarding Virtual Machine Logs (vmware.log) to Syslog Part 1
  • Forwarding Logs From The vCloud Suite To vCenter Log Insight

Categories // Uncategorized Tags // DevStack, nova, OpenStack, SDDC, software defined datacenter, vC Log, vCenter Log Insight, vmware, vSphere

Comments

  1. *protectedAnonymous says

    07/09/2013 at 6:29 am

    Can we see your nova.conf file ?

    Reply
    • *protectedWilliam Lam says

      07/09/2013 at 2:01 pm

      The nova.conf is just the defaults of what DevStack has configured it to. If you follow the steps above, you should get the same

      Reply
  2. *protectedAnonymous says

    07/14/2013 at 10:58 pm

    Folks, what is the username/password for the debian-flat.vmdk image?
    The article below indicates it should be nicira/nicira. It does not seem to work.. Can someone confirm this please? thanks
    https://wiki.openstack.org/wiki/NovaVMware/DeveloperGuide

    Reply
  3. *protectedAnonymous says

    07/23/2013 at 9:12 pm

    Is the vCenter password seriously stored in plain text in a flat file? Is there no way around this?

    Reply
  4. *protectedAnonymous says

    07/26/2013 at 9:51 pm

    The username/password is nicira/nicira but from the vcenter console, it has an issue with repeated keystrokes. Follow the steps here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=196

    This will take care of the problem

    Reply
  5. *protectedAnonymous says

    07/27/2013 at 6:16 am

    Hi there
    I followed the instructions perfectly. I got a VM up and running and can see it through vCenter client. I also see a private IP address assigned to the VM in Horizon. However, I don't see any IP address assigned to the VM thru vCenter client. Port group is br100.
    What am I missing here? Thanks

    Reply
  6. *protectedJhonny Nemonic says

    08/11/2013 at 1:56 pm

    Hails William,

    can you integrate RDO with Vmware as well?, I mean It has Nova on the same node and other modules.

    cheers!

    Reply
  7. *protectedAnonymous says

    09/06/2013 at 9:30 pm

    Hi, I followed the instructions but I am using ESXI instead of Vcenter. After I launched the Vm, the vm status is error and I could not find the instance running on Vsphere. do the instructions also apply to ESXI or I need to modify some configuration? thank you~~

    Reply
    • *protectedWilliam Lam says

      09/07/2013 at 3:20 pm

      As mentioned in the article, you will need vCenter Server for this to work. It will NOT work with just standard ESXi

      Reply
    • *protectedAnonymous says

      09/12/2013 at 8:57 pm

      Hi, now I have vcenter set up, followed the steps again and after i booted a new instance and did a nova list, i found the status of the new created instance is error and i did not see the new booted instance from the vcenter,either. i also can not find any info about vcenter/esxi in the openstack dashboard hypervisor panel. any suggestion for debugging? how can i know if my nova has successfully hand-shaked with vcenter?

      Reply
  8. *protectedMike says

    09/12/2013 at 8:48 pm

    Great write-up, currently using this to prove some stuff to my developers (you CAN use ESXi!). However, I think there's a bit lacking on how to configure guest networking. Following these directions, I think that the 2nd interface on the OpenStack all-in-one VM on the br100 network is missing.

    Otherwise, how can the VMs that only have connections on br100 get NAT'd out to public IPs through eth0 on the OpenStack VM?

    Reply
    • *protectedAnonymous says

      09/12/2013 at 10:43 pm

      hi i followed the instructions but was not able to boot the instance, is there any configuration need to be done on the guest networking in order to make the handshake between nova and vcenter successful?

      Reply
  9. *protectedkesten broughton says

    09/13/2013 at 3:01 am

    This might be an easier option, with fewer dependencies
    http://savanna.readthedocs.org/en/0.2.1/devref/devstack.html

    Reply
  10. *protectedyamini says

    04/13/2014 at 4:11 pm

    Hi Willam,

    i followed the steps you have outlined and i was succesfully able to launch a vm and vm is also provisioned in the esx, checking the same by connecting to the vncenter. but when i try to access the logs tab and console tab associated with vm from the openstack dashboard. i get the following error.

    Unable to get log for instance "e32d5cc4-537e-4d8a-9ec3-100edf186689".

    Instance Console

    console is currently unavailable. Please try again later. Reload

    can you tell me what could be the possible reason? what am i missing here.

    Reply
    • *protectedDivya says

      05/08/2014 at 9:51 am

      Even I am facing the same problem can any of you guys tell us what has to be done to login to instance using Openstack console. I have configured security group to talk on port 22.

      Reply
  11. *protected张建峰 says

    05/29/2014 at 10:07 am

    hello! I want to know how openstack get vsphere remote console ,use VMRC api or vsphere api ?

    Reply
  12. *protectedTim says

    09/02/2015 at 8:53 pm

    Followed your doc here but I get stuck at running stack.sh ... It complains the n-cpu is not running.. and in the ncpu.log I see this

    "lo_vmware.api._create_session" :: held 0.014s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lo
    ckutils.py:265
    2015-09-02 13:33:04.141 54514 CRITICAL nova [-] SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
    failed"

    So it appears that nova is not accepting the SSL cert from the VCSA [vcenter appliance].

    Reply
  13. *protectedamirali says

    09/06/2015 at 4:58 pm

    hi
    is it possible to install openstack on 3 vm nods
    3 nodes which comes from vm workstation in windows enviornment

    Reply
  14. *protectedjayantchutkejayant says

    10/07/2015 at 1:06 pm

    Hello is it possible on vmware essential plus lic kit.

    Reply
  15. *protectedGregor says

    11/11/2018 at 11:01 pm

    Hi, is it possible to update repos?
    [ERROR] /home/gregor/devstack/functions-common:1058 Failed to update apt repos, we're dead now

    Reply

Thanks for the comment!Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025