Kenneth Hui recently published a number of interesting articles diving into the latest VMware vSphere integration with the OpenStack Grizzly release called OpenStack For VMware Admins: Nova Compute With vSphere Part1 and Part2. There has definitely been a lot of chatter around OpenStack lately and I agree with Kenneth, there is also a lot of confusion around the topic in general. Although I have not used OpenStack personally, one very important concept to understand is that OpenStack is really just a framework that allows you to build a Cloud solution that is comprised of the best of breed products that can then be plugged into the underlying compute, network, storage and management infrastructure.
One example of this is OpenStack's Nova compute component which supports a variety of Hypervisor solutions including KVM, XEN and now also VMware vSphere. Another example is OpenStack's Neutron (formally Quantum) networking component which also supports a variety of networking platforms including the leader in this space which is VMware's Nicira NVP (Networking Virtualization Platform).
Having said all that, since I have never worked with OpenStack before, I thought this would be a great opportunity to give OpenStack a test run with my vSphere home lab environment. With a quick Google search, I found an OpenStack Wiki guide for setting up VMware's Nova integration and I thought I should be able to just follow that. As it turns out, some of the commands no longer function due to some recent code changes in OpenStack and the instructions were also incomplete for a few steps. With the assistance of the OpenStack development team at VMware, I was able to get everything working and I wanted to share the details while the Wiki gets corrected.
Here is a diagram of what a vSphere and OpenStack solution could look like and we will be primarily focusing on the Nova component:
- vSphere ready environment with vCenter Server and at least one ESXi host (I recommend using the vCenter Server Appliance for quick setup)
- vanilla installation of Ubuntu 12.04 LTS (you can find more details here)
Here is what my vSphere inventory looks like and the nice about this is you can use an existing vSphere environment. As you can see I have my Apple Mac Mini running ESXi, which is also hosting my vCenter Server along with my OpenStack virtual machine.
Step 1 - Install git and we will be using that to clone out the latest DevStack which is basically a huge shell script that helps you quickly stand up an OpenStack instance for testing/development as it is not a trivial task to install OpenStack. Run the following commands on your Ubuntu OpenStack host:
sudo apt-get -y install git
git clone http://github.com/openstack-dev/devstack.git
Step 2 - Next we will need to setup a Tun/Tap interface which can do userspace networking and this helps ensure we do not mess with our primary interface (eth0) that is used to connect to the OpenStack VM. Run the following commands:
sudo ip tuntap add dev tapfoo mode tap
sudo ifconfig tapfoo 172.30.0.1 up
Note: You can select any IP Address that is not being used, I chose 172.30.0.1
To confirm the software interface was created correctly, you can run the ifconfig command and you should see a "tapfoo" interface with the IP Address that you had specified from above.
Step 3 - Now we need to create a file called localrc in the devstack directory with the following configurations listed below which will be used by DevStack to build and configure our OpenStack instance.
The configurations in BLACK are required, where as the ones in GREEN are optional and I will explain those in a bit.
VMWAREAPI_IP is the IP Address of your vCenter Server
VMWAREAPI_PASSWORD is the password of your vCenter Server
VMWAREAPI_CLUSTER is the name of the vSphere Cluster if you have one, else you can leave it blank
HOST_IP is the IP Address of your OpenStack Ubuntu host
SCREEN_LOGDIR will log all the OpenStack logs to a directory of your choice. By default, DevStack will log to standard out and only visible through the Screen sessions of each component which is not very user friendly nor easy for troubleshooting.
If you wish to forward OpenStack logs to a remote syslog host, you can also enable the following three configurations which should be pretty straight forward:
SYSLOG_HOST is the IP Address of your remote syslog host (more details on this towards the bottom)
SYSLOG_PORT is the port of your remote syslog host, default will be 514
Note: If you want to learn about other DevStack localrc options, take a look a the documentation here
Step 4 - We are now ready to build and deploy our OpenStack instance. To start, just run the following command:
This process will take a few minutes depending on how fast your system is and the connection to download all the necessary packages. If everything was successful, you should see a summary about logging into your OpenStack instance and the URL for the Horizon UI as shown in the screenshot below.
Step 5 - Go ahead and confirm you can access the Horizon UI by opening up a browser and pointing it to the IP Address of your OpenStack instance.
Step 6 - To start using OpenStack, we will need to first upload a virtual machine disk to OpenStack's Glance component which handles VM images. There is a sample Debian VMDK that is available on the OpenStack Wiki that we will be downloading to our OpenStack instance. To do so, we will set our credentials on the command-line for the next step and perform a wget to download the VMDK by running the following commands:
source openrc demo demo
Step 7 - We will now use the following Glance CLI to upload our virtual disk and we can also list it once it is uploaded:
glance image-create --name Debian --is-public=True --container-format=bare --disk-format=vmdk --property vmware-disktype="preallocated" < Debian-flat.vmdk
Step 8 - To deploy a new instance of the image we have just uploaded, we will switch over to the Nova CLI and specify the Image ID from the previous step and run the following command which will deploy to our vSphere environment.
nova boot --image --flavor 1 my-first-openstack-vm
DevStack Syslog Configuration
service rsyslog restart
If we login to our vCenter Log Insight UI, we should now see our OpenStack instance logging remotely. Once you unstack and run stack, the configurations will default back to the original.
Can we see your nova.conf file ?
William Lam says
The nova.conf is just the defaults of what DevStack has configured it to. If you follow the steps above, you should get the same
Folks, what is the username/password for the debian-flat.vmdk image?
The article below indicates it should be nicira/nicira. It does not seem to work.. Can someone confirm this please? thanks
Is the vCenter password seriously stored in plain text in a flat file? Is there no way around this?
The username/password is nicira/nicira but from the vcenter console, it has an issue with repeated keystrokes. Follow the steps here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=196
This will take care of the problem
I followed the instructions perfectly. I got a VM up and running and can see it through vCenter client. I also see a private IP address assigned to the VM in Horizon. However, I don't see any IP address assigned to the VM thru vCenter client. Port group is br100.
What am I missing here? Thanks
Jhonny Nemonic says
can you integrate RDO with Vmware as well?, I mean It has Nova on the same node and other modules.
Hi, I followed the instructions but I am using ESXI instead of Vcenter. After I launched the Vm, the vm status is error and I could not find the instance running on Vsphere. do the instructions also apply to ESXI or I need to modify some configuration? thank you~~
William Lam says
As mentioned in the article, you will need vCenter Server for this to work. It will NOT work with just standard ESXi
Hi, now I have vcenter set up, followed the steps again and after i booted a new instance and did a nova list, i found the status of the new created instance is error and i did not see the new booted instance from the vcenter,either. i also can not find any info about vcenter/esxi in the openstack dashboard hypervisor panel. any suggestion for debugging？ how can i know if my nova has successfully hand-shaked with vcenter?
Great write-up, currently using this to prove some stuff to my developers (you CAN use ESXi!). However, I think there's a bit lacking on how to configure guest networking. Following these directions, I think that the 2nd interface on the OpenStack all-in-one VM on the br100 network is missing.
Otherwise, how can the VMs that only have connections on br100 get NAT'd out to public IPs through eth0 on the OpenStack VM?
hi i followed the instructions but was not able to boot the instance, is there any configuration need to be done on the guest networking in order to make the handshake between nova and vcenter successful?
kesten broughton says
This might be an easier option, with fewer dependencies
i followed the steps you have outlined and i was succesfully able to launch a vm and vm is also provisioned in the esx, checking the same by connecting to the vncenter. but when i try to access the logs tab and console tab associated with vm from the openstack dashboard. i get the following error.
Unable to get log for instance "e32d5cc4-537e-4d8a-9ec3-100edf186689".
console is currently unavailable. Please try again later. Reload
can you tell me what could be the possible reason? what am i missing here.
Even I am facing the same problem can any of you guys tell us what has to be done to login to instance using Openstack console. I have configured security group to talk on port 22.
hello! I want to know how openstack get vsphere remote console ,use VMRC api or vsphere api ?
Followed your doc here but I get stuck at running stack.sh ... It complains the n-cpu is not running.. and in the ncpu.log I see this
"lo_vmware.api._create_session" :: held 0.014s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lo
2015-09-02 13:33:04.141 54514 CRITICAL nova [-] SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
So it appears that nova is not accepting the SSL cert from the VCSA [vcenter appliance].
is it possible to install openstack on 3 vm nods
3 nodes which comes from vm workstation in windows enviornment
Hello is it possible on vmware essential plus lic kit.
Hi, is it possible to update repos?
[ERROR] /home/gregor/devstack/functions-common:1058 Failed to update apt repos, we're dead now