WilliamLam.com

Cool Undocumented Features in vCloud Director 1.5

by // 6 Comments

While working on the updated script in Automating vCloud Director 1.5 & Oracle DB Installation, I did some digging in my lab deployment and noticed a few interesting things about the new vCloud Director 1.5 installation.

The first thing I noticed after configuring a new Provider vDC and the vCloud Agent (stored in /opt/vmware/vcloud-director/agent) is pushed out to the ESXi 5 hosts, a new esxcli module is added for vCloud Director under /usr/lib/vmware/esxcli-vcloud

There are 6 namespaces that ranges from simple configuration query, network fence management, account manage and also something called "esxvm" which I'll go into a little bit later. I am not sure why this is not in the vCloud Director documentation, I was not able to find any reference to the new esxcli operations. You may also notice the use of legacy "vslauser" (Virtual Software Lifecycle Automation) throughout vCloud Director, even though it was re-written from the ground up, it looks like VMware decided to either keep the name or some of the code related to the service account.

Here is an example of running "esxcli vcloud about get" command:

Here is an example of running "esxcli vcloud fence getfenceinfo" command:

Lastly, here is an example of what "esxvm" namespace provides:

As you can see above, there are two operations: disable/enable support for 64-bit nested virtual machines. This is exactly the same configuration as I blogged about in How to Enable Support for Nested 64bit & Hyper-V VMs in vSphere 5 but using esxcli interface with vCloud Director 1.5. Let's take a look at what happens when we run the "enable64bitnested" operation.

No surprise, we see that it automatically appends the required vhv.allow = "TRUE" flag which enables the support of running nested 64-bit virtual machines within a physical ESXi 5 host.

You might be asking, why is this in vCloud Director? Well if you attended VMworld 2011 or previous VMworlds and took part in the hands on labs, you will know that VMware utilizes vPods or nested ESXi to deploy their labs. I suspect, this functionality was added into vCloud Director so that VMware can easily leverage nested ESXi for hands on labs or vSel deployments just like they did with Lab Manager previously.

While look into this, I recall a very interesting article by Jason Boche - Deploy ESX & ESXi With Hidden Lab Manager 4 Switch in which Jason identifies a hidden flag in the Lab Manager database that enables a special feature in deploying nested ESX(i) VMs including customization through the use of a special version of VMware Tools for ESX(i). I was curious to see if something similar existed in the new vCloud Director that provided similar functionality.

Looking at the SQL install scripts located in /opt/vmware/vcloud-director/db/{oracle/mssql}, I noticed an interesting config called "extension.esxvm.enabled" in NewInstall_Data.sql file

As you can see from the insert statement, by default this value is set to "false" and we can also confirm this after vCloud Director has been installed and configured by querying the database. Let's go ahead and update this value to "true" and let's see what happens. 

Once you have verified the value has been successfully updated, I decide to use the same trick that Jason had identified with the special "Uber Admin Screen" to load the changes. To my surprise, the trick still worked but the page was not super Uber .... To enable the screen, you will need to click on the "About" page and then click CTRL+U (ctr + shift + u), which will toggle the "Uber Admin Screen".

The available options are quite limited as you can see but there are some new hidden options such as a new debug and console toggle. When you enable these options, you will see them at the bottom right of your screen including a counter of the amount of memory being used for your vCloud Director deployment.

After toggling the hidden database feature, I was not able to see any additional pages relating to nested ESXi hosts, even after restarting vCloud Director. Through some testing, I found that the "extension.esxvm.enabled" actually controlled whether or not nested 64bit VM was enabled when the vCloud Agent was pushed out to ESXi 5 hosts. Instead of manually adding vhv.allow = "TRUE" or using esxcli vcloud esxvm enable64bitnested, vCloud Director will automatically configure the ESXi hosts for you. I still suspect there is probably a hidden interface in managing vESXi hosts and leveraging a specialized version of VMware Tools to automate the deployment of nested ESXi, but I have not found out yet.

UPDATE: Take a look at this blog post for the full details on building your own vSEL - The Missing Piece In Creating Your Own Ghetto vSEL Cloud

vi-fastpass esxcli and resxtop bug resolved in vMA 5

by // 2 Comments

Awhile back I wrote about an resxtop bug found in vMA 4.1 in which it no longer functions with vMA's vi-fastpass component and still requires you to provide the username and password even though vi-fastpass has been initialized for a given target. There was also a slight quirk when using esxcli and vi-fastpass, in which you had to specify in addition the --server of your ESX(i) host which allowed you to utilize vi-fastpass.

With the latest release of vMA 5, both of these issues have now been resolved for both ESXi 5 and ESX(i) 4.x. I would highly recommend you download the latest version if you would like to make use the vi-fastpass component in vMA.

Here is an example of using vi-fastpass with resxtop:

vi-admin@vma50-1:~> vifptarget -s himalaya.primp-industries.com
vi-admin@vma50-1:~[himalaya.primp-industries.com]> resxtop

Here is an example of using vi-fastpass with esxcli:

vi-admin@vma50-1:~> vifptarget -s himalaya.primp-industries.com
vi-admin@vma50-1:~[himalaya.primp-industries.com]> esxcli

Major Enhancements in esxcli for vSphere 5

by // 10 Comments

esxcli in vSphere 5 has undergone significant updates compared to version in vSphere 4.1 which only had 6 major namespaces and 43 commands.

Namespace Description
corestorage VMware core storage commands
network VMware networking commands
nmp VMware Native Multipath Plugin (NMP) This is the VMware default implementation of the Pluggable Storage Architecture
swiscsi VMware iSCSI commands
vaai Vaai Namespace containing vaai code
vms Limited Operations on Virtual Machines

With latest vSphere 5 release, esxcli now includes a total of 10 namespaces and 251 commands! You will notice some of the updated namespaces from vSphere 4.1, such as corestorage which is under the namespace storage and it also contains other sub-namespaces dealing with the storage stack.

Namespace Description
esxcli Commands that operate on the esxcli system itself allowing users to get additional information
fcoe VMware FCOE commands
hardware VMKernel hardware properties and commands for configuring hardware
iscsi VMware iSCSI commands
license Operations pertaining to the licensing of vmware and third party modules on the ESX host. These operations currently only include updating third party module licenses
network Operations that pertain to the maintenance of networking on an ESX host. This includes a wide variety of commands to manipulate virtual networking components (vswitch, portgroup, etc) as well as local host IP, DNS and general hsot networking settings
software Manage the ESXi software image and packages
storage VMware storage commands
system VMKernel system properties and commands for configuring properties of the kernel core system
vm A small number of operations that allow a user to Control Virtual Machine operations

To get a complete list of the available esxcli commands in vSphere 5, you can run the following:

esxcli esxcli command list

The goal of esxcli is to provide a centralize, consistent and easy way of accessing and managing your ESXi host from both locally within ESXi Shell or remotely using vCLI and/or vMA. With the release of 5.0, the majority of the legacy esxcfg-*/vicfg-* commands have been migrated over to esxcli. At some point, hopefully not in the distant future, esxcli will be parity complete and the esxcfg-*/vicfg-* commands will be completely deprecated and removed including the esxupdate/vihostupdate utilities.

There are still several types of operations that require tools in addition to esxcli such as configuring licenses for standalone host, enabling/disabling local and remote ESXi Shell, enabling/disabling Fault Tolerance & vMotion traffic types just to name a few. For more details on some of the common operations that you may need to perform using both esxcli and other utilities, take a look at my ESXi 5 kickstart tip and tricks post.

The new remote version of esxcli is compatible with both ESXi 5 and ESX(i) 4.1 hosts. Due to the nature of the framework, the capabilities are located on the host and when you initially connect, it will automatically down the available namespaces that can be supported. The new namespaces in ESXi 5 will not be available on 4.1 but the old 4.1 namespaces will be. In the latest release, there are still no vSphere APIs exposed for esxcli, they are all still hidden. To access esxcli, you will need to either use local esxcli, remote esxcli using vCLI and/or vMA or PowerCLI's esxcli cmdlets.

The syntax of esxcli is very simple to understand:

To get more details on a namespace, you just need to specify the --help option after the namespace selection. In this diagram, we are accessing "network" namespace and we can see there are 5 additional namespaces we can access.

Let's say we're interested in the "firewall" and would like to know more, we can again specify --help option again.

Now we have a set of commands, let's go ahead and run the "get" command

As you might have guessed, the "get" command will get the network firewall status. The actual commands are pretty intuitive and you will find that most esxcli namespaces will support get, set and list operations. The other nice thing about esxcli is that the output is consistent whether you are running this locally on ESXi Shell or remotely using vCLI and/or vMA, which makes scripting extremely simple.

Note: With remote esxcli, you will need to specify the ESX(i) host and credentials to login to as it is going through the APIs.

You also have the ability to control the type of output that is generated whether that is xml, csv or keyvalue. To do so, you can specify --formatter option and specify one of those options. The xml and keyvalue pair works across all namespaces and csv works on majority of the namespaces.

Let's go ahead and run the same command as above but output it in keyvalue pair

~ # esxcli --formatter=keyvalue network firewall get
Firewall.DefaultAction.string=DROP
Firewall.Enabled.boolean=false
Firewall.Loaded.boolean=true

In addition to type of format output, you can also display specific columns of information using --format-param, this is useful in-conjunction with csv output.

Here is an example of listing the standard vSwitches without any formatting:

~ # esxcli network vswitch standard list
vSwitch0
Name: vSwitch0
Class: etherswitch
Num Ports: 128
Used Ports: 6
Configured Ports: 128
MTU: 1500
CDP Status: listen
Beacon Enabled: false
Beacon Interval: 1
Beacon Threshold: 3
Beacon Required By:
Uplinks: vmnic1, vmnic0
Portgroups: ESXSecretAgentNetwork, VM Network, vmk2, vmk1, Management Network
vSwitch1
Name: vSwitch1
Class: etherswitch
Num Ports: 128
Used Ports: 3
Configured Ports: 128
MTU: 1500
CDP Status: listen
Beacon Enabled: false
Beacon Interval: 1
Beacon Threshold: 3
Beacon Required By:
Uplinks: vmnic2
Portgroups: VMkernel

Here is the same example, but now only displaying Name, Num of Ports, MTU, CDP and Portgroups using the formatter and format-param options:

~ # esxcli --formatter=csv --format-param=fields="Name,Num Ports,MTU,CDP,Portgroups" network vswitch standard list
Name,NumPorts,MTU,Portgroups,
vSwitch0,128,1500,"ESXSecretAgentNetwork,VM Network,vmk2,vmk1,Management Network,",
vSwitch1,128,1500,"VMkernel,",

If you like to learn more about esxcli, you can take a look at the What's New in vSphere 5 Platform whitepaper. I would also highly encourage you to play with the new esxcli and see what the new capabilities are. Also, when the vSphere 5 documentation is released, a must read document is the Command-Line Management in vSphere 5.0 for Service Console Users which goes into detail about the specific esxcli commands to use in replacement of the esxcfg-*/vicfg-* commands.

Note: Hopefully it was okay to borrow some of these esxcli slides from VMware, I made some slight modifications. The diagrams are a great way to explain how esxcli namespaces work.