WilliamLam.com

How to Access vSphere Remote Console Using vSphere & VMRC API

by // 42 Comments

Similar to my vCloud Director article, you can also provide access to the remote console of a virtual machine in a vSphere environment to your end users. This can be accomplished by leveraging both the vSphere and VMRC (Virtual Machine Remote Console) APIs and can be useful if you are building a custom portal for users to access their virtual machines. To use the VMRC API, you can download the latest VMRC 5.1 SDK which provides the following:

The VMRC SDK allows you to use a Web-based application to connect to a vCenter- or vCloud Director-managed virtual machine and access that virtual machine’s console in a browser window. You can interact with the virtual machine console input and screen. You can also use the VMRC SDK to manage virtual and physical device connections on a vCenter-managed virtual machine. 

The VMRC SDK includes documentation to the API as well as a sample webpage implementing some basic functionality of the VMRC API. I recently received a question on how to get started with the sample as it was not completely intuitive and thought I take you through the required steps to get the sample working.

Step 1 - Download the VMRC 5.1 SDK and extract the contents to your local desktop.

Step 2 - Open the vmrc-embed-example.html using a web browser located in the docs folder.

Note: In my example, I uploaded both the javascript and html file to a web server and accessed the sample by connecting to the server instead of running it locally on my desktop. This was to show how users could access the custom portal using the VMRC SDK.

Step 3 - At the top of the page where it says "VMRC Modes", make "MKS" is selected in the drop down box and click on the "+" icon to add. Then go ahead and click on the "Start" button to start a VMRC instance and ensure you see a success message on the right hand side of the console box.

Step 4 - To authenticate to VMRC, you will need a session ticket which will be obtained through the use of the vSphere API using the acquireCloneTicket() method provided by the SessionManager managed object. In this example, we will be using the vSphere MOB to quickly retrieve our session ticket, but in a real implementation, you would programamtically retrieve the session ticket along with few othe pieces of information to connect to the VMRC. Open up a new tab in your web browser and connect to the following URL:

https://reflex.primp-industries.com/mob/?moid=SessionManager&method=acquireCloneTicket

Note: Make sure you substitute in your vCenter Server IP Address/Hostname

Once you have authenticated, go ahead and click on the "Invoke Method" which should generate a session ticket:

Step 5 - Copy the session ticket string and go back to our VMRC sample page. We will now need to fill out the following sections before we can access the remote console of a virtual machine:

  • Hostname (IP Address/Hostname of your vCenter Server)
  • Allow SSL Validation Errors (Check this if you are using self signed SSL certificates)
  • Ticket (Paste the session ticket from the previous step here)
  • VM ID (This is the MoRef ID of the VM you wish to connect to the remote console)

Once you have filled out the minimum required fields, go ahead and click on the "Connect" button and if everything was successful, you should now see the remote console of the virtual machine you selected.

Detecting A Duplicate IP Address For Your ESXi Hosts Using a vCenter Alarm

by // 6 Comments

The motivation for this article was a tweet I noticed from Duncan Epping this morning. Per Duncan's tweet, it looks like he may have accidentally assigned an IP Address to one of his virtual machines which was already being used by an existing ESXi host causing a duplicate IP Address error. We probably have all experienced this once in our lives and it can be quite difficult and frustrating to troubleshoot. Similar to a Windows OS, ESXi can also detect a duplicate IP Addresses but instead of a notification window, it is just logged in the VMkernel logs which looks like the following:

2013-01-21T15:52:35.989Z cpu1:2049)Tcpip_Vmk: 112: arp: 00:50:56:bd:3b:2b is using my IP address 172.30.0.213 on vmk1! 

The biggest challenge of course is to identify which ESXi host actually has a conflict and then taking a look at the logs to find the offending MAC Address and shutting them down yourself or with the help of a Network Administrator. Wouldn't it be great if we had an alarm to automatically notify us when a duplicate IP Address is detected? Well I am glad you asked and the answer is YES! 🙂

In addition to logging to the VMkernel logs, ESXi also logs this "observation" in /var/log/vobd.log which stands for the VMkernel Observation. These "observations" can provide critical identifying information in case of an error and is usually used during troubleshooting. In our case, we are seeing an intermittent network connectivity to our ESXi host which is in result of a duplicate IP Address. The really neat thing about these VOBs is that you can create vCenter Alarms when a specific VOB has been detected. I have shown an example of this before in my Detecting ESXi Remote Syslog Connection Error Using a vCenter Alarm article.

We can do exactly the same for detecting a duplicate IP Address for an ESXi host. The first thing we need to do is identify the VOB ID by looking in /var/log/vobd.log:

2013-01-21T15:02:07.513Z: [netCorrelator] 917174784727us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for 172.30.0.83 on interface vmk0, current owner being 00:50:56:bd:3b:2b

We can see the VOB ID for this is esx.problem.net.vmknic.ip.duplicate and this will be used in our vCenter Alarm trigger.

Step 1 - Create a new Alarm and specify a name, the Monitor type will be Hosts and Monitor For will be for a specific event:

Step 2 - Copy the VOB ID that we have identified from above and specify that as our alarm Trigger:

Step 3 - If you wish to receive an email notification or send an SNMP trap go ahead and configure additional actions, else just click next which will just display a vCenter Server alert in the UI.

Now that our alarm has been created, we will want to give this a test drive .... who can we ask? Well it just happens that I have a new user in my environment and I provisioned him a new VM which is already connected to the network. Let's hope he does not try to change the IP Address (because this never happens, right?)

After the user statically assigns the IP Address of an existing ESXi host in the VM, we should see our new alarm trigger in vCenter.

As you can see, we have quickly identified the ESXi host that is impacted and we can then login to DCUI via the console to take a look at the logs to find the offending MAC Address. Hopefully duplicate IP Addresses is not a common problem in your environment but it does happen from time to time and having an alarm to help you quickly narrow down the culprit can be quite useful.

Retrieving vscsiStats Using the vSphere 5.1 API

by // Leave a Comment

In my previous article, I talked about the new Service Manager API that was introduced in vSphere 5.1 and how you can retrieve ESXTOP performance data using this new vSphere API. In this article I will show you how to collect vscsiStats data using this same interface. If you are not familiar or have not used vscsiStats before, I would highly recommend you take a look at the Using vscsiStats for Storage Performance Analysis as it goes over some of the basics of vscsiStats and how it works.

UPDATE 02/15/2017 - Please see this updated article here for how to access this API using vCenter Server.

Disclaimer: You should try to limit the use of these interfaces for statistics collection or debugging/troubleshooting purposes as there is a certain amount of overhead when running these commands. It is also important to note that since the output is based on the implementer of the service, there is no guarantee that output would not change from one release to the other.

The first step is to get a reference to the vscsiStats service via the Service Manager (must connect directly to an ESXi 5. host, this is not supported when connecting to vCenter Server) and to invoke an operation for vscsiStats, you will need to use the ExecuteSimpleCommand. For vscsiStats, there are four valid operations:

  • StartVscsiStats
  • FetchAllHistograms
  • ResetVscsiStats
  • StopVscsiStats

To demonstrate the vscsiStats interface, I have written a sample vSphere SDK for Perl script called getVscsiStats.pl which I will use to explain each operation. Please note the data set that is retrieved is in it's raw data form and requires a bit of data processing.

StartVscsiStats

This operation starts the vscsiStats collection for ALL virtual machines residing on your ESXi hosts. This is exactly the same operation if you were to only specify the -s option to the vscsiStats command-line. Here is a screenshot of the "start" operation implemented in the script:

You should see a response of OK from the output and this would indicate the vscsiStats collection has started.

FetchAllHistograms

This operation fetches ALL the vscsiStats histogram data similar to specifying the -p All option in the vscsiStats command-line. The output contains the following:

The tag denotes the details about each Virtual Machine:

  • VM Display Name
  • VM VMX Configuration Path
  • VM BIOS UUID
  • vCenter Server UUID

This is then followed by the tag which provides the VMDK name in the format of scsi:X:Y and within each virtual disk section it will contain 13 tags which represents each of the statistics type and their associated values:

  1. VSCSIVsi_DistanceHistogram: Histogram: distance (in LBNs) between successive commands
  2. VSCSIVsi_DistanceLast16Histogram: Histogram: distance (in LBNs) between each command from the closest of previous 16
  3. VSCSIVsi_DistanceReadsHistogram: Histogram: distance (in LBNs) between successive Read commands
  4. VSCSIVsi_DistanceWritesHistogram: Histogram: distance (in LBNs) between successive Write commands
  5. VSCSIVsi_IoLatencyHistogram: Histogram: latency of IOs in Microseconds (us)
  6. VSCSIVsi_IoLatencyReadsHistogram: Histogram: latency of Read IOs in Microseconds (us)
  7. VSCSIVsi_IoLatencyWritesHistogram: Histogram: latency of Write IOs in Microseconds (us)
  8. VSCSIVsi_IoLengthHistogram: Histogram: IO lengths of commands
  9. VSCSIVsi_IoLengthReadsHistogram: Histogram: IO lengths of Read commands
  10. VSCSIVsi_IoLengthWritesHistogram: Histogram: IO lengths of Write commands
  11. VSCSIVsi_OutstandingIOsHistogram: Histogram: number of outstanding IOs when a new IO is issued
  12. VSCSIVsi_OutstandingIOsReadsHistogram: Histogram: number of outstanding Read IOs when a new Read IO is issued
  13. VSCSIVsi_OutstandingIOsWritesHistogram: Histogram: number of outstanding Write IOs when a new Write IO is issued

Here is a screenshot of the "getstats" operation implemented in the script:

Note: In comparing the output between the vscsiStats command-line and this interface, I found the following three statistics are not available:

  • Histogram: latency of IO interarrival time in Microseconds (us)
  • Histogram: latency of IO interarrival time for Reads in Microseconds (us)
  • Histogram: latency of IO interarrival time for Writes in Microseconds (us)

 

ResetVscsiStats

This operation will reset the vscsiStats collection similar to the -r option in the vscsiStats command-line. Here is a screenshot of the "reset" operation implemented in the script:

StopVscsiStats 

This operation will stop the vscsiStats collection similar to the -x option in the vscsiStats command-line. Make sure you perform this operation once you are done retrieving your vscsiStats data. Here is a screenshot of the "stop" operation implemented in the script:

In addition to the four operations, you can also save the output to a file by specifying the --output option along with the name of the file. vscsiStats is an extremely useful tool to help vSphere administrators profile their virtual machine's IO workload and now you can easily collect this information using the vSphere API. Some really cool things you can do with this data is to create some nifty graphs such as the ones here and here.