vSphere API – Page 22

New vSphere 6.0 APIs for VSAN, VVOLs, NFS v4.1 & more!

02.03.2015 by William Lam // 2 Comments

vSphere 6.0 includes a number of new storage platform capabilities and new features which includes VSAN 6.0, VVOLS and NFS v4.1 to just name a few. For those of you who are interested in consuming these new capabilities programmatically for Automation, here are some of the new vSphere 6.0 APIs to be aware of from a Storage stand point.

Generic Storage Sub-System

HostStorageSystem (Existing Managed Object)
- Methods to manage and configure NFS v4.1
  - ChangeNFSUserPassword
  - ClearNFSUser
  - QueryNFSUser
  - SetNFSUser
- Methods to mark a Disk device either Local or Remote
  - MarkAsLocal_Task
  - MarkAsNonLocal_Task
- Methods to mark a Disk device as either an SSD or Magnetic Disk
  - MarkAsNonSsd_Task
  - MarkAsSsd_Task
- Methods to turn On or Off the LED for supported Disk devices
  - TurnDiskLocatorLedOn_Task
  - TurnDiskLocatorLedOff_Task
- Method to issue VAAI UNMAP operation
  - UnmapVmfsVolumeEx_Task

VSAN 6.0

VsanUpgradeSystem (New Managed Object) - Methods to upgrade to the new VSAN 6.0 on-disk format
- PerformVsanUpgrade_Task
- PerformVsanUpgradePreflightCheck
- QueryVsanUpgradeStatus

HostVsanInternalSystem (Existing Managed Object) - Methods to query & upgrade VSAN Objects
- QueryVsanObjectUuidsByFilter
- UpgradeVsanObjects

HostVsanSystem (Existing Managed Object) - Methods to manage VSAN Node and disks
- EvacuateVsanNode_Task
- RecommissionVsanNode_Task
- UnmountDiskMapping_Task
ComputeResource (Existing Managed Object)
- faultDomainInfo - New Property to configure Fault Domains

VVOL

HostDatastoreSystem (Existing Managed Object) - Methods to create and delete VVOL Datastores
- CreateVvolDatastore
- RemoveDatastoreEx_Task

How to generate specific support log bundles for vCenter & ESXi using vSphere API?

07.15.2014 by William Lam // 1 Comment

When an issue arises in your vSphere environment and you engage with VMware GSS, one of the first task you will mostly like be asked to perform is to generate a support log bundle and submit it. You can generate a support bundle using the vSphere Web/C# Client as seen in the screenshot below.

Last week I recieved a question from one of our development teams about generating support log bundles for both vCenter Server and ESXi hosts, but using the vSphere API instead of the UI. This functionality is available as part of the DiagnosticManager and the method you will want to call is GenerateLogBundles_Task() which allows you to specify which systems to collect support logs from. This by far is the easiest way if you just want to collect all logs and I will go into more detail in a bit on what I mean by this.

When invoking the GenerateLogBundles_Task(), support logs will be generated on each of the respective sub-systems which includes vCenter Server and the ESXi hosts. Once the log bundles have been created, the API will return a list of download URLs to the support log bundles. To demonstrate this functionality, I have created a sample vSphere SDK for Perl script called generateLogBundle.pl

The script assumes you want to collect ALL logs for both vCenter Server and ESXi hosts being managed by the given vCenter Server.

While working on the sample code, one thing I had noticed in the vSphere Web/C# Client is that you have the ability to select the specific logs to export.

I was looking through the vSphere API but I did not find a way to specify this level of granularity in the logs. I was of course curious and wondered if this was using some sort of private vSphere API (which I hoped was not the case). I decided to pull out one of my favorite tools when I get stuck with a question around the vSphere API which is Onyx. After manually generating the support log bundle and selecting a subset of the logs to export, I found out this process actually makes a call to a CGI script on the ESXi host called vm-support.cgi.

The list of available logs to export is generated by making a GET request to the following URL: https://[ESXI-HOSTNAME]/cgi-bin/vm-support.cgi?listmanifests=1

The output is an XML file that contains the manifest of the available logs to export in the log support bundle. To specify the specific logs to collect, you must provide the "id" property as seen in the screenshot above. In this example, I would like to collect the following: System:Base, VirtualMachines:GuestState, Storage:VSAN and VirtualMachines:diskinfo

To generate a support bundle from the ESXi host that contains the above, you will need to issue a GET request using the following URL: https://[ESXI-HOSTNAME]/cgi-bin/vm-support.cgi?manifests=System:Base VirtualMachines:GuestState Storage:VSAN VirtualMachines:diskinfo

This will instruct the ESXi host to generate the log bundle that contains the following manifest log files. To include others, you just need to append to the URL. You can easily test this out by pasting this URL into a web browser and providing your ESXi credentials and the logs will be available for download. Going back to my original observation, this is actually how vCenter Server provides the ability to export specific logs from each of the ESXi hosts when generating a support bundle.

If you would like to use the vSphere API to provide this same capability, you just need to leverage the AcquireGenericServiceTicket() API method which is available as part of the SessionManager and specify the URL for the support bundle. This will allow you to request a support log bundle from within vCenter Server without having to go directly to each of the ESXi hosts manually. You can see a very similar implementation of this API by taking a look at my How to efficiently transfer files to a datastore in vCenter using the vSphere API and specifically the efficent-upload-files-to-datastore.pl sample code.

How to efficiently transfer files to Datastore in vCenter using the vSphere API?

06.18.2014 by William Lam // 19 Comments

A pretty common task for vSphere Administrators is to upload or download content from a vSphere Datastore which usually contains ISOs and floppy images. You can initiate the file transfer using the vSphere Web/C# Client, however this process can be quite tedious when having to manually upload several ISOs. Instead, you will probably want to automate this process and and there are a couple of ways in which you can accomplish this. One option, is to go directly to an ESXi host and upload your files but this is not ideal when you have vCenter Server to centrally manage your infrastructure. The second option is to go through vCenter Server, but depending on the implementation, you can potentially add unnecessary load to the vCenter Server if implemented incorrectly.

Let me explain this further with two diagrams and you can decide which implementation you prefer?

In this first implementation, I directly access the file management API which leverages a simple HTTP GET/PUT operation to upload files to a vSphere Datastore. What I found out while transferring the data was that the data actually traverses through the vCenter Server and then onto the ESXi host before writing to the vSphere Datastore. This of course made the data transfer very inefficient not to mention additional bandwidth and load being added to vCenter Server.

I created a sample vSphere SDK for Perl script that demonstrates this inefficent transfer called inefficent-upload-files-to-datastore.pl

Here is sample execution of the script which accepts the name of the vSphere Datacenter, vSphere Datastore, the source file to transfer and the destination path of where the file will be uploaded to:

./inefficent-upload.pl --config ~/vmware-dev/.vcenter55-1 --datacenter Datacenter --datastore vsanDatastore --sourcefile /Volumes/Storage/Images/CentOS-6.4-x86_64-netinstall.iso --destfile ISO/CentOS-6.4-x86_64-netinstall.iso

After talking to some folks about this problem, I learned about a more efficient method as shown in the diagram below.

As you can see, we can still initiate the transfer using the vCenter Server, but the actual data transfer is than sent to one of the ESXi hosts that has access to the vSphere Datastore. To accomplish this, we need to use the AcquireGenericServiceTicket() method which is part of the sessionManager. Using this method, we can request a ticket for a one time HTTP request to connect directly to an ESXi. To upload a file, the request must include the method which in this case will be a PUT operation and the local URL to an ESXi host that has access to the vSphere Datastore.

Here is an example of a URL: https://vesxi55-1.primp-industries.com/folder/ISO/CentOS-6.4-x86_64-netinstall.iso?dcPath=ha-datacenter&dsName=vsanDatastore

ESXi IP Address/Hostname - In the script, I select the first ESXi host that has access to the vSphere Datastore
vSphere Datastore Directory - Directory into which the contents of the file will be placed in. In this example, we just have one top-level directory called ISO which must already exist
Destination file name - The name of the file that should appear in the vSphere Datastore
Datacenter Name - This should always be ha-datacenter when connecting directly to an ESXi host
vSphere Datastore - The name of the vSphere Datastore

To demonstrate this functionality, I have created a vSphere SDK for Perl script called efficent-upload-files-to-datastore.pl which accepts the name of the vSPhere Datastore along with the source and destination of where the file will be placed:

./upload-files-to-datastore.pl --config ~/vmware-dev/.vcenter55-1 --datastore vsanDatastore --sourcefile /Volumes/Storage/Images/CentOS-6.4-x86_64-netinstall.iso --destfile ISO/CentOS-6.4-x86_64-netinstall.iso

Hopefully after looking at these two implementations, you will also agree that the second option is the best! One last thing that I would like to point out is that even though we are talking about transferring files to a vSphere Datastore, this method can also be used to efficiently transfer other supported files to an ESXi hosts through vCenter Server as described in this blog article.