WilliamLam.com

New vSphere 6.5 APIs worth checking out

by // 11 Comments

With the upcoming new release of vSphere, there are quite a few new vSphere APIs to look forward to and consume from an Automation standpoint. Similiar to what I had done in the past with previous major releases of vSphere, here is a list of of some of the new vSphere APIs (SOAP based) that I think are worth checking out whether they are new features vSphere 6.5 will introduced or new enhancements to existing functionality which will benefit our vSphere Administrators and/or Developers.

If you would like to see the complete list of new vSphere 6.5 (SOAP based) APIs, be sure to check out the vSphere 6.5 API Reference Guide which will include a "What's New" section on all the new Managed Objects, Methods, Properties, etc. when vSphere 6.5 is generally available.

CryptoManager / CryptoManagerKmip - VM Encryption is one of the new features in vSphere 6.5 and with these APIs, you will be able to manage and configure the VM Encryption settings including associating with KMIP server. For enabling/disabling VM Encryption at the VM and disk level, have a look at VirtualMachine->crypto and VirtualMachine->deviceChange->backing property.

  • GenerateClientCsr
  • GenerateKey
  • GenerateSelfSignedClientCert
  • ListKmipServers
  • MarkDefault
  • RegisterKmipServer
  • RemoveKmipServer
  • RetrieveClientCert
  • RetrieveClientCsr
  • RetrieveKmipServerCert
  • RetrieveKmipServersStatus_Task
  • RetrieveSelfSignedClientCert
  • UpdateKmipServer
  • UpdateKmsSignedCsrClientCert
  • UpdateSelfSignedClientCert
  • UploadClientCert
  • UploadKmipServerCert

FailoverClusterConfigurator - To setup the new vCenter Server High Availability (VCHA) feature which is only available in the VCSA, use these APIs which include deploying and configuring the passive and witness nodes.

  • configureVcha_Task
  • createPassiveNode_Task
  • createWitnessNode_Task
  • deployVcha_Task
  • destroyVcha_Task
  • getVchaConfig
  • prepareVcha_Task

FailoverClusterManager -  Have a look at these APIs to initiate a failover or view the current VCHA configuration.

  • getClusterMode
  • GetVchaClusterHealth
  • initiateFailover_Task
  • setClusterMode_Task

HostVStorageObjectManager - An API only feature in vSphere 6.5 which will allow you to create and manage Virtual Disks as a "First Class" citizen. This particular API is for managing First Class Disks (FCD) when talking directly to an ESXi host.

  • HostCloneVStorageObject_Task
  • HostCreateDisk_Task
  • HostDeleteVStorageObject_Task
  • HostExtendDisk_Task
  • HostInflateDisk_Task
  • HostListVStorageObject
  • HostReconcileDatastoreInventory_Task
  • HostRegisterDisk
  • HostRelocateVStorageObject_Task
  • HostRenameVStorageObject
  • HostRetrieveVStorageObject
  • HostRetrieveVStorageObjectState
  • HostScheduleReconcileDatastoreInventory

VcenterVStorageObjectManager - An API only feature in vSphere 6.5 which will allow you to create and manage Virtual Disks as a "First Class" citizen. This particular API is for managing First Class Disks (FCD) when talking directly to a vCenter Server.

  • AttachTagToVStorageObject
  • CloneVStorageObject_Task
  • CreateDisk_Task
  • DeleteVStorageObject_Task
  • DetachTagFromVStorageObject
  • ExtendDisk_Task
  • InflateDisk_Task
  • ListTagsAttachedToVStorageObject
  • ListVStorageObject
  • ListVStorageObjectsAttachedToTag
  • ReconcileDatastoreInventory_Task
  • RegisterDisk
  • RelocateVStorageObject_Task
  • RenameVStorageObject
  • RetrieveVStorageObject
  • RetrieveVStorageObjectState
  • ScheduleReconcileDatastoreInventory

DatastoreNamespaceManager->ConvertNamespacePathToUuidPath() - From a troubleshooting standpoint, do you ever wish you can easily translate the human readable VM path (e.g. /vmfs/volumes/vsanDatastore/myVM/myVM.vmx to the VSAN/VVOL equivalent identifier which is UUID based? Well, this is now possible with this new API!

AuthorizationManager->FetchUserPrivilegeOnEntities() - This is a pretty neat API as it allows you to easily query an existing user to see the current privileges has been assigned. This could could come in handy to quickly audit a particular privilege for a user.

HostImageConfigManager->installDate() - Have a look at this blog post Super easy way of getting ESXi installation date in vSphere 6.5 for more details.

HostImageConfigManager->fetchSoftwarePackages() - This is another nice API to easily retrieve all the VIBs installed on an ESXi host. This is the equilvenet of running "esxcli software vib list" and you will now have all the additional metadata info that was historically only available via ESXCLI. Here is an example PowerCLI function called Get-ESXInstalledVib which exercises this new API.

HostStorageSystem->UpdateVmfsUnmapPriority() - Have a look at the blog post Configure new automatic Space Reclamation (VMFS UNMAP) using vSphere 6.5 APIs for more details.

VirtualMachine->{AttachDisk_Task(),DetachDisk_Task()} - This API allows you to attach and detach First Class Disks that you may have created earlier using the FCD APIs as shown above.

VirtualMachine->config->bootOptions->EfiSecureBootEnabled - To take advantage of the new VM Secure Boot feature in vSphere 6.5, you simply just toggle this property. Here are two PowerCLI functions called Get-SecureBoot/Set-SecureBoot which exercises this new API.

In addition, vSphere 6.5 also introduces a new REST-based API that covers several areas such as basic VM Lifecycle Management (simliar to that of the existing vSphere SOAP-based API), vSphere Content Library, vSphere Tagging and Virtual Appliance Management for the vCenter Server Appliance (VCSA). You can interact with these new APIs by using any of the vSphere Automation SDKs (.Net, Java, Python, Ruby or Perl) or even just simply using cURL from the command-line. It is really that easy!

Lastly, to make exploring these new REST-based APIs easier for both administrators as well as developers, there is now a new API Explorer that is included specifically with the VCSA in vSphere 6.5. You can think of it like a vSphere MOB 2.0 but way easier to use. Some of you may recognize the interface as shown in the screenshot below which uses the Swagger UI. This interface allows you to quickly browse through all the APIs, no need to refer to the documentation as the APIs are self-documented and made available in this interface. Best of all, you can even try out the APIs by simply logging into your vCenter Server and then selecting an API and clicking on the "Try it out now" button!

To access the API Explorer, you simply open a web browser and enter the following URL: https://[VC-HOSTNAME-OR-IP]/apiexplorer/

vsphere-6-5-apis-apiexplorer
There will also be native PowerCLI cmdlets (Get-CisService) to these new REST API and below is a quick example of retrieving the version (GET /system/version) of the VCSA:

$vcsaVersion = Get-CisService -Name  'com.vmware.appliance.system.version'
$vcsaVersion.get()

vsphere-6-5-apis-powercli

Configure new automatic Space Reclamation (VMFS UNMAP) using vSphere 6.5 APIs

by // 6 Comments

Since its first introduction in vSphere 5.5, VMFS UNMAP also know as Space Reclamation for a VMFS based datastore has been a pretty popular Storage capability in vSphere. A commonly asked question from customers is when will the "automatic" capability return? Well, it looks like it is now back in the upcoming vSphere 6.5 release as blogged about here by Duncan Epping. Below is a screenshot of where you can find the setting. VMFS UNMAP is now enabled by default and you will need to have a VMFS 6 datastore to take advantage of this new feature.

vmfs-unmap-vsphere-65-api-0
For customers who wish to automate the configuration of the VMFS UNAMP capability whether that is to check the current settings or to enable/disable it, there are some new vSphere 6.5 APIs that have been introduced which differ from the previous implementations. To change the VMFS UNMAP setting, there is a new vSphere API called UpdateVmfsUnmapPriority() which accepts the UUID of a VMFS 6 datastore as well as an unmapPriority property which can either be "low" which means it is enabled or "none" which means it is disabled. To view the current VMFS UNMAP settings, there is a new property under the Datastore->Info->Vmfs object called UnmapPriority.

To demonstrate this new vSphere API, I have created two small PowerCLI functions called Get-VMFSUnmap and Set-VMFSUnmap which can be downloaded from here.

Here is an example of retrieving the current VMFS UNMAP settings:

Get-Datastore "mini-local-datastore-hdd" | Get-VMFSUnmap

vmfs-unmap-vsphere-65-api-1
Here is an example of enabling automatic VMFS UNMAP setting:

Get-Datastore "mini-local-datastore-hdd" | Set-VMFSUnmap -Enabled $true

vmfs-unmap-vsphere-65-api-2

How to check the size of your Config & SEAT data in the VCDB in vPostgres?

by // 2 Comments

After publishing my article on how to check the size of your vCenter Server's Configuration and Stats, Events, Alarm & Tasks (SEAT) data for both a Microsoft SQL Server and Oracle based database, I had received a few requests for doing the same for the vPostgres database which the vCenter Server Appliance (VCSA) uses exclusively. Thanks to one of our Engineers who works on the VCDB, I was able to quickly get the relevant SQL query to perform the exact same lookup as the other two databases.

Since the VCSA is harden and locked down by default, being able to remotely retrieve this information will actually require some additional configuration changes to your VCSA which may or may not be acceptable. Because of this constraint, I will provide two options in how you can perform this SQL query.

The first option (easy) will be running the SQL query directly from within the VCSA. You just need SSH access and no other information or credentials will be required. The second option (complex) will be to remotely connect to the vPostgres database (generally not recommend) which will require the VCDB's credentials which I will show you how to retrieve. Lastly, I want to quickly mention that in the upcoming vSphere 6.5 release, this information will be super easy to view not only from a UI but also API as shown in tweet below.

 

Option 1:

Step 1 - Download the following shell script called queryVCDBvPostgres.sh which contains the respective VCDB SQL query.

Step 2 - SCP the shell script to your VCSA and then login via SSH.

Step 3 - Run the following command to make the script executable:

chmod +x queryVCDBvPostgres.sh

Step 4 - Run the script by issuing the following command:

./queryVCDBvPostgres.sh

Here is a screenshot of what you should see which is a break down of your Config + SEAT data:

query-vcdb-config-seat-data-vpostgres-0

Option 2:

Step 1 - Login to the VCSA using SSH.

Step 2 - Edit /storage/db/vpostgres/postgresql.conf and add the following entry:

listen_addresses = '*'

This will allow vPostgres to be connected to from any address or if you want to restrict it to a specific IP, you can also just specify that.

Step 3 - Edit /storage/db/vpostgres/pg_hba.conf and add the following entry:

host    all             all             172.30.0.0/24            md5

Similiar to the previous configuration, you can either specify a network range using CIDR notation or a specific IP Address.

Step 4 - Edit /etc/vmware/appliance/firewall/vmware-vpostgres and replace it with the following entry:

{
  "firewall": {
     "enable": true,
        "rules": [
        {
          "direction": "inbound",
          "name": "vpostgres_external",
          "port": "5432",
          "portoffset": 0,
          "porttype": "dst",
          "protocol": "tcp"
        }
     ]
  },
  "internal-ports": {
    "rules": [
      {
        "name": "server_port",
        "port": 5432
      }
    ]
  }
}

This will open up the VCSA's firewall to allow remote connections to the vPostgres port which the default is 5432.

Step 5 - Next, we need to reload the firewall configuration by running the following command:

/usr/lib/applmgmt/networking/bin/firewall-reload

Step 6 - We can verify by running the following command:

iptables -L | grep postgres

Here is a screenshot of what you should see as the output:

query-vcdb-config-seat-data-vpostgres-2
Step 7 - Lastly, we need to restart the vPostgres service by running the following command:

service vmware-vpostgres restart

Step 8 - To verify that you can now remotely connect to the vPostgres DB, run the following command:

netstat -anp | grep LISTEN | grep tcp | grep 5432

Here is a screenshot of what you should see as the output:

query-vcdb-config-seat-data-vpostgres-3
At this point, you have now enabled remote connections to the VCSA's vPostgres DB. The next step is to retrieve the VCDB credentials which you will do so using a PowerShell script that I have written to perform the remote SQL query. This will also require that you setup an ODBC connection on your client system to communicate with the vPostgres DB. Please have a look here for more information on how to setup the ODBC connection.

Step 9 - Login to VCSA via SSH and then look at the /etc/vmware-vpx/vcdb.properties and you should see the password to your VCDB. Go ahead and record this some where as you will need it in the next step. The username for the DB will be vc which you can also make a note of.

Step 10 - Download the following PowerShell script called Get-VCDBUsagevPostgres.ps1 and provide the connection details that you retrieved in Step 9. If everything was properly configured, you can run the PowerShell script and it should produce a similiar output as shown in the screenshot below.

query-vcdb-config-seat-data-vpostgres-1