WilliamLam.com

External replication of vSphere Content Library

by // 17 Comments

As the adoption of vSphere Content Library continues to grow, I am seeing more questions from our field and customers around content distribution. In case you did not know, vSphere Content Library (CL as I will be refering to it going forward) has its own built-in native replication mechanism which allows customers to easily publish and subscribe to libraries from either within a single vCenter Server instance or even between two completely different vCenter Servers (regardless of deployment topology and/or SSO Domain configurations).


Content distribution or replication is handled by CL which is a service within the vCenter Server. If content is being replicated from within a single vCenter Server and the ESXi hosts can communicate with each other, then direct host to host transfer is used, also referred to as Network File Copy (NFC), rather than going through vCenter Server. When content is transfered between two vCenter Servers, then the data travels through vCenter Server using standard HTTPS (443) by default. In the latter scenario, if you have configured Enhanced Linked Mode for your vCenter Servers, then NFC will be used unless ESXi hosts can not communicate with each other than, it will automatically fall back to the default HTTPS which is pretty cool.

One thing that may not be very well known is that customers actually have a choice in how their CL content is replicated. In addition to native replication which currently does not support incremental/delta updates, meaning all file transfers are full copies, CL can also support external replication. In fact, many customers today already have existing methods for efficiently replicating large amounts of data across multiple datacenters whether that is replication built into their storage arrays, network appliances or some other means. For these customers, you can still benefit from CL while continue to take advantage of your existing methods of replication.


[Read more...]

PowerCLI script to help correlate vCenter, ESXi & vSAN build/versions w/o manual VMware KB lookup

by // 10 Comments

I can still remember when I was a VI Admin and how annoying it was to try to correlate the build numbers for my ESX(i) hosts and vCenter Servers that I have deployed with the versions listed on VMware's website. This especially gets challenging when there are multiple patch releases (a, b, c or 01, 02, 03) in between major releases (5.5, 6.0, 6.0u1, 6.0u2, 6.5, etc.). Historically, most customers including myself would retrieve the respective build numbers and then manually comparing them to either the release notes and/or download website which was very tedious.

Although VMware has exposed the version number within our vSphere products since day 1 which can also be retrieved programmatically using the vSphere API (here), it unfortunately does not provide more details than simply the major/minor version (e.g. 5,5, 6.0, 6.5, etc) of the software. Recently, VMware had released a series of VMware KBs which provides a mapping between the build numbers for vCenter Server, ESXi and vSAN to their respective versions which can be found in the links below:

These are definitely a great set of resources that I know many customers including myself have been using since its release. Having said that, the process today is still pretty manual since you need to manually retrieve the build numbers for either a VC, ESXi or vSAN Host (can be automated using vSphere APIs) and then comparing that to the KBs to get the correct versions. How cool would it be if you could *easily* just point to YOUR environment and retrieve the version information for either a vCenter Server (Windows or VCSA), ESXi host(s) or vSAN host(s) without needing to manually perform this lookup each time? Well, I have just done that! I have taken all three KBs and converted that information into a simple PowerCLI script called VCESXivSANBuildVersion.ps1 leveraging our vSphere API and it provides three functions:

  • Get-VCVersion - Retrieves the vCenter Server version given a VC connection
  • Get-ESXiVersion - Retrieves the ESXi version for all hosts given a vSphere Cluster
  • Get-VSANVersion - Retrieves the vSAN version for all hosts given a vSAN Cluster

Here is an example output using the first two functions:


For the vCenter Server version output, you will notice that I am also including the OS platform of your vCenter Server, so you can distinguish between a Windows vCenter Server and a vCenter Server Appliance (VCSA) which can be useful to see if you have been #migrate2vcsa ;). For the ESXi version output, you will notice the "OriginalInstallDate" value, this is actually new API property that was introduced in vSphere 6.5 and it provides you with the original installation date of your ESXi host (more details can be found here) which is pretty neat.

Here is an example output using last function:


If you wanted to take this a step further, you could even take this output and dynamically update the vSphere UI using either Custom Attributes or vSphere Tags so you know what version the software is at any given moment. Its easy enough to set this up as a scheduled task that could run periodically so you always have the latest information provided in the vSphere UIs.

Although this is a significant improvement over the existing manual methods, I think most of you will agree that it would be ideal if this information was natively available within the product which means BOTH UI and APIs. I think we all appreciate versioning of software is not always easy and it can change from release to release for a variety of reasons, most of which may not be technical. If the vSphere platform could dynamically pull this information in either real time and/or through an offline mechanism and provide this association by default, it would greatly improve the experience when needing to troubleshoot or perform maintenance of the vSphere platform. If this is something you would like to see, please leave a comment below providing your feedback. I know I have already pinged our PMs about this and I am sure they would love to hear form you as well!

Additional Information:

Note1: Update levels can be found using the vSphere API, take a look at this article here for more details.

Note2: As of ESXi 6.5 Update 1, the Update levels are also included by default in the Embedded Host Client as shown in the screenshot below:

Note3: As of vSAN 6.2, the vSAN Management API already includes vSAN version information that can be queried. Take a look at this script here which exercises this new API. For example above, I decided to not use this new API since customers may be running older releases of vSAN which is not covered by the vSAN Mgmt API.

Note4: VMware has also published simliar build to version mapping for other VMware products which can find the complete list here.

vSphere Content Library versioning 

by // 1 Comment

A question came up the other day on how versioning is handled when using the vSphere Content Library feature and specifically when you update an existing VM template that already exists in the Content Library (CL, which I will be using throughout the article). Having used CL since vSphere 6.0 which I have also written about it extensively here, I knew that it had a built-in versioning mechanism which would automatically increment as new updates were applied to the VM Templates stored in CL.

One thing that I do not think many folks are aware of is that you can actually retrieve the CL Item Version within the vSphere Web Client. You can do by just adding the "Content Version" column by right clicking on column headers and select show/hide columns to add or remove specific fields.


I figured, this was pretty straight forward and shared my findings with the individual and thought that was it. While working on my Content Library PowerCLI community module, I came to learn there was more than one version that was being returned by the CL API.

Here is an example screenshot showing the three different versions using the Get-ContentLibraryItems (not to be confused with the OOTB Get-ContentLibraryItem cmdlet):


Taking a look at the API documentation, it was still not 100% clear on what all these versions provided and also when would they increment? I reached out to one of the CL Engineers, Eric who then provided me more information on how each of these versions are being used. Lets go through each one of them and I will include an example which also helped me understand how these different "versions" actually work.

Content Version - This particular version will increment as changes are made to the actual file content itself. This is also the same version that is available in the vSphere Web Client UI as shown earlier.

Lets now go through an example workflow to show when this version will get incremented:

  1. Create a new VM called Foo (ContentVersion=N/A)
  2. Clone VM Foo into CL called Foo VM Template (ContentVersion=1)
  3. Clone completes for Foo VM Template (ContentVersion=2) <--The reason this is 2 and not 1 is that when the initial CL Item is created (think of it as a placeholder), the Content Version is incremented
  4. Deploy Foo VM Template to new VM called Bar (ContentVersion=2, no changes on deploy)
  5. Update VM Bar with some changes and clone Bar back to Foo VM Template (ContentVersion=3) <--Changes were made and we now increment the Content Version

Metadata Version - This particular version will only increment for changes made to either the name or description of the CL template.

Version - This particular version is used for concurrency control and this will always increment along with the Metadata Version for local libraries.

Lets now go through an example workflow for a local library to show how these two versions get incremented:

  1. Create a new VM called Foo (MetadataVersion=N/A)
  2. Clone VM Foo into CL called Foo VM Template (MetadataVersion=1, Version=1)
  3. Update the description of Foo VM Template (MetadataVersion=2, Version=2)
  4. Deploy Foo VM Template to new VM called Bar (MetadataVersion=2, Version=2, no changes on deploy)
  5. Update VM Bar with some changes and clone Bar back to Foo VM Template (MetadataVersion=3, Version=3)

Lets now go through an example workflow for a subscribed library to show how these two versions get incremented:

  • Create a new VM called Foo on a local library named CL1 (MetadataVersion=N/A)
  • Clone VM Foo into CL called Foo VM Template (MetadataVersion=1, Version=1)
  • Update the description of Foo VM Template (MetadataVersion=2, Version=2)
  • Create a new subscribed library to CL1 which will download Foo VM Template (MetadataVersion=2, Version=1) <--MetadataVersion must match the publisher, but version does not have to match

After going through a few of these exercises with a few dummy VMs and using my Get-ContentLibaryItems function, I was able to get a better grasp on CL versioning. Hopefully this helpful for anyone who might want to use the CL APIs to track changes between their VM templates and/or files or just being able to see this within the vSphere Web Client UI.