WilliamLam.com

Getting started with VMware Pivotal Container Service (PKS) Part 1: Overview

by // 17 Comments

This past week and half, I have been spending quite a bit of time familiarizing myself with the recently released VMware Pivotal Container Service solution, also referred to as VMware PKS for short (yes, that is a K not a C which is a nod to Google's container scheduler Kubernetes). VMware PKS is part of a project that I am currently working on and I figure I would share the process and steps I took to deploy VMware PKS in my own personal lab, in case other folks are interested in trying out this neat and powerful solution for deploying Cloud Native Apps using Kubernetes which was co-developed between VMware, Pivotal and Google.

If you would like to learn more about this first release of VMware PKS and the benefits it provides to both developers (consumers) and operators (admins/SRE) for Kubernetes infrastructure, check out this blog post here. Merlin Glynn, one of the Product Managers for PKS also did an awesome light board video overview of VMware PKS if you want the sparks notes version. If you simply want to give PKS a try without deploying anything, the CNA folks have also published a PKS HOL which can you find here. Another useful resource is the Getting Started with Kubernetes-as-a-Service post from Michael West who works in CNA team and built the PKS HOL.


This will be the first, in a series of articles outlining my VMware PKS deployment and configuration which hopefully can help benefit others as it took me several attempts while learning about the solution. Although the first few articles will include manual guidance, rest assure, there will be some cool automation towards the end but I figure that folks may want to go through this once by hand to get a good understanding on all the different components and how they interact with each other. Plus, some of the PKS-specific automation is still being worked on by the product team and hopefully I will be able to share some of that real soon.

If you missed any of the previous articles, you can find the complete list here:

[Read more...]

Thunderbolt to 10GbE Network Adapters for ESXi

by // 5 Comments

I was recently made aware of this article in which the author, Karim Elatov, had successfully demonstrated the use of a Sonnett Thunderbolt 2 to 10 Gigabit Ethernet Adapter with ESXi running on an Apple Mac Mini. As far as I am aware of, this may be the first public confirmation that such a device would work with ESXi, not to mention having it functional on the Mac Mini. I know in past years, there have been unconfirmed reports on various forums mentioning a Thunderbolt to 10GbE solution that works with ESXi but it was unclear on whether custom drivers were needed or if it would even work with newer versions of ESXi.


This topic has been popular amongst our customers who virtualize Apple MacOS on vSphere. In fact, several years back I had written an article on Thunderbolt Storage for ESXi, which includes a number of solutions that our customers have implemented to provide remote storage for their vSphere infrastructure running on either an Apple XServe, Mac Pro or Mac Mini. Questions around a functional Thunderbolt to 10GbE has definitely been asked about, but I had never heard from any customer who have had a successful story to share, at least until now.

From Karim's post, it looks like he was able to get this working using ESXi 6.0 but it was unclear if there was anything he needed to do to get the device recognized. I reached out to Karim and he was able to confirm that the Thunderbolt device was recognized by ESXi without any additional driver installation. In fact, if you look at this console output on his blog, you will see that it simply uses the inbox Intel ixgbe driver. I had also asked if Karim tried this with the latest version of ESXi, which is currently at 6.5 Update 1. Karim was kind enough to perform one additional test for me which was to confirm the device would still work with the latest ESXi release, which you can see for yourself in the screenshot below.

UPDATE (02/04/19) - Chad Moon recently shared his experiences on getting 10GbE support with an Intel NUC using the OWC Mercury Helios 3, Thunderbolt3 to PCIe expansion enclosure

[Read more...]

Verify Hypervisor-Assisted Guest Mitigation (Spectre) patches using PowerCLI

by // 80 Comments

VMware recently published a new knowledge base (KB) article 52085 that outlines instructions for enabling the Hypervisor-Assisted Guest Mitigation (CVE-2017-5715), also known as the Spectre vulnerability. This KB also provides steps to verify the updated microcode (included in the ESXi patch) has been applied along with Virtual Machine verification (those applicable) to ensure that they are seeing the new CPU features. While following the KB and patching one of my vSphere environments, I had noticed the verification steps was not only manual but it also to difficult to scale beyond a few VMs as it required customers to look for a specific set of strings from within the vmware.log file which is generated for each powered on VM, which could easily be several hundreds or thousands of VMs.

I figured there had to be a better way to help customers automate this at scale and remove the human factor. The other reason I was not fond of the current method is that it would require customers to either enable ESXi Shell/SSH access or to manually or through automation to download every single vmware.log file to inspect for specific log entries which can take quite a bit of time for any sizable environment. I had an idea on how this could be done without having to look at the vmware.log file while leveraging our vSphere APIs and did some investigation. Before proceeding further, please familiarize yourself with KB 52085. For complete background on both Spectre (CVE-2017-5753 & CVE-2017-5715) and Meltdown (CVE-2017-5754) as it relates to all VMware products, please carefully read through this top level KB 52245 which is being updated as new information is available, so you may want to subscribe to the KB for all the latest updates.

UPDATE 4 (01/23/18) - VMware has just updated KB 52345 with the current list of Intel CPUs affected by Intel Sightings. I have also updated my script to reflect all these changes. Make sure to download the latest version to ensure you have the latest changes.

UPDATE 3 (01/16/18) - I have just enhanced the script further to also collect the current microcode version running on a given ESXi host. Unfortunately, this information can only be collected when SSH is enabled and is something a user must explicitly allow. The benefit here is that Intel Sighting impact reporting is more robust as the code now checks for both impacted CPU signature as well as the microcode affected by Intel Sighting as outline in KB 52345. See below for more details.

UPDATE 2 (01/14/18) - As mentioned in the last update, I have been working on a Intel Sighting remediation script which can help customers automate the temporary workaround as recommended in KB 52345. Please see this blog post for complete details.

UPDATE 1 (01/13/18) - VMware just published a new KB 52345 outlining certain Intel Broadwell and Haswell CPUs being affected by Intel Sightings after applying latest microcode updates. Please refer to the KB for the complete details. I am currently working on a script to help with the remediation as the remediation method outlined in the KB would require customers to enable SSH access and manually update /etc/vmware/config. In the meantime, I wanted to provide a way for customers to easily identify whether their system are affected by Intel Sightings and thanks to community member Adam Robinson who just submitted a patch this morning to update my existing script to include these details. I have also added the CPU model to the output as additional information.

Note: This script only provides information about your existing vSphere environment, it does not make any changes or provides any remediation steps, please follow the KB for those instructions.

The PowerCLI script is called VerifyESXiMicrocodePatch.ps1 and it contains the following two functions:

  • Verify-ESXiMicrocodePatchAndVM
  • Verify-ESXiMicrocodePatch

[Read more...]