WilliamLam.com

Google Chat space notification for failed vCenter Server Appliance (VCSA) backups using VEBA

by // Leave a Comment

It is always good idea to verify that your vCenter Server Appliance (VCSA) backups are occurring and if there are any failures, you are notified about them immediately!

As alluded to at the end of that blog post, you can easily build event-driven automation using the popular VMware Event Broker Appliance (VEBA) solution, triggering notifications based on several types of VCSA backup events:

  • com.vmware.applmgmt.backup.job.failed.event
  • com.vmware.applmgmt.backup.job.finished.event
  • com.vmware.applmgmt.backup.job.start.event

I recently built a PowerShell function that would send a notification to a Google Chat space by enabling their incoming webhook integration as you can seen from the screenshot below:


While the message customization is not as extensive as using Slack (e.g. supporting advanced layouts and emoji), it does get the job done and you will know immediately when a backup has failed or when it has started or has completed.

Note: For existing VEBA users (pre-v0.8 release), the function.yaml needs to be updated to trigger off of the subject keyword rather than the type, which is due to replacing the event router with Tanzu Sources for vSphere. You also need to ensure the prefix of "com.vmware.vsphere" is removed along with trailing "v0" as this is the next syntax for vCenter Events once VEBA v0.8 is released.

spec:
  broker: default
  filter:
    attributes:
      subject: com.vmware.applmgmt.backup.job.failed.event

Quick Tip - Verifying vCenter Server Appliance (VCSA) Backup Status

by // 13 Comments

I recently found out that my vCenter Server Appliance (VCSA) scheduled backups had been failing for some time but I was not aware. While there is a default vCenter Server alarm for failed VCSA backups, however it is only visible when you have selected the vCenter Server inventory object in left hand navigation, which is not a typical item that I select as shown in the screenshot below.


Furthermore, I was actually logged into the VCSA's VMware Appliance Management Interface (VAMI) not too long ago while updating to a newer patch release and I had checked to see if backups were successful, which I thought they were when looking at the "Status" column.


It was not until I took a closer look, did I realize that the schedule backup activity log is ordered from oldest to newest, meaning what I saw was a backup that has happend a long time ago and not necessary the latest backup attempt. I found this to be a really strange UX since as you would need to page to very end to check whether the latest backups were succssful or not.

Note: I have already filed an internal bug to ask for the VCSA schedule backup activity to be ordered from newest to oldest, so users can quickly see if the latest backups are successful or not.

[Read more...]

Experimenting with ESXi CPU affinity and Intel Hybrid CPU Cores

by // 21 Comments

After debugging a recent issue with using VMware Workstation and Intel Hybrid CPUs, it gave me an idea about an experiment to try with ESXi and Intel Hybrid CPUs.

As a refresher, starting with the Intel 12th Generation (Alder Lake) CPU, a new hybrid big.LITTLE CPU architecture was introduced for consumer Intel CPUs. This new hybrid Intel CPU architecture integrates two types of CPU cores: Performance-cores (P-cores) and Efficiency-cores (E-cores) into the same physical CPU die. For more information about this new hybrid Intel CPU design, check out this resource HERE. The ESXi scheduler does not and has no current plans to support this new Intel Hybrid CPU architecture, especially as this type of architecture is nowhere to be found in traditional Enterprise datacenters and is only limited to Intel Consumer CPUs.

The current recommendation to work around the non-uniformity of the CPU cores is to either disable the E or P-cores within the system BIOS, thus making the system "uniform" and allowing ESXi to run like a normal x86 system. While you can apply a workaround to have ESXi ignore the non-uniformity of the CPU cores, in addition to the non-deterministic behaviors, random PSOD can also occur due to scheduling across two different types of cores.

I was curious to see whether applying ESXi CPU affinity on a VM using Intel Hybrid CPU Cores might yield a different outcome?

[Read more...]