WilliamLam.com

ESXi 7.0 Update 2 Upgrade Issue – Failed to load crypto64.efi

by // 34 Comments

I started to notice yesterday that a few folks in the community were running into the following error after upgrading their ESXi hosts to latest 7.0 Update 2 release:

Failed to load crypto64.efi

Fatal error: 15 (Not Found)

UPDATE (04/29/2021) - VMware has just released ESXi 7.0 Update 2a which resolves this issue and includes other fixes. Please make sure to read over the release notes and do not forget to first upgrade your vCenter Server to the latest 7.0 Update 2a release which came out earlier this week.

UPDATE (03/13/2021) - It looks like VMware has just pulled the ESXi online/offline depot and has updated KB 83063  to NOT recommend customers upgrade to ESXi 7.0 Update 2. A new patch is actively being developed and customers should hold off upgrading until that is made available.

UPDATE (03/10/2021) - VMware has just published KB 83063 which includes official guidance relating to the issue mentioned in this blog post.

Issue

It was not immediately clear to me on how folks were reaching this state and I had reached out to a few folks in the community to better understand their workflow. It turns out that the upgrade was being initiated from vCenter Server using vSphere Update Manager (VUM) and applying a custom ESXi 7.x Patch baseline to remediate. Upon reboot, the ESXi host would then hit the error as shown above.


Interestingly, I personally have only used Patch baselines for creating ESXi patches (e.g. 6.7p03, 7.0p01) and never for major ESXi upgrades. I normally would import the ESXi ISO and create an Upgrade baseline. At least from the couple of folks I spoke with, it seems like the use of Patch baseline is something they have done for some time and has never given them issues whether it was for a patch or major upgrade release.

Workaround

I also had some folks internally reach out to me regarding this issue and provided a workaround. At the time, I did not have a good grasp of what was going on. It turns out the community also figured out the same workaround, including how to recover an ESXi host which hits this error as you can not just go through recover workflow.

For those hitting the error above, you just need to create a bootable USB key with ESXi 7.0 Update 2 ISO using Rufus or Unetbootin. Boot the ESXi 7.0 Update 2 Installer and select the upgrade option which will fix the host.

To prevent this from happening, instead of creating or using a Patch baseline, create an Upgrade baseline using ESXi 7.0 Update 2 ISO. You will first need to go to Lifecycle Manager Management Interface in vCenter Server and under "Imported ISOs", import your iage.


Then create ESXi Upgrade baseline and select the desired ESXi ISO image and use this baseline for your upgrade.


I am not 100% sure, but I believe the reason for this change in behavior is mentioned in the ESXi 7.0 Update 2 release notes under "Patches contained in this Release" section which someone pointed me to. In any case, for major upgrades, I would certainly recommend using Upgrade baseline as that is something I have always used even when I was a customer back in the day.

Quick Tip - Using ESXi to send Wake-on-Lan (WoL) packet

by // 1 Comment

The ability to power on a system over the network using Wake-on-Lan (WoL) can be extremely useful, especially if you are not physically near the system or after a power outage. I personally have been using the wakeonlan utility on my macOS system for several years now.

The syntax is super easy, you just provide the MAC Address of your system:

wakeonlan 54:b2:03:9e:70:fc
Sending magic packet to 255.255.255.255:9 with 54:b2:03:9e:70:fc

I recently came to learn that ESXi itself has the ability to send a WoL packet from the ESXi Shell! This could be handy without having to install WoL client, especially if you have access to an ESXi host.

vsish -e set /net/tcpip/instances/defaultTcpipStack/sendWOL 192.168.30.255 9 54:b2:03:9e:70:fc vmk0

This uses the not supported vsish CLI to send WoL packet. The first argument is the network broadcast address, so if you have a network of 192.168.30.0/24, then the address would be 192.168.30.255. The second argument is a value of 9, which is probably related to the magic packet as you can see the same value from the wakeonlan utility abvoce. The third argument is the MAC Address of the system and finally the fourth and final argument is the ESXi VMkernel interface to send the packet out of.

Easily create custom ESXi Images from patch releases using vSphere Image Builder UI

by // 11 Comments

Creating a custom ESXi Image Profile that incorporates additional ESXi drivers such as the recently released Community Networking Driver for ESXi Fling or Community NVMe Driver for ESXi Fling is a pretty common workflow. Due to the infrequency of this activity, many new and existing users sometime struggle with the process to quickly construct a new custom ESXi Image Profile. I personally prefer to use the Image Builder UI that is built right into the vSphere UI as part of vCenter Server.

There are a couple of ways to create a custom new ESXi Image Profile using the Image Builder UI, but the easiest method is to use the Clone workflow, which is especially helpful when you are selecting an ESXi patch release as your base image.

With a regular major release, you only have to deal with two image profiles: standard (includes VMware Tools) and no-tools (does not include VMware Tools).

With an ESXi patch release, you actually have four image profiles: standard (includes VMware Tools + all bug/security fixes), security standard (includes VMware Tools + security fixes only), security no-tools (does not include VMware Tools + security fixes only) and no-tools (does not include VMware Tools + all bug fixes)

If you start with an empty custom image profile and then select your ESXi base image, you will notice there are multiple VIB version packages to select from since patch release you had imported earlier actually contains four different ESXi image profiles. Below are a step by step instructions on using the cloning workflow since this is a question I get from users who run into package conflicts not realizing they have selected the same package multiple times.

[Read more...]