WilliamLam.com

ACPI motherboard layout requires EFI - Considerations for switching VM firmware in vSphere 8 

by // 1 Comment

One of the important settings to consider when creating a new Virtual Machine in vSphere is the VM firmware, which can either be BIOS or EFI and can be configured under VM Options->Boot Options->Firmware. After selecting the desired guest operating system (GOS) in vSphere, the system will default to a recommended firmware type and can also be overridden by the user. Ultimately, the selection of the VM firmware should be determined by what your GOS supports.

If you ever need to change the VM firmware, you typically will need to re-install the GOS because it does not understand the new firmware change (just like in a physical server) and more than likely the GOS will also not boot due to this change and this is the existing behavior from GOS point of view.

For a net new VM creation, prior to vSphere 8, if you had configured a VM using EFI firmware and you have not installed a GOS and realized that you had made a mistake and needed to change the VM firmware to BIOS, you could easily do so using the vSphere UI or API and then install your OS. In vSphere 8 and specifically when using the latest Virtual Machine Compatibility (vHW20), you can not just switch the VM firmware after the initial VM creation, especially if you had started with EFI firmware and wish to change it to BIOS.

In doing so, you will come across the following error message:

ACPI motherboard layout requires EFI. Failed to start the virtual machine. Module DevicePowerOnEarly power on failed.

[Read more...]

Applying additional security hardening enhancements in ESXi 8.0

by // 14 Comments

While responding to a few ESXi security configuration questions, I was referencing our ESXi Security documentation, which includes a lot of useful information and latest best practices. It is definitely worth re-reviewing this section from time to time to take advantage of all the ESXi security enhancements to help protect and secure your vSphere environment.

In certain areas of the ESXi security documentation, I noticed that it mentions CLI and API, but it does not always provide an example that customers can then reference and use in their Automation, which is really the only guaranteed method to ensure configurations are consistent across your vSphere environment. After answering some of the security related questions, especially on the Automation examples, I figure it would be useful to share this information more broadly so that folks are aware of some of the new and existing security enhancements along with some of their implications if you are not implementing them.

Speaking of new ESXi security enhancements, one of the new features that was introduced in ESXi 8.0 is the ability to disable ESXi Shell access for non-root users. While this might sound like a pretty basic feature, applying this towards the vCenter Server service account vpxuser can help add another layer of protection for your ESXi hosts against attackers. It turns out that users with ESXi Shell access can also modify other local users password on ESXi host including the root user. By restricting ESXi Shell access for the vpxuser, you prevent attackers, which can also be insiders who have access to vCenter Server the ability to just change the ESXi root password without knowing the original password. As a result, this can lock you out of your ESXi hosts or worse, enable an attacker to encrypt your workloads, especially as the rise ransomeware attacks has been increasing.

[Read more...]

Video of ESXi install workaround for Fatal CPU mismatch on feature for Intel 12th Gen CPUs and newer

by // 50 Comments

I have been noticing more and more users that have acquired hardware that includes the latest Intel 12th Generation CPU (Alder Lake) and even the newest Intel 13th Generation CPU (Raptor Lake) for use with ESXi. Starting with the Intel 12th Generation CPU, Intel has introduced a new hybrid "big.LITTLE" CPU architecture that integrates two types of CPU cores: Performance-cores (P-cores) and Efficiency-cores (E-cores) into the same physical CPU die.

ESXi is currently not aware of this new consumer architecture and it currently expects all cores within a CPU package to have uniform characteristics. If you boot the ESXi installer, it will PSOD (Purple Screen of Death) by default and you will see a message about "Fatal CPU mismatch on feature" which is due to the different CPU properties across both the P-Cores and E-Cores. However, there is a way to workaround the issue by disabling the CPU uniformity check that ESXi performs as part of its boot up.

UPDATE (01/16/24) - See this blog post on some updated experiments using CPU affinity when both E-Cores and P-Cores are enabled when using ESXi.

UPDATE (04/22/23) - If you decide NOT to disable either E-Cores or P-Cores, you may also run into an additional PSOD when powering on a VM with GP exception in world message. To workaround this problem, please see this blog post HERE.

UPDATE (03/24/23) - It is possible and recommended to actually disable the E-cores within the Intel NUC BIOs following the instructions HERE to prevent ESXi from PSOD'ing due to non-uniform CPU cores rather than applying the ESXi boot option workaround as described in the video below.

I initially wrote about the solution back in Feb of 2022 where this new CPU was first introduced in the Intel NUC line with the Intel NUC 12 Extreme (Dragon Canyon) and subsequently, I had also wrote about the solution reviewing both the Intel NUC 12 Pro (Wall Street Canyon) and the Intel NUC 12 Enthusiast (Serpent Canyon).

While the majority of folks have not had any issues applying the workaround, I have started seeing some folks running into challenges, perhaps its familiarity with ESXi or applying kernel options. In any case, I figured it might help to record a video demonstrating the workaround for those that rather visualize the solution along with the written instructions (included below).

[Read more...]