WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple
You are here: Home / ESXi / Video of ESXi install workaround for Fatal CPU mismatch on feature for Intel 12th Gen CPUs and newer

Video of ESXi install workaround for Fatal CPU mismatch on feature for Intel 12th Gen CPUs and newer

01.09.2023 by William Lam // 50 Comments

I have been noticing more and more users that have acquired hardware that includes the latest Intel 12th Generation CPU (Alder Lake) and even the newest Intel 13th Generation CPU (Raptor Lake) for use with ESXi. Starting with the Intel 12th Generation CPU, Intel has introduced a new hybrid "big.LITTLE" CPU architecture that integrates two types of CPU cores: Performance-cores (P-cores) and Efficiency-cores (E-cores) into the same physical CPU die.

ESXi is currently not aware of this new consumer architecture and it currently expects all cores within a CPU package to have uniform characteristics. If you boot the ESXi installer, it will PSOD (Purple Screen of Death) by default and you will see a message about "Fatal CPU mismatch on feature" which is due to the different CPU properties across both the P-Cores and E-Cores. However, there is a way to workaround the issue by disabling the CPU uniformity check that ESXi performs as part of its boot up.

UPDATE (01/16/24) - See this blog post on some updated experiments using CPU affinity when both E-Cores and P-Cores are enabled when using ESXi.

UPDATE (04/22/23) - If you decide NOT to disable either E-Cores or P-Cores, you may also run into an additional PSOD when powering on a VM with GP exception in world message. To workaround this problem, please see this blog post HERE.

UPDATE (03/24/23) - It is possible and recommended to actually disable the E-cores within the Intel NUC BIOs following the instructions HERE to prevent ESXi from PSOD'ing due to non-uniform CPU cores rather than applying the ESXi boot option workaround as described in the video below.

I initially wrote about the solution back in Feb of 2022 where this new CPU was first introduced in the Intel NUC line with the Intel NUC 12 Extreme (Dragon Canyon) and subsequently, I had also wrote about the solution reviewing both the Intel NUC 12 Pro (Wall Street Canyon) and the Intel NUC 12 Enthusiast (Serpent Canyon).

While the majority of folks have not had any issues applying the workaround, I have started seeing some folks running into challenges, perhaps its familiarity with ESXi or applying kernel options. In any case, I figured it might help to record a video demonstrating the workaround for those that rather visualize the solution along with the written instructions (included below).

Video Instructions

Manual Instructions

Step 1 - During the bootup of the ESXi installer, you will see an option to to append additional ESXi kernel boot settings. Press SHIFT+O and append the following kernel option cpuUniformityHardCheckPanic=FALSE to the command line and the press enter to continue the boot process.

Step 2 - Install ESXi by following the install wizard and once you are prompted to reboot, do not reboot yet. We need to add the kernel option again so that ESXi can successfully boot after the installation. To do so, switch into the ESXi shell by pressing ALT+F1 and login using root and blank password as ESXi has not gone through full reboot and is not using the configured password.

Step 3 - Edit /vmfs/volumes/BOOTBANK1/boot.cfg and append following kernel option cpuUniformityHardCheckPanic=FALSE to existing kernelopt entry, which should look like the following:

kernelopt=weaselInstalled autoPartition=FALSE cpuUniformityHardCheckPanic=FALSE

Save your changes and then change back to the reboot prompt by pressing ALT+F2 and then reboot.

Instead of having to manually append the ESXi kernel option, you will notice that it has been appended due to Step2 and applying the setting in the boot.cfg file.

Step 3 - To permanently configure the ESXi kernel boot setting, in case of updates/upgrades in the future, we can set the kernel setting using ESXCLI. We first need to login to DCUI by pressing F2 and then navigating to Troubleshooting Option and enable ESXi Shell. Switch into the ESXi shell by pressing ALT+F1 and login with root and the password you had configured during installation. Now run the following ESXCLI command to configure the kernel option:

esxcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE

More from my site

  • Google Coral USB Edge TPU Accelerator on ESXi
  • How to enable passthrough for USB Network Adapters claimed by ESXi CDCE Driver?
  • Quick Tip - Enabling ESXi Coredumps to be stored on USB
  • Automated ESXi Installation with a USB Network Adapter using Kickstart
  • ESXi with Intel Arc 750 / 770 GPU

Categories // ESXi, vSphere 7.0, vSphere 8.0 Tags // ESXi 7.0, ESXi 8.0

Comments

  1. *protectedgbmaryland says

    01/09/2023 at 2:19 pm

    Excellent post man! I noticed a lot of people having this problem and truth be told if you haven’t worked for Vmware color or you haven’t done a bunch of automation for Vmware… You might not be familiar with the OS options. You

    Reply
    • *protectedJoao says

      11/26/2023 at 5:05 am

      I just bought Intel NUC 13 Pro I5 and with the steps in this article it was possible to install and stop the psod. Thanks.

      Reply
  2. *protectedJ says

    01/09/2023 at 7:12 pm

    Does this give you access to the e-cores as well and p-cores?

    Reply
    • William Lam says

      01/09/2023 at 7:16 pm

      Yes

      Reply
      • *protectedAkshay Bharambe says

        11/16/2024 at 10:28 am

        Hello William,

        I am also testing this on the NUC 14 Pro+ 185H and ran into PSOD. I did disable E cores. do we need E cores for better performance of the VMs?

        Reply
        • William Lam says

          11/16/2024 at 10:38 am

          Please follow the guidance in my NUC 14 reviews, simply disabling won’t help 🙂

          Reply
  3. *protectedMatjahS; says

    01/10/2023 at 12:23 am

    Very helpful. When dealing with keyboard shortcuts at different prompts, a video is a so much easier to follow. 🙂 Thank you!

    Reply
  4. tahasever+*protectedTaha says

    01/10/2023 at 12:37 pm

    How about the scheduler? Because esxi doesn’t aware of the cores, did you see any scheduler related performance issues or power consumption issues like assigning the tasks to wrong core type? In my opinion AMD is golden standard for me since 12th gen intels. Or does it see the core types as different numa nodes? Or are you able to choose core type to assign to the vm. For example little cores for light tasks like dns-dc-automation services etc. and p cores for personal vm , dbs, and more power hunger vms?

    Reply
    • William Lam says

      01/10/2023 at 12:46 pm

      When I say ESXi does not understand the architecture, I'm referring to scheduler.

      In my limited testing in deploying vCenter and other infrastructure including Tanzu Kubernetes Grid (TKG), I've not seen any noticeable difference in behavior. It runs snappier than previous generations (as I'd expect) but again, it may vary based on workloads. Most VMware-based homelabs, the resource that is usually stressed is memory and CPU is generally idle ... so may not make a difference but again, all workload dependent and there's no expected behavior given the ESXi scheduler doesn't see a difference and you could be schedule on either E or P cores. On some Intel systems, you can disable the E-Cores all together, but not something I've seen on the Intel NUCs

      Reply
  5. *protectedmtorrijosgtp says

    01/11/2023 at 9:02 am

    Thank you, thank you, thank you, thank you....it works 100%

    Reply
  6. *protectedDerran says

    01/11/2023 at 12:57 pm

    Great video. It helped a lot.
    Thanks

    Reply
  7. *protectedlovegoodbest says

    01/25/2023 at 6:39 pm

    Thanks for your guide.

    I think the P core hyper threading is not well supported yet on 12th gen.

    My hardware is 1240p, which has 4P cores (total 8 threads in theory) and 8E cores (total 8 threads).
    I can only see 12 cpus in ESXI8, and hyper threading can't be enabled.

    Reply
    • William Lam says

      01/25/2023 at 6:44 pm

      HT is indeed disabled when the cores are non-uniform

      Reply
  8. *protectedDennis de Kok says

    02/15/2023 at 7:04 am

    Hi William thx for the blog it helped me a lot, i used the command esxcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE with SSH and that works better for me after the reboot.

    Reply
  9. *protectedLapaj Go says

    02/26/2023 at 10:04 pm

    This workaround no longer works after Dell BIOS update!

    Anybody have any idea when will VMware get serious and FIX this bug? (along with the 100s things that break so easily, and as a result require an actual developer to troubleshoot - no wonder professionals prefer to pay more for AWS.)

    Reply
    • *protectedTaha Sever says

      02/27/2023 at 5:06 am

      Probably it is not going to happen in the near future. Because VMware is driving the ESXi server and certified hardware exclusive more and more each day. Which is not so bad in my opinion. because intel's big little architecture is designed directly for consumer electronics not for Servers yet. The best and robust way to resolve this issue is disabling the e cores from bios and this will give you avx support back too or if you are building a system just choose the cpu which is not using the mixed core design. I am wondering if new amd 7900x3d and 7950x3d chips are going to be effected from this issue because their only one chiplet has 3dvcache. I belive 7800x3d will not be effected because it has one chiplet and all cores have same design.

      Reply
  10. *protectedFrancys says

    03/14/2023 at 5:54 pm

    same here, i applied the command but stills gives me the pink screen before it finishes, any other work around?

    Reply
    • William Lam says

      03/14/2023 at 8:17 pm

      You need to re-watch the video and ensure you've followed the instructions carefully.

      Reply
      • *protectedDon L says

        01/15/2025 at 3:49 pm

        You have been great and much appreciate the time and effort in sharing your knowledge. I recently purchased a NUC 14 and have tried the bypasscpu check and disable/enable the e cores and no matter what I do I still get the PSOD. If you had any thoughts or directions on what to try next to it would be GREATLY appreciated.

        Reply
        • William Lam says

          01/15/2025 at 4:39 pm

          Please re-read/watch video as you’ve probably missed setting after initial install. Also, see my NUC 14 review and why it’s different and you can’t disable cores anymore 🙂

          Reply
  11. *protectedMustafa E. says

    04/06/2023 at 1:21 am

    In this way, does it allow to use both E core and P cores at full load / efficiency?

    Reply
    • William Lam says

      04/06/2023 at 10:30 am

      You'll be able to use all the E and P Cores (no HT)

      Reply
      • *protectedMatt says

        04/20/2023 at 12:19 pm

        If you cannot enable HT on these new CPU’s, is the new CPU the best choice then? Doesn’t an older generation i5 or i7 with HT give you more performance?

        Reply
        • William Lam says

          04/21/2023 at 12:47 pm

          You _can_ enable HT if you disable the E-Cores (assuming your BIOS allows for it). Here's how you do it for Intel NUC https://williamlam.com/2023/03/how-to-disable-the-efficiency-cores-e-cores-on-an-intel-nuc.html and ultimately it'll depend on your needs. While you can certainly look at older platforms, some may want to invest in modern systems that'll last for the next several years and with older systems, there's always a risk and also some of the consumer Intel NUC have a shorter support lifecycle. I know the latest Intel NUC 13 Pro now have 5yrs of support where it is typically 2-3 at best. If you want to use the last generation w/o hybrid architecture, look at 11th Gen NUCs

          Reply
          • *protectedAlex H says

            12/29/2023 at 5:13 pm

            Glad I came back to read old posts and comments. I was wondering why I have HT disabled on my new little box. I found I am better served with 8 more e-cores, even if slower, than 4 HT p-cores. I'm really surprised and excited with the punch these little NUCs pack - jumping from an i5-3400 to an i7-1360p with double the memory, the difference is night and day. Glad I spent years exercising patience.

            What I also found out is that the Watchdog setting, if enabled on the BIOS, makes the installation to fail. I was puzzled at first why it was stopping at TPM initialization (even though you said it was irrelevant to ESXi). Disabled it, then the installation began failing on the network interfaces prep-work. Ended getting the BIOS set to the very basic settings - one P core, absolutely everything disabled and went turning things on until they passed. With the time set on the watchdog, I got crashes again, which were indeed timely: from moving from the boot screen to loading something, the BIOS reset the system as per watchdog's request - most likely because it cannot interpret (or ESXi does not publish) the OS as alive.

  12. *protectedMark H says

    04/21/2023 at 9:58 am

    William,
    My CPU is a i9-13900k. I watched the video and got my ESXI 8.0 host up and running, thank you!. I've applied each patch as they've been released. Today I applied the ESXI 8.0 Update 1. The host still boots up fine, however now, when I try to power on a guest VM it crashes. I'd love to send you a screenshot if your interested. I'll give you the top two lines of the crash dump output on the screen. line1 = "#GP Exception 13 in world 2099894:vmx-vcpu-2:S @ 0x42000df74ec9" line2 = "Module(s) involved in panic: [vmkernel Version Releasebuild-21495797]

    I open to any and all of your suggestions 🙂

    Reply
    • *protectedMark H says

      04/21/2023 at 11:29 am

      I reviewed your updated article around these CPUs. I went ahead and disabled all the efficiency cores in the BIOS as suggested. Now ESXI shows 8 cores with 16 threads (HT enabled). I know I'm losing 16 efficiency cores, but now I can power on my VM guests after the updated. This is a prime example of why a XEON processor of the same performance costs $3000+ LOL.

      Reply
      • William Lam says

        04/22/2023 at 8:11 am

        Mark,

        I just recalled there was a workaround after debugging issue w/Engr, so if you still want to enable all cores, then you will need one additional workaround which I have just published https://williamlam.com/2023/04/esxi-psod-due-to-gp-exception-13-in-world-with-intel-13th-generation-cpu.html

        As mentioned already, this workaround shouldn't be needed in a future ESXi update and only the ESXi kernel option for ignoring the CPU uniformity in the future.

        Reply
    • William Lam says

      04/21/2023 at 12:44 pm

      I was actually going to suggest disabling the E-Cores (https://williamlam.com/2023/03/how-to-disable-the-efficiency-cores-e-cores-on-an-intel-nuc.html) as this behavior looks to only occur with the 13th Gen CPUs. I've already reported this issue internally and a fix will be available in a future update of ESXi, but for now, disabling E-Core will be your best option.

      Reply
  13. *protectedkelvin koh says

    04/25/2023 at 4:41 am

    so far i tested enable P core and Ecore in my ESXi 8 with my 13Gen i13900 CPU and install VM with windows 2022 and Centos 9 without issue. however seem like there is no way to enable hyperthreading, the Host summary show Hyperthreading = Yes, Disabled. i try to check the advance setting and seem Hyperthreading is enable.

    Reply
    • William Lam says

      04/25/2023 at 4:58 am

      Please see https://williamlam.com/2023/03/how-to-disable-the-efficiency-cores-e-cores-on-an-intel-nuc.html for your answer

      Reply
      • *protectedKelvin Koh says

        07/08/2023 at 2:29 pm

        Hi William, seem like disabled E-Core is the only way to be able to use HT, I m currently running disabled it. 8 P core with HT is 16. if with e-Core enable total will be 24, wonder which option will be better performance

        Reply
  14. *protectedchuan says

    06/12/2023 at 8:09 am

    any latest update on esxi supporting 12th Intel CPU P & E Cores
    still need to disable E core wasted the E core ?

    Reply
    • William Lam says

      06/12/2023 at 8:22 am

      There are no plans as these CPU are only found in consumer grade platforms which VMware does not support. You can either disable as mentioned in the article or apply the workaround, pick the one that fits your needs

      Reply
  15. *protectedStone says

    08/17/2023 at 1:15 am

    Quick question, performance wise, which way is better? disable E cores, or cpuUniformityHardCheckPanic ? Thanks,

    Reply
    • *protectedTaha Sever says

      08/17/2023 at 4:16 am

      Actually there is no single correct answer for this question.

      - in total, keeping the ecores enabled gives you more threads and more performance points. but Unfortunately vmwarecpu scheduler doesn't aware oh the asymmetric design of the cores. if your CPU performance depenend application will be assigned to e core you will see the performance drop.

      - On the other hand if you will sacrifice the ecore performance, then disable them. And you will have more consistent performance without having crush risks

      Reply
  16. *protectedDaniel says

    09/01/2023 at 3:43 pm

    hey,

    I currently have an Intel NUC 13 ANHi7 and am currently trying to install VMware ESXi 8.0.1 21813344.
    The problem with cpuUniformityHardCheckPanic=FALSE or disabling the efficient cores is not a problem but for me the installation gets stuck at the following point.

    cndi_igc loaded successfully
    activating: vmkdevmgr

    I can't find a post here or on other sites to fix this and I keep reading that others have this problem too.

    Is there already a solution for this?

    I will try it now with version 7.x.

    Reply
  17. *protectedDaniel says

    09/19/2023 at 2:03 pm

    Hi, I have a HP elite mini G9 with an intel i7-12700.
    I set cpuUniformityHardCheckPanic=FALSE and ignoreMsrFaults=TRUE.

    Unfortunately I get a PSOD after done hours with following message:
    Machine Check Exception on PCPU9 in world 2101451:vmast .210144
    System has encountered a Hardware Error - Please contact the hardware vendor

    Has anyone an idea?

    Best regards

    Reply
    • William Lam says

      09/19/2023 at 2:52 pm

      MCE typically point to hardware issue ... you could run diagnostic using the vendor utilities and update all the firmware/BIOS if you've done so

      Reply
      • *protectedDaniel says

        09/20/2023 at 10:33 pm

        Hi William,
        Thanks for your answer.
        I can run the diagnostic tool.
        But actually the machine is running since 2 days. I disabled the complete energy management for the cpu.
        Is it possible, that the esx can’t handle the c-stats? Or is that a reason of the a hardware fault?

        Best regards

        Reply
        • William Lam says

          09/21/2023 at 6:35 am

          ESXi can handle C-States fine https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-resource-management/GUID-4D1A6F4A-8C99-47C1-A8E6-EF3865603F5B.html, so I don't think its related to that ... its consumer hardware after all, so this shouldn't come as a surprise if something else is having issues, but MCE is typically accurate that its actual HW issue.

          Reply
          • *protectedDaniel says

            10/01/2023 at 1:35 am

            Hi, Unfortunately the problem occurred again. CPU stress test was successful. I have an old photo of the PSOD. Can you see more clearly what the problem is?

            https://cloud.danigl.de/s/za8SgxaDmczF28y

            Many thanks and best regards

  18. *protectedSerge says

    01/06/2024 at 9:28 am

    Thank you so much for your tutorial, it helped me a lot. I am so glad that all works fine now.

    The NUC 12 Pro (Intel® Core i7-1260P, NUC12WSHi7 ) is a great product and the Vmware ESXI8 is a great software.

    I have a very small issue with the sound.
    When i use the virtual machine installed on the esxi8, it don't have any sound on the client side.

    The virtual machine seem okay but no sound is heard on the real PC.
    (on client side)

    It seems that there is no compatible sound driver NUC 12 Pro for the esxi8.
    It's very frustrating.

    Can you please help me ?
    Where can we find a sound driver for ESXI8 for the NUC 12 Pro ?

    Thanks for your great tutorial.

    Reply
  19. *protectedANTONIO BUSSI says

    04/24/2024 at 12:35 pm

    Hi William,
    Thanks for your guides.
    I have an HP Elite Mini 800 G9 i9 12th and when I try to install ESXi 7.x I get the PSOD (Purple Screen of Death) error. I tried entering the command cpuUniformityHardCheckPanic=FALSE while booting ESXi but without success.
    It is not possible to disable the E cores in the PC BIOS and I cannot currently install the ESXi environment. Do you have any suggestions on this problem?

    A thousand thanks

    Best regards

    Anthony

    Reply
    • William Lam says

      04/24/2024 at 12:54 pm

      Are you sure the PSOD is due to non-uniform CPU cores ... often times, it can be other things like memory, storage, etc.

      I also don't know what you mean by entering command and its not successful. This also leads me to believe, this may not be related and its something else causing PSOD 🙂

      Reply
  20. *protectedANTONIO BUSSI says

    04/25/2024 at 1:10 am

    Hello William,
    I certainly explained myself badly, when booting ESXi 7.0u3 from the USB stick, I get the error message: HW features incompatiblity detected..., restarting the boot and inserting the cpuUniformity.... string, once the loading is complete the PC reboots again and only after the second reboot by always adding the cpuUnifornity.... string the PC complete the loading.
    Is it normal for the PC to reboot a second time after completing the first load?

    Many thanks

    Best regards

    Antonio

    Reply
  21. *protectedPouria says

    06/06/2024 at 10:25 am

    Hello William,

    Please tell me which keys will save the changes on edite boot option.

    Thanks

    Reply
  22. *protectedDemo says

    06/06/2024 at 7:45 pm

    Thank you so much you are a life saver

    Reply
  23. *protectedFilmon says

    06/19/2024 at 3:02 am

    Dear William Lam
    After installing VM8, I do not have access to the disks. Do you have any idea how I can resolve this issue? Thanks.

    Reply
  24. *protectedFerdi says

    08/10/2024 at 4:02 am

    Thanks for preparing such document.

    I have another problem, can you pls support if you faced similar issues?
    I recently got a Lenovo m90q and has the SSD as "t10.NVMe____SKHynix_HFS001TEJ9X162N".

    I installed ESXi 7.0.3 (many options Lenovo Custom and other subversions) and tried many diferrent Lenovo drivers. But still the I/O data rate doesnt exceed 1MB/s, I cant upload a file to datastore or create a VM.

    I also changed many CPU features (enabled/disabled) on BIOS but nothing. Any idea what the reason can be?

    Reply

Thanks for the comment!Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025
  • vCenter Identity Federation with Authelia 04/16/2025
  • vCenter Server Identity Federation with Kanidm 04/10/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025