WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / ESXi / ESXi on Intel NUC 12 Pro (Wall Street Canyon)

ESXi on Intel NUC 12 Pro (Wall Street Canyon)

09.15.2022 by William Lam // 107 Comments

Many in the VMware Community, including myself, started with the classic 4x4 Intel NUC for running a VMware homelab. Over the years, this powerful little Intel NUC continues to enable a wide variety of new VMware use cases from running vSphere, vSAN, NSX, Tanzu and even vRealize (now Aria). It felt like it was just yesterday that I had switched from using an Apple Mac Mini to the latest Intel NUC (6th Generation) to build my new vSphere/vSAN Homelab, which was more than 6 years ago! 😲

While Intel continues to expand and grow their "NUC" portfolio to include many other form factors, the classic 4x4 design still has a special place for many people in the VMware community. The classic Intel NUC is not only small, portable but also extremely capable, especially with last few releases which makes this an ideal kit for those just getting started with a new VMware Homelab.

If you are in the market for an upgrade this year, definitely check out the latest refresh of this classic 4x4 design with Intel's recent launch of the Intel NUC 12 Pro, formally known as the Wall Street Canyon NUC.

Let's take a closer look at this new Intel NUC šŸ˜€

Compute

There are five different CPU configurations to select from across Intel Core i7, i5 and i3 processors, two of which include support for Intel vPro.

  • Intel 12th Generation IntelĀ® Core i7-1270P (vPro)
    • 12 Processor Cores (4P+8E), 16 threads, 18MB IntelĀ® Smart Cache, 35W TDP
    • P-Cores: 4.8GHz Turbo; E-Cores : 3.5GHz Turbo
  • Intel 12th Generation IntelĀ® Core i7-1260P
    • 12 Processor Cores (4P+8E), 16 threads, 18MB IntelĀ® Smart Cache, 35W TDP
    • P-Cores: 4.8GHz Turbo; E-Cores: 3.4GHz Turbo
  • Intel 12th Generation IntelĀ® Core i5-1250P (vPro)
    • 12 Processor Cores (4P+8E), 16 threads, 12MB IntelĀ® Smart Cache, 35W TDP
    • P-Cores: 4.4GHz Turbo; E-Cores: 1 3.3GHz Turbo
  • Intel 12th Generation IntelĀ® Core i5-1250P
    • 12 Processor Cores (4P+8E), 16 threads, 12MB IntelĀ® Smart Cache, 35W TDP
    • P-Cores: 4.4GHz Turbo; E-Cores: 3.3GHz Turbo
  • Intel 12th Generation IntelĀ® Core i3-1220P
    • 10 Processor Cores (2P+8E), 12 threads, 12MB IntelĀ® Smart Cache, 20W TDP
    • P-Cores: 4.4GHz Turbo; E-Cores: 3.3GHz Turbo

Similar to the Intel NUC 11 Pro, the Intel NUC 12 Pro will also include a "Slim" K or "Tall" H chassis option and the latter providing a secondary onboard network interface and two additional USB ports. All kits support up to 64GB SO-DIMM (DDR4-3200) similiar to previous Intel NUC generations.

For a more detailed breakdown across the various Intel NUC 12 Pro kits, please refer to the product brief.

Network


The built-in onboard network interface includes an Intel i225 (2.5GbE) which is similiar to previous Intel NUC models and is automatically recognized when using ESXi 8.0 or using the Community Networking Driver for ESXi Fling for those that wish to run earlier ESXi 7.0 versions.

For those interested in the Tall chassis option, you also have the ability to add a secondary Intel I225 (2.5GbE) expansion module that includes two additional USB ports. Again, ESXi 8.0 will automatically recognize the network adapter or you can use the Community Networking Driver for ESXi Fling if you plan to run ESXi 7.0. A couple of purchasing options for the expansion module is either SimplyNUC or Gorite.


If you need even more networking, you can take advantage of the two Thunderbolt 4 ports using these Thunderbolt 10GbE solutions for ESXi or look at USB-based networking by using the popular USB Network Native Driver for ESXi Fling.

Storage

There is support for 1 x M.2 PCIe x4 Gen 4 (2280) and 1 x M.2 SATA (2242) for the Slim chassis option. For those interested in vSAN, I recommend looking at the vendor KingShark, which has a compatible M.2 SATA (2242) which I have shared in a previous blog post using their 256GB SATA SSD. Historically, the Slim chassis only supported a single M.2 NVMe and now with an extra 2242 slot, you can run vSAN while still getting the benefit of the Slim chassis option.

For those interested in the Tall chassis option, you also have the ability to add an additional 2.5" SATA SSD on the back of the chassis lid, which will future proof your investment since you will now have up to three storage devices, one of which can be used to install the ESXi OSDATA. Although ESXi can be installed from USB, this option has been deprecated and will be removed post-ESXi 8.0, so something to really consider. See this blog post for additional considerations for vSphere 8.

If you need more storage or performant external storage options, you can also use the two Thunderbolt 4 ports and add these Thunderbolt M.2 NVMe solutions for ESXi which will give you plenty more storage capacity.

Graphics

Please see this blog post here for details on how to use the iGPU in passthrough mode with a VM.

ESXi


The latest release of ESXi 7.0 Update 3 installs on the Intel NUC 12 Pro without any issues, but it will require the use of the Community Networking Driver for ESXi Fling to recognize the onboard network devices. For those interested in running ESXi 8.0, no additional drivers are required as the Community Networking Driver for ESXi has been productized as part of the ESXi 8.0 release.

The following ESXi kernel optionĀ cpuUniformityHardCheckPanic=FALSEĀ needs to beĀ appendedĀ to the existing kernel line by pressing SHIFT+O during the boot up. If this entry is not added, then booting ESXi with processors that contain both P-Cores and E-Cores will result in an expected PSOD with following message "Fatal CPU mismatch on feature".

Here is aĀ video demonstrating the ESXi workaroundĀ for those interested in seeing the workaround applied visually.

Note:Ā Once ESXi has been successfully installed, you can permanently set the kernel option by running the following ESXCLI command:Ā esxcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSEĀ after rebooting or you can reboot host and take out the USB device and manually update efi\boot\boot.cfg and append the kernel option.

More from my site

  • VMware Cloud Foundation on Intel NUC?
  • ESXi on Intel NUC 12 Enthusiast (Serpent Canyon)
  • Updated findings for passthrough of Intel NUC Integrated Graphics (iGPU) with ESXi
  • ESXi on Intel NUC 12 Extreme (Dragon Canyon)
  • Considerations for future vSphere Homelabs due to upcoming removal of SD card/USB support for ESXi

Categories // ESXi, Home Lab, vSphere 7.0, vSphere 8.0 Tags // Intel NUC, Wall Street Canyon

Comments

  1. Jeff Newman says

    09/15/2022 at 1:03 pm

    Does ESXi properly handle the two core types in these 12-Generation processors? I thought that Windows 11 was the only operating system that currently does.

    Reply
    • Rhys Hammond says

      10/13/2022 at 8:02 pm

      I'd also like to understand if ESXi and it's hosted VMs are stable on this big.little architecture.

      Reply
      • William Lam says

        10/13/2022 at 8:51 pm

        Yes, I’ve been running workloads including VC for past month+ without any noticeable issues

        Reply
        • John M says

          03/09/2023 at 10:44 am

          Did you have to make any changes in the BIOS to run the virtual workload?

          Reply
          • William Lam says

            03/09/2023 at 10:44 am

            No, the default should be fine as long as Intel-VT is enabled. You may still want to customize based on your own needs, but typical stock BIOS will work out of the box

  2. c c says

    09/19/2022 at 11:32 pm

    Hi, how to order the additional LAN module?

    Reply
    • c c says

      09/19/2022 at 11:39 pm

      nvm, found the correct kit that includes the additional LAN and USB ports. Look for NUC12WSHv50L.

      Reply
  3. Occamsrazor says

    10/01/2022 at 3:08 pm

    The i7 Gall with Dual LAN and vPro looks like an awesome machine - NUC12WSHv70L - but will it ever actually become available? Why don’t they just sell the dual-LAN expansion module separately, this would simplify all the SKUs…

    Reply
  4. Martin Aslak says

    10/06/2022 at 11:00 am

    Make sure not to remove
    runweasel cdromboot else you will strugle alot to get it to work. my bootcfg statement is like this without ""
    "kernelopt=runweasel cdromBoot cpuUniformityHardCheckPanic=FALSE"
    the issue i had, was i wasnt able to install esxi on the harddrive it just booted from the usb disk.
    hope this can help others to save some hours šŸ˜‰

    Reply
    • Andrew Roderos says

      10/08/2022 at 8:38 am

      Thanks for this tip! William directed me to this and it's installed now. My current issue is configuration changes. It's not saving anything I do. Assigning license, localcli command covered here, etc.

      Reply
      • Andrew Roderos says

        10/08/2022 at 3:30 pm

        Seems like newer version of ESXi 7 doesn't like USB sticks anymore. Installed on NVMe and it's saving configs now.

        Reply
    • Derran says

      01/07/2023 at 12:08 pm

      I can't get ESXI 8 to install on a NUC12. Fatal CPU Mismatch.

      During boot if I do the shift O and type: cpuUniformityHardCheckPanic=FALSE
      it lets me continue, but doesn't finish the install, basically I'm booted from the usb disk.
      If i edit the boot.cfg on the usb when installing it doesn't work at all and i get a PSOD. Fatal CPU Mismatch.
      My boot.cfg statement is: kernelopt=runweasel cdromBoot cpuUniformityHardCheckPanic=FALSE

      Any help would be appreciated.

      Thanks,

      Reply
      • Derran says

        01/07/2023 at 12:15 pm

        The above was the one line from my boot.cfg, this is what my boot.cfg looks like:

        bootstate=0
        title=Loading ESXi installer
        timeout=5
        prefix=
        kernel=/b.b00
        kernelopt=runweasel cdromBoot cpuUniformityHardCheckPanic=FALSE
        modules=/jumpstrt.gz --- /useropts.gz --- /features.gz --- /k.b00 --- /uc_intel.b00 --- /uc_amd.b00 --- /uc_hygon.b00 --- /procfs.b00 --- /vmx.v00 --- /vim.v00 --- /tpm.v00 --- /sb.v00 --- /s.v00 --- /atlantic.v00 --- /bcm_mpi3.v00 --- /bnxtnet.v00 --- /bnxtroce.v00 --- /brcmfcoe.v00 --- /cndi_igc.v00 --- /dwi2c.v00 --- /elxiscsi.v00 --- /elxnet.v00 --- /i40en.v00 --- /iavmd.v00 --- /icen.v00 --- /igbn.v00 --- /ionic_en.v00 --- /irdman.v00 --- /iser.v00 --- /ixgben.v00 --- /lpfc.v00 --- /lpnic.v00 --- /lsi_mr3.v00 --- /lsi_msgp.v00 --- /lsi_msgp.v01 --- /lsi_msgp.v02 --- /mtip32xx.v00 --- /ne1000.v00 --- /nenic.v00 --- /nfnic.v00 --- /nhpsa.v00 --- /nmlx5_co.v00 --- /nmlx5_rd.v00 --- /ntg3.v00 --- /nvme_pci.v00 --- /nvmerdma.v00 --- /nvmetcp.v00 --- /nvmxnet3.v00 --- /nvmxnet3.v01 --- /pvscsi.v00 --- /qcnic.v00 --- /qedentv.v00 --- /qedrntv.v00 --- /qfle3.v00 --- /qfle3f.v00 --- /qfle3i.v00 --- /qflge.v00 --- /rdmahl.v00 --- /rste.v00 --- /sfvmk.v00 --- /smartpqi.v00 --- /vmkata.v00 --- /vmksdhci.v00 --- /vmkusb.v00 --- /vmw_ahci.v00 --- /bmcal.v00 --- /clusters.v00 --- /crx.v00 --- /drivervm.v00 --- /elx_esx_.v00 --- /btldr.v00 --- /esx_dvfi.v00 --- /esx_ui.v00 --- /esxupdt.v00 --- /tpmesxup.v00 --- /weaselin.v00 --- /esxio_co.v00 --- /loadesx.v00 --- /lsuv2_hp.v00 --- /lsuv2_in.v00 --- /lsuv2_ls.v00 --- /lsuv2_nv.v00 --- /lsuv2_oe.v00 --- /lsuv2_oe.v01 --- /lsuv2_sm.v00 --- /native_m.v00 --- /qlnative.v00 --- /trx.v00 --- /vdfs.v00 --- /vmware_e.v00 --- /vsan.v00 --- /vsanheal.v00 --- /vsanmgmt.v00 --- /tools.t00 --- /xorg.v00 --- /gc.v00 --- /imgdb.tgz --- /basemisc.tgz --- /resvibs.tgz --- /esxiodpt.tgz --- /imgpayld.tgz
        build=8.0.0-1.0.20513097
        updated=0

        Reply
        • Derran says

          01/09/2023 at 10:43 am

          So I figured out my issue. Editing the boot.cfg by adding "kernelopt=runweasel cdromBoot cpuUniformityHardCheckPanic=FALSE" didn't work for me. I had to do the SHIFT-O, however when I was adding the "cpuUniformityHardCheckPanic=FALSE" I wasn't appending it. I was deleting the runweasel cdromBoot. Once I appended it, I still get a quick error, but it allows me to install ESXI to the SSD. After that I added the ESXCLI command: "localcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE" to make the change permanent.

          thanks for the feedback

          Reply
          • William Lam says

            01/09/2023 at 12:21 pm

            boot.cfg is the same as SHIFT+O, you're basically editing the kernel options and as you said, you do NOT delete the existing lines (blog makes this clear) and you append.

            For the boot.cfg, you need to edit the one located in efi directory, since you're most likely using EFI firmware

            I didn't think this would be challenging but it seems a number of folks may not be familiar with ESXi Kernel Boot options and have ran into issues, so I've recorded a video that outlines this process to hopefully help those that need to see the steps visually. Should have that posted later today

          • William Lam says

            01/09/2023 at 12:48 pm

            Here's the blog post w/video demonstrating the workaround visually https://williamlam.com/2023/01/video-of-esxi-install-workaround-for-fatal-cpu-mismatch-on-feature-for-intel-12th-gen-cpus-and-newer.html

  5. 360coolp says

    10/10/2022 at 12:26 pm

    The TPM chip of the NUC 10 is not supported in combination with ESXi. Do you know whether the TPM chip of the NUC12 works?

    Reply
    • virtuguru says

      10/14/2022 at 2:15 pm

      Wondering that too. You need at least the vPro version, as this is the only version with TPM onboard.

      Reply
      • virtuguru says

        10/19/2022 at 12:18 am

        See the new post on this forum: the TPM chip in the Intel NUC 12 Pro is not supported by ESXi 8.0. Disappointing

        Reply
        • gbmaryland says

          12/30/2022 at 4:41 pm

          Nuts!

          I noticed that when I powered on the NUC and added it to vCenter. Very disappointing.

          "Host TPM attestation alarm"
          "TPM 2.0 device detected but a connection cannot be established"

          Honestly, I even have issues with TPM 2.0 devices on Dell servers, that came preinstalled with ESXi. -sigh-

          Reply
          • William Lam says

            01/02/2023 at 8:56 am

            Not all TPM devices are implemented equally ... see https://williamlam.com/2022/10/quick-tip-tpm-2-0-connection-cannot-be-established-after-upgrading-to-esxi-8-0.html for more details on the requirements for ESXi to use a TPM, this is really an issue with HW vendors and what they've chosen to implement

  6. dockyr says

    10/12/2022 at 8:39 am

    Why your esxi GUI screen show only 12 CPU?
    Not should be 16 with?:
    * 4 Core P hyperthreading --> 8?
    * 8 Core E

    Reply
    • dockyr says

      10/12/2022 at 8:42 am

      Sorry, my bad this amount is displaying over logical amount,
      Thanks for your work

      Reply
      • dockyr says

        10/13/2022 at 2:31 am

        And again xD
        I installed esxi or NUC12 today with 1260p and only 12 logical core appear :/

        Reply
        • Colin says

          10/22/2022 at 11:17 am

          I just managed to get 7.03g installed on my i5 12th gen nuc. I was also surprised to see that I see 12 logical processors rather than 16. So I wasnt sure if it was reading hyper-threading correctly.
          Also, on the esxi UI, when I look at hyperthreading it says "Yes, Disabled". Which is weird.

          Reply
  7. virtuguru says

    10/14/2022 at 2:04 pm

    Will a Samsung 980 Pro 2TB NMVe disk with heatfink fit in the small NUC version?

    Reply
  8. yosh says

    10/22/2022 at 1:43 am

    I am curious about gpu passthrough.
    NUC11 doesn't work.
    Does this get code 43 too?

    Reply
  9. Colin says

    10/22/2022 at 11:21 am

    Just a shout out to William as the tip about the "cpuUniformityHardCheckPanic=FALSE" really saved my frustration levels.
    I have one question for you, when you share your UI screenshots, they always look nicer than what I ever see. I thought it was a vCentre theme or something, but your ESXi 8 screenshot was a stand-alone. Keep up the great work, us home lab geeks really appreciate it. Thanks very much. Colin.

    Reply
    • William Lam says

      10/22/2022 at 12:53 pm

      Thanks Colin. The screenshot in this post is a standalone ESXi host, the vSphere UI from vCenter Server will look a different. You can also customize standalone ESXi Embedded Host Client UI, see https://williamlam.com/2022/10/quick-tip-accessing-new-custom-theme-editor-for-esxi-8-0-host-client.html

      Reply
  10. Gladki says

    10/31/2022 at 10:57 am

    I am new to building VMWare lab with NUC and just doing my research. What do I need vPRO Platform for ? What is it used for in general terms and to run VMWare lab?

    Thank you

    Reply
  11. Ivan says

    11/07/2022 at 8:58 pm

    Great article, was planning to refresh my lab with AMD built machines and now I've gone with these instead. One note .. you should change the 1260p comment at the top. The CPU does do 16 threads, but ESXi v7 (maybe 8 also) doesn't enable it. I am guessing that the disable panic directive also disables the hyperthreading on the P cores; as you just get "Inactive (Active on restart)" as a comment.

    Reply
  12. Al says

    11/12/2022 at 10:29 am

    Is this what the boot.cfg is supposed to look like to correct the PSOD ?

    bootstate=0
    title=Loading ESXi installer
    timeout=5
    prefix=
    kernel=/b.b00
    kernelopt=runweasel cdromBoot
    modules=/jumpstrt.gz --- /useropts.gz --- /features.gz --- /k.b00 --- /uc_intel.b00 --- /uc_amd.b00 --- /uc_hygon.b00 --- /procfs.b00 --- /vmx.v00 --- /vim.v00 --- /tpm.v00 --- /sb.v00 --- /s.v00 --- /atlantic.v00 --- /bnxtnet.v00 --- /bnxtroce.v00 --- /brcmfcoe.v00 --- /elxiscsi.v00 --- /elxnet.v00 --- /i40en.v00 --- /iavmd.v00 --- /icen.v00 --- /igbn.v00 --- /ionic_en.v00 --- /irdman.v00 --- /iser.v00 --- /ixgben.v00 --- /lpfc.v00 --- /lpnic.v00 --- /lsi_mr3.v00 --- /lsi_msgp.v00 --- /lsi_msgp.v01 --- /lsi_msgp.v02 --- /mtip32xx.v00 --- /ne1000.v00 --- /nenic.v00 --- /netcommu.v00 --- /nfnic.v00 --- /nhpsa.v00 --- /nmlx4_co.v00 --- /nmlx4_en.v00 --- /nmlx4_rd.v00 --- /nmlx5_co.v00 --- /nmlx5_rd.v00 --- /ntg3.v00 --- /nvme_pci.v00 --- /nvmerdma.v00 --- /nvmetcp.v00 --- /nvmxnet3.v00 --- /nvmxnet3.v01 --- /pvscsi.v00 --- /qcnic.v00 --- /qedentv.v00 --- /qedrntv.v00 --- /qfle3.v00 --- /qfle3f.v00 --- /qfle3i.v00 --- /qflge.v00 --- /rste.v00 --- /sfvmk.v00 --- /smartpqi.v00 --- /vmkata.v00 --- /vmkfcoe.v00 --- /vmkusb.v00 --- /vmw_ahci.v00 --- /bmcal.v00 --- /crx.v00 --- /elx_esx_.v00 --- /btldr.v00 --- /esx_dvfi.v00 --- /esx_ui.v00 --- /esxupdt.v00 --- /tpmesxup.v00 --- /weaselin.v00 --- /esxio_co.v00 --- /loadesx.v00 --- /lsuv2_hp.v00 --- /lsuv2_in.v00 --- /lsuv2_ls.v00 --- /lsuv2_nv.v00 --- /lsuv2_oe.v00 --- /lsuv2_oe.v01 --- /lsuv2_oe.v02 --- /lsuv2_sm.v00 --- /native_m.v00 --- /qlnative.v00 --- /trx.v00 --- /vdfs.v00 --- /vmware_e.v00 --- /vsan.v00 --- /vsanheal.v00 --- /vsanmgmt.v00 --- /tools.t00 --- /xorg.v00 --- /gc.v00 --- /imgdb.tgz --- /basemisc.tgz --- /resvibs.tgz --- /imgpayld.tgz
    build=7.0.3-0.55.20328353
    updated=0
    cpuUniformityHardCheckPanic=FALSE

    Reply
    • William Lam says

      11/12/2022 at 2:01 pm

      No. As article says, it’s appended to kernelopt line

      Reply
  13. Anton says

    11/16/2022 at 4:59 am

    I've got problem with installing ESXi 7.x and 8.0 on NUC 12 series. I've tried two ways: bootable flash and installations files and zalman hdd case with iso mount function, but got the same - ESXi boots from flash or iso (zalman) like it was already installed before... Installation wizard didn't start. Any suggestions?

    Reply
    • William Lam says

      11/16/2022 at 6:55 am

      You most likely did not correctly create bootable installer. Use unetbootin, it’s multi-platform and you give it ISO and USB device and it’s ready to go

      Reply
      • Anton says

        11/16/2022 at 11:34 am

        Thanks for suggestion William, but it didn't help. I've got the same - ESXi boots from Flash as it was already installed on it... I've tried to install Ubuntu and other Linux - no problem...

        Reply
        • William Lam says

          11/16/2022 at 12:07 pm

          Did you change any of the kernel option by chance when creating the installer? If you properly created an installer via USB, you will be prompted to install after it boots up, it will not just boot into ESXi. This only would happen if you somehow manually edited and specifically deleted lines that existed prior like runweasel string

          Reply
          • Anton says

            11/16/2022 at 12:17 pm

            I've re-downloaded ESXi from VMware, use the App you've suggested and press Shift+O before installation start for adding this option "cpuUniformityHardCheckPanic=FALSE" that's all. Should I check any BIOS settings?

          • William Lam says

            11/16/2022 at 12:39 pm

            correct, but make sure you append to line and not remove any of existing entries. If you do that, it will boot into installer and you should get to EULA screen. If you don’t see that, then something is wrong with your setup

          • Anton says

            11/16/2022 at 1:48 pm

            Read all messages once again in this post and found
            Martin Aslak posts 10/06/2022 at 11:00 am - it helped

          • William Lam says

            11/16/2022 at 1:52 pm

            which is exactly what I said in the blog post and in reply, APPEND do not remove existing entries šŸ™‚

  14. Ron Henderson says

    11/18/2022 at 3:08 pm

    I am interested in purchasing the IntelĀ® NUC 12 Pro X Kit - NUC12DCMv9 with vPro. Will this model work with ESXi 8.0.

    Processor: IntelĀ® Coreā„¢ i9-12900 Processor (30M Cache, up to 5.10 GHz)
    Chipset Support: IntelĀ® W680 Chipset

    Reply
    • William Lam says

      11/18/2022 at 3:55 pm

      That's the NUC 12 Extreme and yes, it works fine. See https://williamlam.com/2022/02/esxi-on-intel-nuc-12-extreme-dragon-canyon.html šŸ™‚

      Reply
  15. Antoine says

    12/04/2022 at 3:08 pm

    Hi William !

    Thx a lot for all those informations.
    i already got a home made esxi lab with an IntelĀ® Coreā„¢ i5-10500 Processor and i'd like to add a new server to create a cluter.
    I did some rearch and i'd like to order an intel nuc NUC12WSHI50Z with an IntelĀ® Coreā„¢ i5-1240P Processor and i was wondering if the CPUs will be compatible or if i won't be able to create a cluster and use vmotion.

    Maybe should i use EVC Mode ? In that case i really don't know what option i'll have to use and if i'll have some trouble with performances.

    can you help me about this please ? thx a lot

    ps : sorry english isn't my original language

    Reply
  16. Nick S says

    12/16/2022 at 9:00 am

    Hi William,

    First of all, great blog, very useful and informative!

    I recently purchased a NUC12 (NUC12WSHi7), with 500GB NVMe SSD as well as a 4TB SATA SSD. ESXi is installed on the NVMe storage and after some time once it's booted up (could be 10min or 2 hours), I get this message in the ESXi GUI:

    Lost connectivity to the device t10.NVMe____WD_BLACK_SN770_500GB____________________8B45634E8B441B00 backing the boot filesystem /vmfs/devices/disks/t10.NVMe____WD_BLACK_SN770_500GB____________________8B45634E8B441B00. As a result, host configuration changes will not be saved to persistent storage.

    I tried with both ESxi 7.0.3 and 8.0, same result. I am now thinking of maybe installing ESXi on the SATA SSD instead of the NVMe to see if that changes anything. But this is a strange issue. I have a NUC10 with same configuration and I don't have any problem.

    Would you have any idea what could cause this?

    Thanks!

    Reply
    • William Lam says

      12/16/2022 at 10:03 am

      The system is literally telling you what's wrong šŸ™‚

      >>Lost connectivity to the device ....

      It is probably the drive and/or connector or just bad NUC, its hardware and these things will happen. Double check that everything is secured as even slight looseness could cause issues

      Reply
      • Nick S says

        12/16/2022 at 11:55 am

        thanks, you just confirmed what I was suspecting but didn't want to admit to myself yet...now will try to find out which piece of hardware is causing this.

        Reply
      • Nick S says

        12/22/2022 at 5:02 pm

        For anyone facing similar issue, what worked me was to upgrade the firmware of the NVMe SSD. NUC is now fully operational!

        Reply
        • Cris says

          01/01/2023 at 10:32 pm

          How did you find the correct driver for your NVME SSD?

          Reply
          • Nick S says

            01/06/2023 at 6:44 am

            I had to install the SSD on a windows setup and use the driver utility update software from the vendor (WD in my case). I simply updated to the last firmware version.

  17. Seth says

    12/26/2022 at 9:31 am

    Hi:
    Happy Holidays and thanks for all you do! I've edited my boot.cfg file like so and am still getting the PSOD! What am I doing wrong?
    bootstate=0
    title=Loading ESXi installer
    timeout=5
    prefix=
    kernel=/b.b00
    kernelopt=runweasel cdromBoot cpuUniformityHardCheckPanic=FALSE
    modules=/jumpstrt.gz --- /useropts.gz --- /features.gz --- /k.b00 --- /uc_intel.b00 --- /uc_amd.b00 --- /uc_hygon.b00 --- /procfs.b00 --- /vmx.v00 --- /vim.v00 --- /tpm.v00 --- /sb.v00 --- /s.v00 --- /atlantic.v00 --- /bcm_mpi3.v00 --- /bnxtnet.v00 --- /bnxtroce.v00 --- /brcmfcoe.v00 --- /cndi_igc.v00 --- /dwi2c.v00 --- /elxiscsi.v00 --- /elxnet.v00 --- /i40en.v00 --- /iavmd.v00 --- /icen.v00 --- /igbn.v00 --- /ionic_en.v00 --- /irdman.v00 --- /iser.v00 --- /ixgben.v00 --- /lpfc.v00 --- /lpnic.v00 --- /lsi_mr3.v00 --- /lsi_msgp.v00 --- /lsi_msgp.v01 --- /lsi_msgp.v02 --- /mtip32xx.v00 --- /ne1000.v00 --- /nenic.v00 --- /nfnic.v00 --- /nhpsa.v00 --- /nmlx5_co.v00 --- /nmlx5_rd.v00 --- /ntg3.v00 --- /nvme_pci.v00 --- /nvmerdma.v00 --- /nvmetcp.v00 --- /nvmxnet3.v00 --- /nvmxnet3.v01 --- /pvscsi.v00 --- /qcnic.v00 --- /qedentv.v00 --- /qedrntv.v00 --- /qfle3.v00 --- /qfle3f.v00 --- /qfle3i.v00 --- /qflge.v00 --- /rdmahl.v00 --- /rste.v00 --- /sfvmk.v00 --- /smartpqi.v00 --- /vmkata.v00 --- /vmksdhci.v00 --- /vmkusb.v00 --- /vmw_ahci.v00 --- /bmcal.v00 --- /clusters.v00 --- /crx.v00 --- /drivervm.v00 --- /elx_esx_.v00 --- /btldr.v00 --- /esx_dvfi.v00 --- /esx_ui.v00 --- /esxupdt.v00 --- /tpmesxup.v00 --- /weaselin.v00 --- /esxio_co.v00 --- /loadesx.v00 --- /lsuv2_hp.v00 --- /lsuv2_in.v00 --- /lsuv2_ls.v00 --- /lsuv2_nv.v00 --- /lsuv2_oe.v00 --- /lsuv2_oe.v01 --- /lsuv2_sm.v00 --- /native_m.v00 --- /qlnative.v00 --- /trx.v00 --- /vdfs.v00 --- /vmware_e.v00 --- /vsan.v00 --- /vsanheal.v00 --- /vsanmgmt.v00 --- /tools.t00 --- /xorg.v00 --- /gc.v00 --- /imgdb.tgz --- /basemisc.tgz --- /resvibs.tgz --- /esxiodpt.tgz --- /imgpayld.tgz
    build=8.0.0-1.0.20513097
    updated=0

    TIA, Seth

    Reply
    • Seth says

      12/27/2022 at 10:31 am

      UPDATE:
      I hit shift-O on boot and manually entered " kernelopt=runweasel cdromBoot cpuUniformityHardCheckPanic=FALSE" and then hit enter, the unit boots up but stops at "vmtoolsd". After about 2 minutes, I can see a PSOD in the background and then it reboots into an Install screen again.
      I'm using just one WD SN570 NVMe SSD on board as my install drive, do I need to load drivers for it? Why am I getting stuck on "vmtoolsd"?
      Thanks..

      Reply
      • Derran says

        01/07/2023 at 12:19 pm

        This is exactly where I am stuck, and I'm using the same hard drive.
        Did you find any solution

        Reply
        • virtuguru says

          01/07/2023 at 1:39 pm

          Did you disable secure boot?

          Reply
          • Derran says

            01/07/2023 at 2:50 pm

            It is now. Same error. Still won't let me finish the install.
            Anything else that should be disabled in Bios?
            thanks,

        • Seth says

          01/08/2023 at 9:21 am

          Not yet. Tried with ESXi 8.0. Created bootable ISO on USB via Rufus and edited /EFI/boot/boot.cfg as per the above. 1st time install works fine, but after taking out USB and reboot it gets stuck in an install loop. Going to try recreating ISO on USB and *not* editing anything on the USB itself to see if that works.

          Reply
          • virtuguru says

            01/08/2023 at 9:26 am

            As far as I understand, after the reboot, you have to press Shift-O again. Only after ESXi is running, you can make the change permanent.

          • Derran says

            01/09/2023 at 10:44 am

            Seth, try this:

            So I figured out my issue. Editing the boot.cfg by adding "kernelopt=runweasel cdromBoot cpuUniformityHardCheckPanic=FALSE" didn't work for me. I had to do the SHIFT-O, however when I was adding the "cpuUniformityHardCheckPanic=FALSE" I wasn't appending it. I was deleting the runweasel cdromBoot. Once I appended it, I still get a quick error, but it allows me to install ESXI to the SSD. After that I added the ESXCLI command: "localcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE" to make the change permanent.

            thanks for the feedback

  18. robertdepriest says

    12/26/2022 at 11:49 am

    Looking at purchasing 4 x NUC12WSHi5 (tall option) to run vSAN. Should I get two drives or three? With the deprecation of USB, I was thinking of the 256GB Kingshark (2242) for the equivalent of a boot drive, 500 GB M2 (2280) for a general drive, and 2TB SSD in the expansion slot in the tall module for the vSAN capacity drive. Is this necessary, or can I get by with two drives overall? Do I need a dedicated cache SSD for vSAN?

    Thanks,

    Robert

    Reply
  19. gbmaryland says

    01/01/2023 at 6:33 am

    In one of the post someone mentioned the use of EVC with the CPUs for the NUC 12 Pro i5-1240ps.

    I did some playing around with the EVC settings, and unfortunately if you have old Broadwell based hosts (think Tinker Try Supermicro hosts), you will not get an EVC setting compatible with the NUC12WSHi5 and other variants.

    (I got the Supermicro mini servers when I was still working for VMware as they supported 128GB of RAM and we very low power. Unfortunately, they are old now...)

    Reply
  20. virtuguru says

    01/06/2023 at 4:23 am

    Hi William. I am at the point of ordering an Intel 12th gen NUC for my ESXi 8.0 environment. I doubt between the NUC12WSKi5 and NUC12WSKv5. They are exactly the same, except for the NUC12WSKv5 having vPro and TPM. The TPM version in this NUC is currently not supported in ESXi 8.0. Don't know if it will be supported in the future, maybe after a firmware update? Might it be wise anyway to choose the vPro version? Of just a waste of money?

    Reply
  21. gbmaryland says

    01/10/2023 at 12:24 pm

    I've got 3 x NUC12WSKi5 "Pros" setup as ESXi 8 Hosts.

    I'm currently using Thunderbolt 3 > 10G SFP+ adapters.

    I've tried to add NVMe Thunderbolt 3 strorage, but the ESXi 8 hosts never detect it.

    Anyone know if there is a specifics driver set that needs to be added? Or if the NUC12s don't like being fully populated? (They do not have the additional 2.5gbit expansion module installed, but do have SATA storage in the lid; no SATA storage on the mainboard.)

    Reply
    • William Lam says

      01/10/2023 at 12:28 pm

      Not all SSD will be detected, especially lesser know consumer brands. I typically recommend Samsung, Intel and WD as these just work

      Reply
      • gbmaryland says

        01/10/2023 at 12:33 pm

        Thanks for the quick reply SuperLam!!

        Humph. They are all Samsung. 940/950/970 EVOs and one Pro.

        I'm going to assume ESXi doesn't like the OWC Envoy Express.

        Reply
        • William Lam says

          01/10/2023 at 12:43 pm

          Few things to try - Reseat the devices, you'd be surprised at how often the issue is physical including the SSD themselves šŸ™‚

          Ensure the TB ports are enabled, BIOS may not always have everything enabled, so good thing to check and ensure BIOS sees the devices. You can always rule out TB chassis by simply plugging the device in directly on motherboard and ensure its functional. I know folks use 970 pretty often, not sure about the others but I'd expect ESXi to be able to see them. Also, make sure devices are clean (use parted util or something as partitions may reside causing issues but typically you'd still see the device just not able to use it because it contains data)

          Reply
          • gbmaryland says

            01/10/2023 at 2:44 pm

            Interestingly, looking at the BIOS, there is really nothing related to thunderbolt other than a single checkbox.

            I did check to see whether everything was seated properly, etc., but it doesn’t seem to be the issue.

    • virtuguru says

      01/10/2023 at 1:11 pm

      Make sure your hardware is also on the NUC qualified vendor list.

      Reply
      • gbmaryland says

        01/10/2023 at 2:41 pm

        Oddly enough, when you type in NUC ESXi HCL or ā€œhardware compatibility listā€ you don’t get anything useful.

        Can you provide a link to the NUC HCL?

        Reply
        • virtuguru says

          01/10/2023 at 8:39 pm

          https://compatibleproducts.intel.com/ProductDetails?activeModule=Intel%C2%AE%20NUC&prdName=NUC12WSKi5

          Reply
          • gbmaryland says

            01/11/2023 at 2:12 am

            Rock on, thanks!

  22. gbmaryland says

    01/11/2023 at 12:28 pm

    After playing around with a ton of Thunderbolt 3 devices, I've decided that the NUC12WSHi5 doesn't want the second Thunderbolt port to light up. I suspect that the power draw might be an issue. ...but that's just a suspicion.

    It's very interesting that all three NUC12s exhibit the same symptoms.

    The BIOS is the latest available, and there are no robust BIOS options for Thunderbolt.

    FWIW: The HCL for Thunderbolt on Intel's site is extremely limited, and most of the devices are older. It's clear that Thunderbolt is being eclipsed by USB4. (...but I'm glad to have it.)

    Humph. The mystery continues.

    Reply
    • William Lam says

      01/11/2023 at 1:09 pm

      You could also look at TB chassis that support more than single NVMe, this way you can use it if it’s due to power (I’ve not tested beyond single BUT I have done easy-chaining since TB supports that) https://williamlam.com/2019/06/thunderbolt-3-enclosures-with-single-dual-quad-m-2-nvme-ssds-for-esxi.html

      Reply
  23. gbmaryland says

    01/14/2023 at 12:11 pm

    Ok, How to get both Thunderbolt Ports to work with ESXi 8 on the NUC 12 Pro WSH:

    Turns out that the ports on the NUC 12 Pro are REALLY sensitive to detecting and allowing newly plugged in devices to work.

    The primary thing you have to (aside from turning around two times in a circle, while patting your head and running your belly) is that you MUST unplug the NUC prior to connecting new thunderbolt devices. TURNING OFF THE NUC DOES NOT WORK!!

    I tried every permutation to get a thunderbolt 10G NIC (QNAP) and an OWC Envoy Express (Samsung 970 EVO 1TB) to work AT THE SAME TIME. Nothing.

    As such, there is clearly another issue: It doesn't appear that multiple non-self powered Thunderbolt devices will work on the NUC 12 Pro. (There is also squat for information on Intel's website regarding this.)

    However, IF you get a thunderbolt powered HUB, you can then plug one of the devices into the hub... and then BOTH devices work. (The one powered but the NUC 12 Pro, and the other powered by the hub.) [OWC Thunderbolt Hub - OWCTB4HUB5P].

    I have noticed an issue connecting the 10G QNAP and storage device to the HUB at the same time. Neither appear on the ESXi host, and the hub never appears to come online. However, if I plug only one device on the hub, I can get it to work IF, and only IF, I plug all the devices in as I want them with the power cord UNPLUGGED from the NUC 12 Pro.

    I'm going to try adding addition storage devices to the hub and see if they are all detected / presented under ESXi.

    The cool thing about the OWC hub is that it lights up blue if there is a viable connection established with the NUC upon boot / driver load completion; it remains white if it's not working.

    Reply
    • gbmaryland says

      01/14/2023 at 3:51 pm

      Oh, and one more thing: I had to use my MacBook Pro to erase the prior ESXi installations from the thunderbolt NVMe SSDs, or the hosts would boot into the previous ESXi installation on those NVMe drives.

      Reply
    • TheDragon says

      02/11/2023 at 7:42 am

      Hi there

      Could you confirm if you tested any further with multiple devices using a hub?

      Would like to run a few 10gbe SFP+ thunderbolt adaptors, currently I can only get one working which I understand is due to the lack of power on the second thunderbolt interface.

      So I was thinking with a powered thunderbolt hub connected to the second interface I could hopefully use two adaptors this way, is it possible to use anymore than 1 device on the second interface via a powered thunderbolt hub?

      Reply
      • GBMaryland says

        02/11/2023 at 9:36 am

        I’ve had hit and miss luck… But it does appear that if you connect multiple 10 gig thunderbolt adapters to the external hub, they will all work.

        It seems that if the devices are a mix of thunderbolt three and thunderbolt four, they don’t work.

        The advantage of the external hub is that it’s actually powered and this allows your devices to work, but I haven’t been able to plug all my devices into the hub and get that to work. Specifically, I haven’t been able to get networks and storage to work plugged into the hub but I can get it to work if I plug the thunderbolt 10g adapter into the nuc and the hub gets the storage.

        Reply
  24. gbmaryland says

    01/14/2023 at 12:53 pm

    One other thing that I’ve noticed is that the ESXI hosts are crashing because they get confused with the P cores and the E cores.

    I’m getting purple screens of death as the NUC 12 becomes heavily utilized during operations, such as cloning a virtual machine.

    I wish there was an effective way to turn off the E or P cores, and allow hyperthreading as necessary.

    Honestly, if the host keep crashing, the new lab won’t be a usable thing.

    Reply
    • Notedd says

      02/02/2023 at 4:03 pm

      Anyone else having this problem? I am thinking about getting a few of these for lab builds, but if they crash under load I will look of another option.

      Reply
      • virtuguru says

        02/02/2023 at 11:49 pm

        No problem here. Did some vMotions and clones, no sweat.

        Reply
  25. Shofiqul says

    01/15/2023 at 6:03 pm

    Hi.. I am greatful to your article but my question is a bit different.. from which store I get the secondary Intel I225 (2.5GbE) expansion module? I am really want to get that but unable to find it on web. Would you please help me sharing the link of the nic card?

    Reply
    • GBMaryland says

      01/16/2023 at 5:11 am

      SimplyNUC either in the US/CA, UK, or EU.

      I was able to order them from there.

      Reply
      • Shofiqul says

        01/16/2023 at 5:39 am

        Thanks a lot.. I will check if they take only the expansion order

        Reply
        • William Lam says

          01/16/2023 at 5:59 am

          You can also purchase from https://www.gorite.com/lan-gigabit-add-on-module-with-dual-usb-2-0-ports-for-tiger-canyon-and-wall-street-nuc which Intel shared awhile back when Tiger Canyon was released. I'll update the article to mention this and SimplyNUC as options for purchasing the expansion adapter

          Reply
          • Shofiqul says

            01/16/2023 at 6:08 am

            on simplynuc and gorite.com out of stock..but thanks a lot

          • gbmaryland says

            01/16/2023 at 4:42 pm

            SimpleNUC ordered them for me and it took a week for them to arrive. So they can be ordered….

  26. virtuguru says

    01/28/2023 at 10:44 am

    Did anyone already figure out the most optimal BIOS settings for the NUC12? Especialy the power settings?

    Reply
    • GBMaryland says

      01/28/2023 at 11:25 am

      I left them the default settings and set the fans the balanced.

      I then used the NUC 11 settings from either your post or William Lams post on the matter.

      I will say that it is very important to update the bios the moment you get the computer, because there are some changes in the most recent versions of the bios.

      Reply
      • virtuguru says

        01/28/2023 at 11:28 am

        Yes, I updated the BIOS too. But set the power to max performance. The NUC seems pretty fast now šŸ™‚ Sadly, there are many settings in the BIOS where there is no description to be found. Intel's documentation is way behind...

        Reply
        • GBMaryland says

          01/29/2023 at 3:02 am

          Yeah, I have to agree that entails documentation on pretty much everything with the NUC12 is nonexistent. I’ve been looking for any hint of thunderbolt compatibility / thunderbolt use caveats with the NUC12… and there is nothing to be found

          Reply
          • virtuguru says

            01/30/2023 at 6:50 am

            Well, I have my NUC running for 2 days now. Works pretty wel. I have set the power to max performance in the BIOS. However, when I look in esxtop > p, I notice that P-states P0, P1, P2 and P3 are not used. The first used P-state is P4. According to the Pstate Mhz table, this is 1700Mhz, whereas P0 would be 2101Mhz. How can we get P0 to P3 to be active as well? Sadly, I can't post a screenshot here...

  27. virtuguru says

    01/30/2023 at 11:25 am

    Anyone wanting to build an ESXi server that supports TPM in Windows 11, please read this.
    I have a NUC setup which natively supports the TPM chip. To be able to do so, you’ll need the vPro version of the NUC. These are the NUCs of which the model number ends with ā€œv5ā€ or ā€œv7ā€. I purchased the NUC12WSKv5. It has vPro onboard, which means a discrete TPM chip is present (instead of a virtual TPM chip like in other models, which is not supported in ESXi 8).
    If you install ESXi 8.0 and vCenter, you can create a cluster. This will allow the TPM chip to be passed on to any VM. Now, you can install Windows 11 with the TPM chip (add the TPM security device from vCenter). vCenter then reports for the VM "Encrypted with a native key providerā€

    My setup:
    1x Intel NUC 12 Pro NUC12WSKv5
    2x Crucial CT32G4SFD832A
    1x Transcend 430S 256GB (as boot device)
    1x Samsung 980 Pro (without heatsink) 2TB (primary datastore)

    Reply
    • gbmaryland says

      01/30/2023 at 2:57 pm

      I’m confused: Don’t the NUC12WSHi5 / NUC12WSKi5 have TPM 2.0s?

      Reply
      • virtuguru says

        01/30/2023 at 9:32 pm

        Not according to the specs as listed on the Intel site. These offer Intel Platform Trust Technology (PTT) which includes fTPM 2.0. This might be the reason of the ""TPM 2.0 device detected but a connection cannot be established" errors, as it is not a real chip but a firmware implemenatation of TPM. The v5 NUC has a real chip, which works fine as I discovered.

        Reply
  28. proad says

    02/05/2023 at 1:43 pm

    Why does esxi display 12 cpu instead of 16 on your screenshot ? hyperthreading is not supported ?

    Reply
    • William Lam says

      02/05/2023 at 1:51 pm

      Correct, due to the non-uniform CPU features, ESXi can’t support HT and those are disabled. For systems that allow for E-Cores to be turned off, then you’d get benefit of P-Cores + HT

      Reply
  29. Rafal says

    02/19/2023 at 10:38 am

    I have NUC NUC12WSHI5 Core i5-1240P 2000GB 64GB, I've tried install ESXI 8 but doesn't see SSD drive, how can I fix it?

    Reply
    • virtuguru says

      02/19/2023 at 10:40 am

      what SSD?

      Reply
      • Rafal says

        02/19/2023 at 11:09 am

        Hard drive SSD (M.2) 2GB

        Reply
        • virtuguru says

          02/19/2023 at 11:12 am

          I mean, what brand and type etc.

          Reply
          • Rafal says

            02/19/2023 at 11:24 am

            SATA III, PCIe type M.2 rest of information will check in BIOS

          • Rafal says

            02/20/2023 at 12:14 am

            SDD nvme - SAMSUNG MZVL22T0HBLB-00B00

  30. Rafal says

    02/19/2023 at 11:10 am

    Can't install system because ESXI doesn't see any hard drive

    Reply
    • virtuguru says

      02/20/2023 at 6:57 am

      Although the SAMSUNG MZVL22T0HBLB-00B00 isn't on the compatible devices list of Intel, it's the OEM version of the 980 Pro 2TB I have, and the 512GB and 256GB versions of the PM91A1 are on the list. So, you might assume it should work. Can you see the device in the BIOS? Did you re-seat the card? Did you do a bios update? If you install Windows, do you see the device? I suspect there is something off, but I can't pinpoint it from here.

      Reply
  31. David says

    02/25/2023 at 9:40 pm

    Hi, is there any way to get temperature from nuc12. I don’t found anything. I installed esx 8 successfully and now I would like to have a monitoring. thanks

    Reply
    • William Lam says

      02/25/2023 at 9:47 pm

      No, there are no IPMI drivers to get that information

      Reply
  32. Mark says

    03/12/2023 at 5:23 pm

    I noticed on the Wall Street Canyon that with the 101.4146 driver it doesn't throw the BSOD thread error. When it starts and having svga present it has error code 43 in dev mgr. If I disable and then reenable then it shows normal. but I can't seem to get any displays recognized. It seems really close to working but not sure what else it might be

    Reply
  33. wujean says

    03/22/2023 at 4:52 am

    How do I use the Thunderbolt 3 interface in esxi8.0, it cannot be assigned to a virtual machine on my nuc9

    Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • Changing the default HTTP(s) Reverse Proxy Ports on ESXi 8.0 03/22/2023
  • Quick Tip - How to download ESXi ISO image for all releases including patch updates? 03/15/2023
  • SSD with multiple NVMe namespaces for VMware Homelab 03/14/2023
  • Is my vSphere Cluster managed by vSphere Lifecycle Manager (vLCM) as a Desired Image or Baseline? 03/10/2023
  • Interesting VMware Homelab Kits for 2023 03/08/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...