Earlier this week I found out the new Intel NUC "Skull Canyon" (NUC6i7KYK) has been released and have been shipping for a couple of weeks now. Although this platform is mainly targeted at gaming enthusiast, there have also been a lot of anticipation from the VMware community on leveraging the NUC for a vSphere based home lab. Similiar to the 6th Gen Intel NUC system which is a great platform to run vSphere as well as VSAN, the new NUC includes a several new enhancements beyond the new aesthetics. In addition to the Core i7 CPU, it also includes a dual M.2 slots (no SATA support), Thunderbolt 3 and most importantly, an Intel Iris Pro GPU a Thunderbolt 3 Controller. I will get to why this is important ...
UPDATE (05/26/16) - With some further investigation from folks like Erik and Florian, it turns out the *only* device that needs to be disabled for ESXi to successfully boot and install is the Thunderbolt Controller. Once ESXi has been installed, you can re-enable the Thunderbolt Controller and Florian has also written a nice blog post here which has instructions as well as screenshots for those not familiar with the Intel NUC BIOs.
UPDATE (05/23/16) - Shortly after sharing this article internally, Jason Joy, a VMware employee shared the great news that he has figured out how to get ESXi to properly boot and install. Jason found that by disabling unnecessary hardware devices like the Consumer IR/etc in the BIOS, it allowed the ESXi installer to properly boot up. Jason was going to dig a bit further to see if he can identify the minimal list of devices that needed to be disabled to boot ESXi. In the meantime, community blogger Erik Bussink has shared the list of settings he has applied to his Skull Canyon to successfully boot and install latest ESXi 6.0 Update 2 based on the feedback from Jason. Huge thanks to Jason for quickly identifying the workaround and sharing it with the VMware community and thanks to Erik for publishing his list. For all those that were considering the new Intel NUC Skull Canyon for a vSphere-based home lab, you can now get your ordering on! 😀
Below is an except from his blog post Intel NUC Skull Canyon (NUC6I7KYK) and ESXi 6.0 on the settings he has disabled:
BIOS\Devices\USB
- disabled - USB Legacy (Default: On)
- disabled - Portable Device Charging Mode (Default: Charging Only)
- not change - USB Ports (Port 01-08 enabled)
BIOS\Devices\SATA
- disabled - Chipset SATA (Default AHCI & SMART Enabled)
- M.2 Slot 1 NVMe SSD: Samsung MZVPV256HDGL-00000
- M.2 Slot 2 NVMe SSD: Samsung MZVPV512HDGL-00000
- disabled - HDD Activity LED (Default: On)
- disabled - M.2 PCIe SSD LEG (Default: On)
BIOS\Devices\Video
- IGD Minimum Memory - 64GB (Default)
- IGD Aperture Size - 256 (Default)
- IGD Primary Video Port - Auto (Default)
BIOS\Devices\Onboard Devices
- disabled - Audio (Default: On)
- LAN (Default)
- disabled - Thunderbolt Controller (Default is Enabled)
- disabled - WLAN (Default: On)
- disabled - Bluetooth (Default: On)
- Near Field Communication - Disabled (Default is Disabled)
- SD Card - Read/Write (Default was Read)
- Legacy Device Configuration
- disabled - Enhanced Consumer IR (Default: On)
- disabled - High Precision Event Timers (Default: On)
- disabled - Num Lock (Default: On)
BIOS\PCI
- M.2 Slot 1 - Enabled
- M.2 Slot 2 - Enabled
- M.2 Slot 1 NVMe SSD: Samsung MZVPV256HDGL-00000
- M.2 Slot 2 NVMe SSD: Samsung MZVPV512HDGL-00000
Cooling
- CPU Fan HEader
- Fan Control Mode : Cool (I toyed with Full fan, but it does make a lot of noise)
Performance\Processor
- disabled Real-Time Performance Tuning (Default: On)
Power
- Select Max Performance Enabled (Default: Balanced Enabled)
- Secondary Power Settings
- disabled - Intel Ready Mode Technology (Default: On)
- disabled - Power Sense (Default: On)
- After Power Failure: Power On (Default was stay off)
Over the weekend, I had received several emails from folks including Olli from the nucblog.net (highly recommend a follow if you do not), Florian from virten.net (another awesome blog which I follow & recommend) and few others who have gotten their hands on the "Skull Canyon" system. They had all tried to install the latest release of ESXi 6.0 Update 2 including earlier versions but all ran into a problem while booting up the ESXi installer.
The following error message was encountered:
Error loading /tools.t00
Compressed MD5: 39916ab4eb3b835daec309b235fcbc3b
Decompressed MD5: 000000000000000000000000000000
Fatal error: 10 (Out of resources)
Raymond Huh was the first individual who had reach out to me regarding this issue and then shortly after, I started to get the same confirmations from others as well. Raymond's suspicion was that this was related to the amount of Memory-Mapped I/O resources being consumed by the Intel Iris Pro GPU and does not leave enough resources for the ESXi installer to boot up. Even a quick Google search on this particular error message leads to several solutions here and here where the recommendation was to either disable or reduce the amount of memory for MMIO within the system BIOS.
Unfortunately, it does not look like the Intel NUC BIOS provides any options of disabling or modifying the MMIO settings after Raymond had looked which including tweaking some of the video settings. He currently has a support case filed with Intel to see if there is another option. In the mean time, I had also reached out to some folks internally to see if they had any thoughts and they too came to the same conclusion that without being able to modify or disable MMIO, there is not much more that can be done. There may be a chance that I might be able to get access to a unit from another VMware employee and perhaps we can see if there is any workaround from our side, but there are no guarantees, especially as this is not an officially supported platform for ESXi. I want to thank Raymond, Olli & Florian for going through the early testing and sharing their findings thus far. I know many folks are anxiously waiting and I know they really appreciate it!
For now, if you are considering purchasing or have purchased the latest Intel NUC Skull Canyon with the intention to run ESXi, I would recommend holding off or not opening up the system. I will provide any new updates as they become available. I am still hopeful that we will find a solution for the VMware community, so crossing fingers.