Deploying an ESXi scripted installation aka Kickstart running within a VM (Nested ESXi) has a number of benefits, especially for testing and development purposes. This was something I did regularly as a customer, especially with new releases of ESXi to ensure our existing automation scripts and processes continued to work before rolling out into production. ESXi kickstart itself is pretty straight forward, but the required supporting infrastructure (PXE Server, DHCP, TFTP, etc) that needs to be configured, especially for a greenfield deployment can often be challenging for new comers.
Even with an existing PXE infrastructure, it can often be difficult to configure or troubleshoot depending on your level of access which does not add any value in actually testing or automating the ESXi scripted installation process. In ESXi 7.0 Update 2, an enhancement was made to the Virtual Machine's UEFI firmware called VirtualEFI that would enable ESXi to perform an HTTP Boot given the ESXi bootloader URL and without requiring any of the traditional PXE infrastructure.
To take advantage of this new capability, you just need to have a physical server running ESXi 7.0 Update 2 and a VM that is configured with the latest vHW19 compatibility. To configure HTTP boot, you will need to add the following two VM Advanced Settings:
- networkBootProtocol - httpv4 or httpv6
- networkBootUri - HTTP URL to the ESXi bootloader (bootx64.efi)
Disclaimer: Nested ESXi and Nested Virtualization is not officially supported by VMware
Step 1 - Download and extract the desired ESXi ISO to your HTTP server. In my example, I am using ESXi 7.0 Update 1c and have extracted the contents into a directory called esxi70u1c and placed that into the root of my HTTP server which is /etc/httpd/html
Step 2 - You will need to update the ESXi boot.cfg configuration which is located in efi/boot directory with the following:
- Remove the leading "/" character in front of all module names
- Update the prefix parameter with the base URL of the extracted ESXi image
Here is what my /etc/httpd/html/esxi70u1c/efi/boot/boot.cfg looks like after applying the two changes mentioned above:
title=Loading ESXi installer
modules=jumpstrt.gz --- useropts.gz --- features.gz --- k.b00 --- uc_intel.b00 --- uc_amd.b00 --- uc_hygon.b00 --- procfs.b00 --- vmx.v00 --- vim.v00 --- tpm.v00 --- sb.v00 --- s.v00 --- bnxtnet.v00 --- bnxtroce.v00 --- brcmfcoe.v00 --- brcmnvme.v00 --- elxiscsi.v00 --- elxnet.v00 --- i40en.v00 --- i40iwn.v00 --- iavmd.v00 --- icen.v00 --- igbn.v00 --- iser.v00 --- ixgben.v00 --- lpfc.v00 --- lpnic.v00 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsi_msgp.v01 --- lsi_msgp.v02 --- mtip32xx.v00 --- ne1000.v00 --- nenic.v00 --- nfnic.v00 --- nhpsa.v00 --- nmlx4_co.v00 --- nmlx4_en.v00 --- nmlx4_rd.v00 --- nmlx5_co.v00 --- nmlx5_rd.v00 --- ntg3.v00 --- nvme_pci.v00 --- nvmerdma.v00 --- nvmxnet3.v00 --- nvmxnet3.v01 --- pvscsi.v00 --- qcnic.v00 --- qedentv.v00 --- qedrntv.v00 --- qfle3.v00 --- qfle3f.v00 --- qfle3i.v00 --- qflge.v00 --- rste.v00 --- sfvmk.v00 --- smartpqi.v00 --- vmkata.v00 --- vmkfcoe.v00 --- vmkusb.v00 --- vmw_ahci.v00 --- crx.v00 --- elx_esx_.v00 --- btldr.v00 --- esx_dvfi.v00 --- esx_ui.v00 --- esxupdt.v00 --- tpmesxup.v00 --- weaselin.v00 --- loadesx.v00 --- lsuv2_hp.v00 --- lsuv2_in.v00 --- lsuv2_ls.v00 --- lsuv2_nv.v00 --- lsuv2_oe.v00 --- lsuv2_oe.v01 --- lsuv2_oe.v02 --- lsuv2_sm.v00 --- native_m.v00 --- qlnative.v00 --- vdfs.v00 --- vmware_e.v00 --- vsan.v00 --- vsanheal.v00 --- vsanmgmt.v00 --- tools.t00 --- xorg.v00 --- gc.v00 --- imgdb.tgz --- imgpayld.tgz
Note: Make sure the extracted ESXi directory has the correct permissions. I simply used chmod 655 -R esxi70u1c/ to recursively apply the change for all files/directories.
Step 3 - Create new vHW19 Nested ESXi VM and configure the two required VM Advanced Settings which specifies the boot protocol (IPv4/IPv6) and the ESXi bootloader URL:
- networkBootProtocol = httpv4
- networkBootUri = http://192.168.30.6/esxi70uc1/efi/boot/bootx64.efi
and if everything was setup correctly, we should the ESXi image booting over the network from our HTTP server. In my example above, I did not specify kickstart location, so it will just boot into the interactive installer. To do so, just update the kernelopt line to reference the KS URL as described in Step 6 of the official VMware documentation.
After all these years working with ESXi kickstart, I never thought this could be simplified even further when running inside of a VM form factor, huge props to the VMware Engineers who challenged the status quo and provided an even better user experience when it comes to automating Nested ESXi deployments!