WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / Automation / Nested ESXi installation using HTTPS boot over VirtualEFI in vSphere 8

Nested ESXi installation using HTTPS boot over VirtualEFI in vSphere 8

01.13.2023 by William Lam // Leave a Comment

In vSphere 7.0 Update 2, an enhancement was made to the Virtual Machine's UEFI firmware called VirtualEFI that would enable ESXi to run in a VM (Nested ESXi) and perform an HTTP Boot given the ESXi bootloader URL without requiring any traditional PXE infrastructure.

This was especially useful for anyone testing or developing ESXi automation for use with ESXi Kickstart, where you can quickly prototype your automation without additional infrastructure dependencies and once the automation has been vetted, you can then leverage that exact same automation in your physical ESXi provisioning infrastructure.

The original solution had only supported HTTP and I recently came to learn that we can now also support HTTPS in vSphere 8!

To take advantage of this new capability, you just need to have a physical server running ESXi 8.0 or Workstation 17 and a VM that is configured with the latest vHW20 compatibility.

To configure HTTPS boot, you will need to add the following four VM Advanced Settings:

  • networkBootProtocol - httpv4 or httpv6
  • networkBootUri - HTTPS URL to the ESXi bootloader (bootx64.efi)
  • uefi.tls.CaCertificateDefault.present - TRUE
  • uefi.tls.CaCertificateDefault.file0 - DER encoded file of the HTTPS server TLS certificate that is accessible to the VM on the ESXi host

Disclaimer: Nested ESXi and Nested Virtualization is not officially supported by VMware

Step 1 - Download and extract the desired ESXi ISO to your HTTP server. In my example, I am using ESXi 8.0a and have extracted the contents into a directory called esxi80a and placed that into the root of my HTTPS server, which in my setup is /etc/httpd/html

Step 2 - You will need to update the ESXi boot.cfg configuration which is located in efi/boot directory with the following:

  • Remove the leading "/" character in front of all module names
  • Update the prefix parameter with the base URL of the extracted ESXi image

Here is what my /etc/httpd/html/esxi80a/efi/boot/boot.cfg looks like after applying the two changes mentioned above:

bootstate=0
title=Loading ESXi installer
timeout=5
prefix=https://web.lab.local/esxi80a
kernel=b.b00
kernelopt=runweasel cdromBoot
modules=jumpstrt.gz --- useropts.gz --- features.gz --- k.b00 --- uc_intel.b00 --- uc_amd.b00 --- uc_hygon.b00 --- procfs.b00 --- vmx.v00 --- vim.v00 --- tpm.v00 --- sb.v00 --- s.v00 --- atlantic.v00 --- bcm_mpi3.v00 --- bnxtnet.v00 --- bnxtroce.v00 --- brcmfcoe.v00 --- cndi_igc.v00 --- dwi2c.v00 --- elxiscsi.v00 --- elxnet.v00 --- i40en.v00 --- iavmd.v00 --- icen.v00 --- igbn.v00 --- ionic_en.v00 --- irdman.v00 --- iser.v00 --- ixgben.v00 --- lpfc.v00 --- lpnic.v00 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsi_msgp.v01 --- lsi_msgp.v02 --- mtip32xx.v00 --- ne1000.v00 --- nenic.v00 --- nfnic.v00 --- nhpsa.v00 --- nmlx5_co.v00 --- nmlx5_rd.v00 --- ntg3.v00 --- nvme_pci.v00 --- nvmerdma.v00 --- nvmetcp.v00 --- nvmxnet3.v00 --- nvmxnet3.v01 --- pvscsi.v00 --- qcnic.v00 --- qedentv.v00 --- qedrntv.v00 --- qfle3.v00 --- qfle3f.v00 --- qfle3i.v00 --- qflge.v00 --- rdmahl.v00 --- rste.v00 --- sfvmk.v00 --- smartpqi.v00 --- vmkata.v00 --- vmksdhci.v00 --- vmkusb.v00 --- vmw_ahci.v00 --- bmcal.v00 --- clusters.v00 --- crx.v00 --- drivervm.v00 --- elx_esx_.v00 --- btldr.v00 --- esx_dvfi.v00 --- esx_ui.v00 --- esxupdt.v00 --- tpmesxup.v00 --- weaselin.v00 --- esxio_co.v00 --- loadesx.v00 --- lsuv2_hp.v00 --- lsuv2_in.v00 --- lsuv2_ls.v00 --- lsuv2_nv.v00 --- lsuv2_oe.v00 --- lsuv2_oe.v01 --- lsuv2_sm.v00 --- native_m.v00 --- qlnative.v00 --- trx.v00 --- vdfs.v00 --- vmware_e.v00 --- vsan.v00 --- vsanheal.v00 --- vsanmgmt.v00 --- tools.t00 --- xorg.v00 --- gc.v00 --- imgdb.tgz --- basemisc.tgz --- resvibs.tgz --- esxiodpt.tgz --- imgpayld.tgz
build=8.0.0-1.10.20842819
updated=0

Note: Make sure the extracted ESXi directory has the correct permissions. I simply used chmod 655 -R esxi80a/ to recursively apply the change for all files/directories.

Step 3 - Next, we need to get a copy of the SSL certificate of the HTTPS server (not the private key) but just the certificate file and encode that to DER format. Here is an example if you were installing HTTPS server from scratch, you would need to generate a TLS certificate using something like the following:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/apache-selfsigned.key -out /etc/ssl/certs/apache-selfsigned.crt

The apache-selfsigned.crt is the file we are interested in. To convert from the default PEM format to DER, you can run the following command:

openssl x509 -in /etc/ssl/certs/apache-selfsigned.crt -outform der -out apache-selfsigned.der

Once you have a copy of the TLS certificate in the DER format, you will need to transfer that to ESXi host so that the VM has access and can validate when it attempts to boot from the HTTPS server. You can place the file within the VM directory but if you intend to use this for more than a single VM, then I would recommend creating a new directory on one of your ESXi datastores and place the TLS certificate there which can then be referenced by multiple VMs.

In my example, I am placing it on a vSAN Datastore under a directory that I created called uefi-boot-tls-cert and the full path is the following: /vmfs/volumes/sm-vsanDatastore/uefi-boot-tls-cert/apache-selfsigned.der

Step 4 - Create new vHW20 Nested ESXi VM and configure the four required VM Advanced Settings containing the values from your environment:

  • networkBootProtocol = httpv4
  • networkBootUri = https://web.lab.local/esxi80a/efi/boot/bootx64.efi
  • uefi.tls.CaCertificateDefault.present = "TRUE"
  • uefi.tls.CaCertificateDefault.file0 = "/vmfs/volumes/sm-vsanDatastore/uefi-boot-tls-cert/apache-selfsigned.der"

Step 5 - Finally, power on the VM and you should see the VM attempt EFI Network boot over HTTPS and you should see the ESXi installer start to boot as shown in the screenshot below.


I did not specify kickstart location, so this will simply boot into the ESXi interactive installer. To do so, just update the kernelopt line to reference the KS URL as described in Step 6 of the official VMware documentation.

More from my site

  • ACPI motherboard layout requires EFI - Considerations for switching VM firmware in vSphere 8 
  • Using vSphere Lifecycle Manager (vLCM) to remediate Nested ESXi host with CPU on the host is not supported 
  • Simplified Nested ESXi installation in ESXi 7.0 Update 2 using HTTP Boot over VirtualEFI
  • USB Network Native Driver Fling for ESXi 8.0 Update 1
  • How to bootstrap vSAN Express Storage Architecture (ESA) on unsupported hardware?

Categories // Automation, ESXi, Nested Virtualization, vSphere 8.0 Tags // efi, ESXi 8.0, Nested ESXi, UEFI, vSphere 8.0

Thanks for the comment!Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC) across Private, Hybrid and Public Cloud

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • Disabling vCenter Lifecycle Manager automatic download using vSphere API 10/02/2023
  • ESXi on Lenovo ThinkStation P3 Ultra 09/29/2023
  • Quick Tip - vSphere 7.0 Update 3o also supports disabling/enabling vSphere Cluster Services (vCLS) in vSphere UI 09/29/2023
  • Heads Up - New image identifier required by VM Service in vSphere 8.0 Update 2 09/27/2023
  • How to setup private GitLab on a Synology for Project Keswick? 09/26/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...