WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / Automation / Configuring dnsmasq as PXE Server for ESXi 

Configuring dnsmasq as PXE Server for ESXi 

07.09.2020 by William Lam // 2 Comments

One really cool thing that I came to learn while setting up the infrastructure to network boot the latest Raspberry Pi 4 was the use of dnsmasq, which I have used in the past but I did not realize it could do so much more. In addition to providing DNS services, it can also be configured to run TFTP and provide DHCP capabilities which can then be used to support PXE installations.

Another neat feature of dnsmasq is ability to proxy to an existing DHCP server which is extremely useful for anyone with an existing DHCP infrastructure. Given the simplicity of dnsmasq and having already set this up for the rPI, I figure it would also be useful to take folks through in setting up dnsmasq to also support ESXi installations over PXE, since this still comes up from new folks just getting started with ESXi kickstart automation.

For more details about PXE installation of ESXi, I highly recommend this whitepaper and although it states 6.0, the concepts and configurations are still applicable to the latest ESXi 7.0 release.

Step 1 - Install an Ubuntu VM, I happen to have 16.04 lying around in my vSphere Datastore but you can use any recent version

Step 2 - Apply latest patches and install the following packages:

apt update
apt install -y nfs-kernel-server dnsmasq kpartx unzip

Step 3 - Download the desired ESXi ISO and upload that to root of the Ubuntu VM. In my example, I am using the VMware-VMvisor-Installer-7.0.0-15843807.x86_64.iso.

Step 4 - Mount the ESXi ISO and copy the installation files to our TFTP root directory. In my example, I use the name esxi7 but you can use any name you wish and if you decided to change it, make sure all steps after this also matches the name.

mount -o loop VMware-VMvisor-Installer-7.0.0-15843807.x86_64.iso /mnt
cp -rf /mnt /srv/tftpboot/esxi7/
umount /mnt

Step 5 - We need to remove the trailing "/" from both boot.cfg as the installation files will be served from the current path.

sed -i 's#/##g' /srv/tftpboot/esxi7/boot.cfg
sed -i 's#/##g' /srv/tftpboot/esxi7/efi/boot/boot.cfg

Step 6 - There are a number of ways you can serve the kickstart configuration file (ks.cfg) to the ESXi host that is being PXE booted. In this example, I will be using NFS as I already had that setup from the Raspberry Pi 4 article. You can certainly adjust this to use other supported options.

mkdir -p /srv/nfs/esxi7
echo "/srv/nfs/esxi7 *(rw,sync,no_subtree_check,no_root_squash)" >> /etc/exports

Below is a very basic ks.cfg that you can use testing purposes and save this to /srv/nfs/esxi7/ks.cfg

vmaccepteula
install --firstdisk --overwritevmfs --novmfsondisk
reboot

rootpw VMware1!
network --bootproto=dhcp --device=vmnic0

%firstboot --interpreter=busybox

# enable & start SSH
vim-cmd hostsvc/enable_ssh
vim-cmd hostsvc/start_ssh

# enable & start ESXi Shell
vim-cmd hostsvc/enable_esx_shell
vim-cmd hostsvc/start_esx_shell

For the ESXi installer to locate our ks.cfg, we need to update the respective boot.cfg with kernelopt=ks=nfs://<kickstart-server>/srv/nfs/esxi7/ks.cfg and this will be covered in the next section depending if you are going to be using EFI or BIOS firmware.

Step 7 - Create the dnsmasq configuration /etc/dnsmasq.conf which can serve BIOS and UEFI systems. If you intend to support both, please follow BOTH step 8a and 8b section. If not using one of them, please comment it out as dnsmasq will throw an error if those files are not found.

There are a number of dnsmasq configurations that you can tweak, but if you have an existing DHCP server and simply want to use it, simply specify your network broadcast address and "proxy" keyword as shown in the example below.

dhcp-range=192.168.30.255,proxy
log-dhcp
enable-tftp
tftp-root=/srv/tftpboot

pxe-prompt="Booting PXE Client", 1

# ESXi x86 BIOS
dhcp-boot=esxi7/pxelinux
pxe-service=X86PC, "Boot ESXi x86 BIOS Installer", esxi7/pxelinux

# ESXi x86 EFI
dhcp-match=set:efi64,60,PXEClient:Arch:00007
dhcp-boot=tag:efi64,esxi7/efi/boot/bootx64.efi

# ;)

Step 8 -

For BIOS:

Step 8ai - For legacy BIOS, you must use syslinux 3.86 and the following commands will download pxelinux.0

wget https://mirrors.edge.kernel.org/pub/linux/utils/boot/syslinux/3.xx/syslinux-3.86.zip
unzip syslinux-3.86.zip -d syslinux
rm syslinux-3.86.zip
cp syslinux/core/pxelinux.0 /srv/tftpboot/esxi7/

Step 8aii - Next, we create the expected pxelinux directory structure and simply use the "default" file for testing purposes. In general, you will want to create a file with the ESXi physical MAC Address (e.g. 00-11-22-33-44-55-66) so that you can control which hosts are PXE booted and the version of ESXi they will receive.

mkdir -p /srv/tftpboot/esxi7/pxelinux.cfg
cp /srv/tftpboot/esxi7/isolinux.cfg /srv/tftpboot/esxi7/pxelinux.cfg/default

Step 8aiii - Update the boot.cfg to specify how to download our ks.cfg and replace the IP Address with address of your Kickstart system

sed -i 's#kernelopt=.*#kernelopt=ks=nfs://192.168.30.176/srv/nfs/esxi7/ks.cfg#g' /srv/tftpboot/esxi7/boot.cfg

For UEFI:

Step 8bi - Update the prefix to point to our root esxi7 directory or else it will assume the installation files will be served from efi/boot

sed -i 's/prefix.*/prefix=esxi7/g' /srv/tftpboot/esxi7/efi/boot/boot.cfg

Step 8bii - Update the boot.cfg to specify how to download our ks.cfg and replace the IP Address with address of your Kickstart system

sed -i 's#kernelopt=.*#kernelopt=ks=nfs://192.168.30.176/srv/nfs/esxi7/ks.cfg#g' /srv/tftpboot/esxi7/efi/boot/boot.cfg

Step 9 - Enable and start the services

systemctl enable dnsmasq
systemctl enable rpcbind
systemctl enable nfs-server
systemctl start dnsmasq
systemctl start rpcbind
systemctl start nfs-serverq

Step 10 - Create either a Nested ESXi VM (BIOS or EFI) or use a physical host and power on the system. If everything was configured correctly, you should see something like the following and after the ESXi installation files are loaded, it should mount the NFS volume to retrieve the ks.cfg and start the unattended installation and finally reboot.

More from my site

  • Quick Tip - Using ESXi Scripted Installation (kickstart) to configure IPv6 networking
  • Automated network scripted installation of ESXi-Arm without SD Card
  • How to prevent physical CD-ROM from ejecting after installing or upgrading ESXi?
  • Automated ESXi Installation to USB using Kickstart
  • Revisiting prompting for user input during an interactive or scripted install of ESXi

Categories // Automation, ESXi, vSphere 7.0 Tags // dnsmasq, esxi, kickstart, pxe boot

Comments

  1. Hariven says

    08/10/2021 at 8:04 pm

    Thanks for the article William - I think I followed the information provided for UEFI esxi 7 install but I get this error message -

    Loading operating system

    Loading /boot.cfg
    Configuration error while parsing /boot.cfg
    Fatal error: 24 (TFTP error)

    Appreciate ifs you can let me know where to look.

    Thanks in Advance

    Reply
    • HelarNL says

      10/02/2021 at 5:59 am

      Same here. A symlink in /srv/tftproot works for me:
      ln -s esxi7/efi/boot/boot.cfg boot.cfg

      Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • How to bootstrap ESXi compute only node and connect to vSAN HCI Mesh? 01/31/2023
  • Quick Tip - Easily move or copy VMs between two Free ESXi hosts? 01/30/2023
  • vSphere with Tanzu using Intel Arc GPU 01/26/2023
  • Quick Tip - Automating allowed and not allowed Datastores for use with vSphere Cluster Services (vCLS) 01/25/2023
  • ESXi with Intel Arc 750 / 770 GPU 01/24/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023

 

Loading Comments...