Updated method for automatically joining vCenter Server using ESXi Kickstart

Have you ever written a script that gets the job done and years later, you go back and ask yourself ... what the heck did I do or why did I do it that way!?! 😅

Probably not as frequent as we should but I have found myself in these situations over the years, whether it is addressing an issue/enhancement or needing to explain the code to another colleague.

While copious free time is never on our side in IT, it does help to re-evaluate your automation and processes over the years to see if there are new techniques, enhancements that can make the automation more effective.

I was recently CC'ed on a Reddit thread about some automation that I had worked on more than 14 years ago that would automatically join an ESXi host to vCenter Server using an ESXi Kickstart script.

Knowing what I know now and probably even earlier, the solution of calling into the vSphere MOB and manually crafting the XML payload to join the ESXi host to a desired vSphere Cluster in vCenter Server was certainly not ideal, but it got the job done. We can certainly improve the solution leveraging a more modern solution, while still staying true to the original requirements of being able to perform vSphere Cluster addition directly from ESXi 8.x/9.x Kickstart without relying on external dependencies.

Step 1 - I will assume you already have an ESXi Kickstart infrastructure up and running.

Note: For development purposes, I highly recommend leveraging Nested ESXi and HTTP Boot over Virtual EFI, so you can quickly test your ESXi Kickstart automation before rolling that out to your physical infrastructure. This allowed me to quickly prototype the final solution that you see here without much effort.

Step 2 - With the help from ChatGPT, I was able to create a modern version of the script leveraging Pyvmomi (vSphere SDK for Python) that would use the exact same vSphere API to add the ESXi host to vCenter Server and attach it that to the desired vSphere Cluster. Download the add_host_to_cluster.py script and place that on your web server where you are serving your ESXi Kickstart configuration file or you can also embed that directly into your kickstart file.

The script can be run locally on a system that has Pyvmomi installed or it can run directly on an ESXi 8.x / 9.x system with the following syntax:

python add_host_to_cluster.py \
     --vcenter vc03.williamlam.local --vc-user '*protected email*' --vc-pass 'VMware1!' \
     --datacenter 'Datacenter' --cluster 'Cluster-01' \
     --host-user 'root' --host-pass 'VMware1!' --insecure --vmk vmk0

Step 3 - Update your ESXi Kickstart to run the add_host_to_cluster.py script with your desired input. In my example, I have hosted both the ESXi kickstart and add_host_to_cluster.py script on my web server, so my kickstart would first download the script to ESXi host as part of the %firstboot and then run the script with the credentials to my vCenter Server along with the credentials of the ESXi host which is required as part of calling the vSphere API.

Note: You probably should NOT be using the administrator account to add your ESXi host but create a service account that only has the privilege of adding hosts to a vSphere Cluster and no other roles, since the credentials would need to be provided in plain text.

vmaccepteula
rootpw VMware1!
install --firstdisk --overwritevmfs
network --bootproto=static --device=vmnic0 --ip=192.168.30.61 --netmask=255.255.255.0 --gateway=192.168.30.1 --hostname=esx01.williamlam.local --nameserver=192.168.30.29 --addvmportgroup=1
reboot

%firstboot --interpreter=busybox

# Ensure hostd is ready
while ! vim-cmd hostsvc/runtimeinfo; do
sleep 10
done

# Configure NTP
esxcli system ntp set -e true -s 10.0.0.221

# Enable HTTP Outbound
esxcli network firewall ruleset set -e true -r httpClient

# Add DNS Search Domain
esxcli network ip dns search add -d williamlam.local

# Download vSphere Cluster Host Add script
wget http://192.168.30.29/add_host_to_cluster.py -O /tmp/add_host_to_cluster.py

# Run Add Host script
python /tmp/add_host_to_cluster.py \
     --vcenter vc03.williamlam.local --vc-user '*protected email*' --vc-pass 'VMware1!' \
     --datacenter 'Datacenter' --cluster 'Cluster-01' \
     --host-user 'root' --host-pass 'VMware1!' --insecure --vmk vmk0

Here is a screenshot of the final solution, where I was able to quickly demonstrate this by leveraging a Nested ESXi VM and HTTP boot over Virtual EFI (see reference in Step 1) that would automatically install ESXi, just like you would on a bare-metal system and then as part of the post-customization, it will download our Pyvmomi script and then join the desired vSphere Cluster!