Creating a custom VIB for ESXi 8.x

Back in 2012, a VMware Fling was released called VIB Author, which allowed users to create their own custom vSphere Infrastructure Bundles (VIB) that typically would include configuration changes that was not possible when using the vSphere API such as enabling custom ESXi firewall ports or even bundling up custom utilities that could run within the ESXi Shell.

The VIB Author tool was eventually deprecated and removed due to the lack of support from Engineering, after all, it was released as a Fling. While the need for opening non-standard ESXi firewall port has greatly improved over the years, with the majority of 2nd and 3rd party solutions simply incorporating that into their solution offering, there are still use cases for requiring a custom VIB.

Even with the VIB Author Fling being deprecated, many in the community was still able to construct custom VIBs which were still compatible with later ESXi 5.x to 7.x releases. In fact, I even use the VIB Author to make it easier to distribute and install the popular ghettoVCB solution which can be installed using either a VIB or an Offline Bundle, another format the VIB Author tool supports creating.

Prior to vSphere 8.0, the format of an ESXi VIB has mostly been unchanged from when it was possible for end users to create their own custom VIBs starting with the vSphere 5.0 release. In vSphere 8.0, a new field was introduced in the VIB specification that requires a SHA-256 checksum to be included in VIB descriptor file and since the VIB Author was not aware of this new field, any VIBs built using the VIB Author utility would fail to install.

Like most in the community, I had assumed it was no longer possible to create a custom VIB that would be compatible with ESXi 8.x and I simply accepted the outcome, especially with the VIB Author being deprecated more than 11 years ago. A few months back, I was made aware from my good friend Timo, that he had found someone who had figured out the SHA-256 requirement and came up with an alternative way of constructing an ESXi 8.x VIB.

Funny enough, the script was actually based on my original create_ghettoVCB_vib.sh script and the author had refactored the script to remove the use of VIB Author and replaced it with the ar utility which is something that Duncan Epping had written about back in 2011. I was able to incorporate the required changes back into my version of the create_ghettoVCB_vib.sh script which allowed me to continue to creating ghettoVCB VIBs and offline bundles (using VIB Author) that could then be used by anyone interested in using the ghettoVCB utility running on the latest ESXi 8.x releases.

Note: VMware does provide our eco-system partners an official way to build and package VIBs that integrate with ESXi through VMware's I/O Vendor Partner (IOVP) program.

While requests for creating custom VIBs have significantly reduced over the years, albeit due to various factors including the depreciation of the VIB Author Fling, I still see the asks from time to time. I was recently asked about this and I figured I would share this update in case anyone still has a use case for creating custom VIBs for ESXi 8.x. The most common use case that I have come across in the past couple of years still goes back to enabling non-standard ESXi ports.

To demonstrate the updated VIB creation process, I have extracted the high level details from my create_ghettoVCB_vib.sh shell script and provide a basic example of adding a utility to the /bin directory within an ESXi host. You can easily replace this with any random file for testing purposes to better understand the process.

Note: It is recommended that you use an Ubuntu VM which includes all the required utilities including ar. A macOS system can not be used as the stat command functions slightly differently and alternatively, you can also use a Docker Container, which is actually what I use for my ghettoVCB project. For simplicity purposes, an Ubuntu VM setup will suffice.

#!/bin/bash

set -euo pipefail

CUSTOM_VIB_TEMP_DIR=/tmp/vib-temp-$$
CUSTOM_VIB_NAME=ipmitool
CUSTOM_VIB_VERSION="1.8.18"
CUSTOM_VIB_VENDOR="williamlam.com"
CUSTOM_VIB_VENDOR_URL="https://williamlam.com"
CUSTOM_VIB_SUMMARY="Custom VIB summary text"
CUSTOM_VIB_DESCRIPTION="Custom VIB description text"
CUSTOM_VIB_BUILD_DATE=$(date '+%Y-%m-%dT%H:%I:%S')

# clean up any prior builds
CUSTOM_VIB_FILE_NAME=${CUSTOM_VIB_NAME}.vib
rm -f ${CUSTOM_VIB_FILE_NAME}

# Setting up VIB spec confs
VIB_DESC_FILE=${CUSTOM_VIB_TEMP_DIR}/descriptor.xml
VIB_PAYLOAD_DIR=${CUSTOM_VIB_TEMP_DIR}/payloads/payload1

# Create VIB temp & spec payload directory
mkdir -p ${CUSTOM_VIB_TEMP_DIR}
mkdir -p ${VIB_PAYLOAD_DIR}

# Create ESXi folder structure for file(s) placement
CUSTOM_VIB_BIN_DIR=${VIB_PAYLOAD_DIR}/bin
mkdir -p ${CUSTOM_VIB_BIN_DIR}

# Copy file(s) to destination folder
cp ipmitool ${CUSTOM_VIB_BIN_DIR}

# Create tgz with payload
tar czf ${CUSTOM_VIB_TEMP_DIR}/payload1 -C ${VIB_PAYLOAD_DIR} bin

# Calculate payload size/hash
PAYLOAD_FILES=$(tar tf ${CUSTOM_VIB_TEMP_DIR}/payload1 | grep -v -E '/$' | sed -e 's/^/    <file>/' -e 's/$/<\/file>/')
PAYLOAD_SIZE=$(stat -c %s ${CUSTOM_VIB_TEMP_DIR}/payload1)
PAYLOAD_SHA256=$(sha256sum ${CUSTOM_VIB_TEMP_DIR}/payload1 | awk '{print $1}')
PAYLOAD_SHA256_ZCAT=$(zcat ${CUSTOM_VIB_TEMP_DIR}/payload1 | sha256sum | awk '{print $1}')
PAYLOAD_SHA1_ZCAT=$(zcat ${CUSTOM_VIB_TEMP_DIR}/payload1 | sha1sum | awk '{print $1}')

# Create descriptor.xml
cat > ${VIB_DESC_FILE} << __VIB_DESC__
<vib version="5.0">
  <type>bootbank</type>
  <name>${CUSTOM_VIB_NAME}</name>
  <version>${CUSTOM_VIB_VERSION}</version>
  <vendor>${CUSTOM_VIB_VENDOR}</vendor>
  <summary>${CUSTOM_VIB_SUMMARY}</summary>
  <description>${CUSTOM_VIB_DESCRIPTION}</description>
  <release-date>${CUSTOM_VIB_BUILD_DATE}</release-date>
  <urls>
    <url key="website">${CUSTOM_VIB_VENDOR_URL}</url>
  </urls>
  <relationships>
    <depends>
    </depends>
    <conflicts/>
    <replaces/>
    <provides/>
    <compatibleWith/>
  </relationships>
  <software-tags>
  </software-tags>
  <system-requires>
    <maintenance-mode>false</maintenance-mode>
  </system-requires>
  <file-list>
${PAYLOAD_FILES}
  </file-list>
  <acceptance-level>community</acceptance-level>
  <live-install-allowed>true</live-install-allowed>
  <live-remove-allowed>true</live-remove-allowed>
  <cimom-restart>false</cimom-restart>
  <stateless-ready>true</stateless-ready>
  <overlay>false</overlay>
  <payloads>
    <payload name="payload1" type="tgz" size="${PAYLOAD_SIZE}">
        <checksum checksum-type="sha-256">${PAYLOAD_SHA256}</checksum>
        <checksum checksum-type="sha-256" verify-process="gunzip">${PAYLOAD_SHA256_ZCAT}</checksum>
        <checksum checksum-type="sha-1" verify-process="gunzip">${PAYLOAD_SHA1_ZCAT}</checksum>
    </payload>
  </payloads>
</vib>
__VIB_DESC__

# Create VIB using ar utility
touch ${CUSTOM_VIB_TEMP_DIR}/sig.pkcs7
ar r ${CUSTOM_VIB_FILE_NAME} ${VIB_DESC_FILE} ${CUSTOM_VIB_TEMP_DIR}/sig.pkcs7 ${CUSTOM_VIB_TEMP_DIR}/payload1

If there are any errors, the script will automatically stop processing but if everything works correctly, you should see a new file with .vib extension created after running the script as shown in the screenshot below.

For those interested, here is file structure for creating an updated custom VIB:

The last step to confirm that the custom VIB is functional is to install that onto an ESXi 8.x setup and in this example, I am using the latest ESXi 8.0 Update 1a release.