Back in 2012, a VMware Fling was released called VIB Author, which allowed users to create their own custom vSphere Infrastructure Bundles (VIB) that typically would include configuration changes that was not possible when using the vSphere API such as enabling custom ESXi firewall ports or even bundling up custom utilities that could run within the ESXi Shell.
The VIB Author tool was eventually deprecated and removed due to the lack of support from Engineering, after all, it was released as a Fling. While the need for opening non-standard ESXi firewall port has greatly improved over the years, with the majority of 2nd and 3rd party solutions simply incorporating that into their solution offering, there are still use cases for requiring a custom VIB.
Even with the VIB Author Fling being deprecated, many in the community was still able to construct custom VIBs which were still compatible with later ESXi 5.x to 7.x releases. In fact, I even use the VIB Author to make it easier to distribute and install the popular ghettoVCB solution which can be installed using either a VIB or an Offline Bundle, another format the VIB Author tool supports creating.
Prior to vSphere 8.0, the format of an ESXi VIB has mostly been unchanged from when it was possible for end users to create their own custom VIBs starting with the vSphere 5.0 release. In vSphere 8.0, a new field was introduced in the VIB specification that requires a SHA-256 checksum to be included in VIB descriptor file and since the VIB Author was not aware of this new field, any VIBs built using the VIB Author utility would fail to install.
Like most in the community, I had assumed it was no longer possible to create a custom VIB that would be compatible with ESXi 8.x and I simply accepted the outcome, especially with the VIB Author being deprecated more than 11 years ago. A few months back, I was made aware from my good friend Timo, that he had found someone who had figured out the SHA-256 requirement and came up with an alternative way of constructing an ESXi 8.x VIB.
Funny enough, the script was actually based on my original create_ghettoVCB_vib.sh script and the author had refactored the script to remove the use of VIB Author and replaced it with the ar utility which is something that Duncan Epping had written about back in 2011. I was able to incorporate the required changes back into my version of the create_ghettoVCB_vib.sh script which allowed me to continue to creating ghettoVCB VIBs and offline bundles (using VIB Author) that could then be used by anyone interested in using the ghettoVCB utility running on the latest ESXi 8.x releases.
Note: VMware does provide our eco-system partners an official way to build and package VIBs that integrate with ESXi through VMware's I/O Vendor Partner (IOVP) program.
While requests for creating custom VIBs have significantly reduced over the years, albeit due to various factors including the depreciation of the VIB Author Fling, I still see the asks from time to time. I was recently asked about this and I figured I would share this update in case anyone still has a use case for creating custom VIBs for ESXi 8.x. The most common use case that I have come across in the past couple of years still goes back to enabling non-standard ESXi ports.
To demonstrate the updated VIB creation process, I have extracted the high level details from my create_ghettoVCB_vib.sh shell script and provide a basic example of adding a utility to the /bin directory within an ESXi host. You can easily replace this with any random file for testing purposes to better understand the process.
Note: It is recommended that you use an Ubuntu VM which includes all the required utilities including ar. A macOS system can not be used as the stat command functions slightly differently and alternatively, you can also use a Docker Container, which is actually what I use for my ghettoVCB project. For simplicity purposes, an Ubuntu VM setup will suffice.
#!/bin/bash
set -euo pipefail
CUSTOM_VIB_TEMP_DIR=/tmp/vib-temp-$$
CUSTOM_VIB_NAME=ipmitool
CUSTOM_VIB_VERSION="1.8.18"
CUSTOM_VIB_VENDOR="williamlam.com"
CUSTOM_VIB_VENDOR_URL="https://williamlam.com"
CUSTOM_VIB_SUMMARY="Custom VIB summary text"
CUSTOM_VIB_DESCRIPTION="Custom VIB description text"
CUSTOM_VIB_BUILD_DATE=$(date '+%Y-%m-%dT%H:%I:%S')
# clean up any prior builds
CUSTOM_VIB_FILE_NAME=${CUSTOM_VIB_NAME}.vib
rm -f ${CUSTOM_VIB_FILE_NAME}
# Setting up VIB spec confs
VIB_DESC_FILE=${CUSTOM_VIB_TEMP_DIR}/descriptor.xml
VIB_PAYLOAD_DIR=${CUSTOM_VIB_TEMP_DIR}/payloads/payload1
# Create VIB temp & spec payload directory
mkdir -p ${CUSTOM_VIB_TEMP_DIR}
mkdir -p ${VIB_PAYLOAD_DIR}
# Create ESXi folder structure for file(s) placement
CUSTOM_VIB_BIN_DIR=${VIB_PAYLOAD_DIR}/bin
mkdir -p ${CUSTOM_VIB_BIN_DIR}
# Copy file(s) to destination folder
cp ipmitool ${CUSTOM_VIB_BIN_DIR}
# Create tgz with payload
tar czf ${CUSTOM_VIB_TEMP_DIR}/payload1 -C ${VIB_PAYLOAD_DIR} bin
# Calculate payload size/hash
PAYLOAD_FILES=$(tar tf ${CUSTOM_VIB_TEMP_DIR}/payload1 | grep -v -E '/$' | sed -e 's/^/ <file>/' -e 's/$/<\/file>/')
PAYLOAD_SIZE=$(stat -c %s ${CUSTOM_VIB_TEMP_DIR}/payload1)
PAYLOAD_SHA256=$(sha256sum ${CUSTOM_VIB_TEMP_DIR}/payload1 | awk '{print $1}')
PAYLOAD_SHA256_ZCAT=$(zcat ${CUSTOM_VIB_TEMP_DIR}/payload1 | sha256sum | awk '{print $1}')
PAYLOAD_SHA1_ZCAT=$(zcat ${CUSTOM_VIB_TEMP_DIR}/payload1 | sha1sum | awk '{print $1}')
# Create descriptor.xml
cat > ${VIB_DESC_FILE} << __VIB_DESC__
<vib version="5.0">
<type>bootbank</type>
<name>${CUSTOM_VIB_NAME}</name>
<version>${CUSTOM_VIB_VERSION}</version>
<vendor>${CUSTOM_VIB_VENDOR}</vendor>
<summary>${CUSTOM_VIB_SUMMARY}</summary>
<description>${CUSTOM_VIB_DESCRIPTION}</description>
<release-date>${CUSTOM_VIB_BUILD_DATE}</release-date>
<urls>
<url key="website">${CUSTOM_VIB_VENDOR_URL}</url>
</urls>
<relationships>
<depends>
</depends>
<conflicts/>
<replaces/>
<provides/>
<compatibleWith/>
</relationships>
<software-tags>
</software-tags>
<system-requires>
<maintenance-mode>false</maintenance-mode>
</system-requires>
<file-list>
${PAYLOAD_FILES}
</file-list>
<acceptance-level>community</acceptance-level>
<live-install-allowed>true</live-install-allowed>
<live-remove-allowed>true</live-remove-allowed>
<cimom-restart>false</cimom-restart>
<stateless-ready>true</stateless-ready>
<overlay>false</overlay>
<payloads>
<payload name="payload1" type="tgz" size="${PAYLOAD_SIZE}">
<checksum checksum-type="sha-256">${PAYLOAD_SHA256}</checksum>
<checksum checksum-type="sha-256" verify-process="gunzip">${PAYLOAD_SHA256_ZCAT}</checksum>
<checksum checksum-type="sha-1" verify-process="gunzip">${PAYLOAD_SHA1_ZCAT}</checksum>
</payload>
</payloads>
</vib>
__VIB_DESC__
# Create VIB using ar utility
touch ${CUSTOM_VIB_TEMP_DIR}/sig.pkcs7
ar r ${CUSTOM_VIB_FILE_NAME} ${VIB_DESC_FILE} ${CUSTOM_VIB_TEMP_DIR}/sig.pkcs7 ${CUSTOM_VIB_TEMP_DIR}/payload1
If there are any errors, the script will automatically stop processing but if everything works correctly, you should see a new file with .vib extension created after running the script as shown in the screenshot below.
For those interested, here is file structure for creating an updated custom VIB:
The last step to confirm that the custom VIB is functional is to install that onto an ESXi 8.x setup and in this example, I am using the latest ESXi 8.0 Update 1a release.
Hi William, this post came very handy for me as I was trying to deploy an application I'm building to run on esxi8.0.0
Is there any way to also build offline bundle for esxi 8.0.0
Yup. I didn't cover offline bundle as that's typically an advanced use case. However, once you've got a valid ESXi 8.x VIB, you'll need to use vibauthor tool to produce offline bundle. To do so, you can use the Docker Container which I've created and I use this for my ghettoVCB project, which you can see how its done at https://github.com/lamw/ghettoVCB/blob/master/build/create_ghettoVCB_vib.sh#L121
thanks for your reply. I have built a VIB package using the method you provided, which can be successfully installed on esxi7.0. However, the additional operator.log file cannot be written. And I already gave it write permission before packaging, if anything shoudle to be done in the descriptor.xml? how shoudle I do ?
bootbank
ipmitool
1.8.18
williamlam.com
Custom VIB summary text
Custom VIB description text
2023-11-02T16:04:31
https://williamlam.com
false
opt/MP/ipmitool
opt/MP/operator.log
partner
true
true
false
true
false
8862a126e5c47ca23b548a61fc88c8d368059947fe5e5eda695dbdd255429b5f
ad96ddfb461a9314d9c4593030d51e336614266c45afb72bd4b62e28fa13d618
6c92f59434f328d6dac427f29992c51447362ba1
How are you testing that it can't not be written? Can you manually open the file and add text/save?
Yes, I just to edit this file, it shows readonly. I can open and read it, but cannot overwrite it. the permissions of this file is 401, read and execute.
Dear william, i'm interested with this article , can you give me pointer to create custom nic driver like realtek for notebook? If i have the linux driver should this possible?
It’s not possible as you need access to SDK to build such a driver which you must be in our partner program to get access
Would it be possible to use this to open a firewall port?
Yes, that's one of the primary use cases for this. Please see https://williamlam.com/2012/09/creating-custom-vibs-for-esxi-50-51.html
If this is a custom port for 2nd or 3rd party solution, please have the vendor do this as part of their offering, so that you don't have to do it manually
Hi William,
Is it necessary to add binary data in the repository the script is pointing to? I just want the script to create me a VIB that will lead the ESXi to reboot (only that) for some big scale automating reasons (I know not the most efficient). Every time I run it, it runs no error, but comes out blank, no .vib. Thanks in advance!
Yea, we do same internally for testing 🙂
Hi, where to take "ipmitool" file from?
Hi William,
I'm trying to get it work too but my head doesn't work right now.
What should be filled in line 31: cp impitool ${CUSTOM_VIB_BIN_DIR} because I keep getting: "cp: cannot stat 'ipmitool': No such file or directory"
Hope you can get me out
Its looking for the ipmitool file in current working directory of the where you're running the script, you don't need to worry about the directories as that is handled for you 🙂
Hi WIlliam - Many thanks for your excellent script. I've created an ipmitool.vib from the 1.8.18 binary and installed it on my esxi 8.0 host.
I also ran the following to disable signature checks :-
esxcli system settings advanced set -o /User/execInstalledOnly -i 0
However, if I attempt to run /bin/ipmitool I get :-
-sh: ./ipmitool: Permission denied
I tried a chmod +x, however I also get a permission denied.
I cannot change permissions either with chmod. Permissions are
-r--r--r-- 1 root root 1.7M Apr 11 21:30 ipmitool
I'm logged into the host as root, but don't seem to have any permissions.
Any ideas?
Kind Regards
See https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-security/GUID-DF6A7974-62F9-47DB-A990-963F3B3AEA77.html
Thanks for the quick reply William! - Yeah, I presume you are referring to the following?, which I have run already.
esxcli system settings advanced set -o /User/execInstalledOnly -i 0
Same issue unfortunately.
It might be easier if you simply set the executable bit before you create your VIB, this way when its installed, its already got the right permissions
Hi Chris,
Did it work? I'm looking for ipmitool for exsi 8 to run on a nuc 9 xeon. If this works for you, can you share?
Hi Chris, This is the error I see,
[LiveInstallationError]
Error in running [/usr/lib/vmware/secureboot/bin/secureMount.py williamlam.com_bootbank_ipmitool_1.8.18 payload1 /tmp/stageliveimage/data/payload1.t00]:
Return code: 255
Output: The old parameter format is deprecated, please switch to the new format. See secureMount.py -h for more details.
Failed to validate metadata against schema for vib ipmitool: ('williamlam.com_bootbank_ipmitool_1.8.18', ['(line 24: col 0) Element vib failed to validate content'], "VIB (williamlam.com_bootbank_ipmitool_1.8.18) failed a check of extensibility rules for acceptance level 'community': ['(line 24: col 0) Element vib failed to validate content'].")
ERROR:root:Failed to mount: [Errno 1] Operation not permitted: '/tardisks/payload1.t00'
Traceback (most recent call last):
File "/lib64/python3.8/shutil.py", line 794, in move
FileExistsError: [Errno 17] File exists: '/tmp/stageliveimage/data/payload1.t00' -> '/tardisks/payload1.t00'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/vmware/secureboot/bin/secureMount.py", line 348, in legacyParsing
MountTardisk(True, FindVibInDB(sys.argv[1]), sys.argv[2], sys.argv[3])
File "/usr/lib/vmware/secureboot/bin/secureMount.py", line 157, in MountTardisk
shutil.move(tardiskPath, dest)
File "/lib64/python3.8/shutil.py", line 814, in move
File "/lib64/python3.8/shutil.py", line 435, in copy2
File "/lib64/python3.8/shutil.py", line 264, in copyfile
PermissionError: [Errno 1] Operation not permitted: '/tardisks/payload1.t00'
Hi William
it would be possible for you to rewrite "vghetto-not-april-fools-2020.vib" for config.xml file unkock on esxi 8?
I tried to install esxi 7 vib without success and I'm curious to test your solution on esxi8.