Earlier this week I received a question from a customer who was interested in creating a custom ESXi VIB that could execute a specific script within the payload as part of the installation and only running it once. This was a fairly common request that I have seen in the past and as far as I knew, this type of behavior was not possible. What was unique about this particular custom inquiry was that they mentioned they found some references on this being possible. Being the curious person I am, I decided to take another look and reach out to a few folks in Engineering.
After speaking with one of the Engineers familiar with our VIB specification, to my surprise I learned that this type of behavior was actually indeed possible but was not very well documented externally. The typical use case for this is usually to apply certain configurations which are not exposed through the traditional ESXi interfaces like the vSphere API/CLI which includes Host Profiles. One example that comes to mind is being able to deploy a VIB across several hundred ESXi hosts that could configure a specific ESXi Advanced Setting which might be marked hidden. Another example would be updating a configuration file like /etc/vmware/config or running a series of ESXi Shell commands which can not available in the vSphere API and require the use of SSH and the ESXi Shell. There are many other examples, but this should give you an idea of some of the common use cases that I have heard from customers.
Here is what is required to execute a specific script as part of the VIB installation:
- Created an "init" script which needs to be stored under /etc/init.d as part of your VIB payload (e.g. /etc/init.d/lamw-special-vib)
- You must ensure that the "live-install-allowed" param is set to true in your VIB's descriptor.xml file (this assumes the changes can be applied without requiring reboot to take affect)
- The init script will need to parse special keywords passed as command-line arguments
Here is a table showing the pair of special keywords that are passed to the init script as command-line arguments during a VIB install, upgrade or remove:
| Arg1 | Arg2 | VIB Operation |
|---|---|---|
| start | install | VIB install |
| start | upgrade | VIB upgrade |
| stop | remove | VIB removal |
It is up to creator of the init script to handle the different VIB operations by parsing the command-line arguments which would then determine the operations that would get executed within the script. This not only allows you to control the commands that are executed during an installation of a VIB but it also allows you specify the commands to run upon the removal of a VIB which is quite handy for properly cleaning up an uninstall. In addition, since these special keywords are not passed as part of the ESXi boot up process, the commands within the script will not execute and ensures it only runs once during the install.
Here is very simple shell script (you could also do this in Python as well) which demonstrates how to handle the three different types of VIB operations and then uses the "logger" utility to write some output to syslog:
#!/bin/sh
ARG1=$1
ARG2=$2
if [[ "${ARG1}" == "start" ]] && [[ "${ARG2}" == "install" ]]; then
# commands to run go here #
/bin/logger "William's custom script ran on start-install"
elif [[ "${ARG1}" == "start" ]] && [[ "${ARG2}" == "upgrade" ]]; then
/bin/logger "William's custom script ran on start-upgrade"
# commands to run go here #
elif [[ "${ARG1}" == "stop" ]] && [[ "${ARG2}" == "remove" ]]; then
/bin/logger "William's custom script ran on stop-remove"
# commands to run go here #
fi
As you can see, this allows you to perform a variety of tasks through the use of a custom ESXi VIB which is a great way to be able to roll out a set of changes that may not be possible using either the vSphere API or Host Profiles for example. Best of all, this solution does not require the use of SSH which is great since many customers already disable this by default.
For more information on creating a custom VIB, be sure to check out my blog post here and you can even use this Docker image I created for building custom ESXi VIBs.
Do you have to change your host acceptance level to use a homegrown vib?
Yes, custom VIBs require the acceptance level to be changed to "Community Supported". Take a look at the vibauthor Fling page for more details
Hi William,
but with a "Community Supported" VIB you cannot create a file in /etc/init.d. You need to use at least "PartnerSupported" as the acceptance level. That also means that you need to install the VIB with the --no-sig-check (or -f) flag when using esxcli, and you won't be able to install it via VUM at all.
At least you can build a customized ESXi installation image with such a VIB using PowerCLi Imagebuilder, so for running a script at ESXi installation time this will work fine.
For details see my post here: http://www.v-front.de/2012/11/update-esxi5-community-packaging-tools.html
Thanks
Andreas
Hey Andreas,
I'm not aware any such limitation with "CommunitySupported" VIB?
In fact, I was able to install this even without using -f as long as my acceptance level was set to CommunitySupported which is what's required when installing Custom VIBs built from vibauthor Fling
[[email protected]:~] esxcli software acceptance get
CommunitySupported
[[email protected]:~] esxcli software vib install -v /vmfs/volumes/vesxi60-7-local-storage/lamw.vib
Installation Result
Message: Operation finished successfully.
Reboot Required: false
VIBs Installed: virtuallyGhetto_bootbank_lamw_1.0.0-0.0.0
VIBs Removed:
VIBs Skipped:
You're right that this won't work with VUM and creating a custom ESXi Image would be the alternative to just deploying VIB using ESXCLI which many customers do today.
Interesting ... I stumbled over this limitation earlier (back with ESXi 5.x), and it is also documented (somehow) in the vibauthor.pdf (p.6).
Maybe this changed in ESXi 6.0. I will give it a try.
Yes, indeed: It works on ESXi 6.0, but with ESXi 5.5 you get this error when the VIB is "CommunitySupported":
~ # vmware -v
VMware ESXi 5.5.0 build-2702864
~ # esxcli software acceptance get
CommunitySupported
~ # esxcli software vib install -v /vmfs/volumes/FreeNAS01/incept1-1.0.0-1.x86_64.vib
[DependencyError]
VIB VFrontDe_bootbank_incept1_1.0.0-1 violates extensibility rule checks: [u'(line 21: col 0) Element vib failed to validate content']
Please refer to the log file for more details.
~ #
It's great to see that this limitation was lifted in ESXi 6.0 (hopefully intentionally ...)!
Yep and in this case, you just need to specify -f as you already know but there's no need to change the acceptance level and it does allow you to create files under /etc/init.d
I just tried this on a VMware ESXi 6 server - the VIB installs successfully during an Auto Deploy but the init script does not appear to get called with 'start install' as expected. I logged into the server via SSH and ran the script manually without error. Is there a different behavior during an Auto Deploy?
Hi dishmael3,
I had reached out to Engineering regarding the Auto Deploy question. Basically, when you Auto Deploy an ESXi, you are loading the Image Profile which contains the VIB and an installation is not performed as its already part of the Image Profile. In this case, only arg1 is passed which is "start" but "install" keyword is not specified. In the current sample script, this is why you're finding that its not running the script. There's a couple of options, if you know you're working with only Auto Deploy, you could just check for arg1 OR another suggestion from Engineering was to check "esxcli system boot device get" and see if Auto Deploy was being used and then handle case as needed so you could have a more generic script.
So is there a combination of commands that will work, so you can use custom VIB with vUM and ESXi 5.5?
VUM does not work directly with VIBs, but rather offline-bundles. You can create both VIBs and offline bundles using the VIB Author Fling. However, the answer to your question is no still because VUM does not support custom VIB/offline bundles. If you need to deploy custom VIBs, you'll have to use ESXCLI. This is something I've brought up internally and hopefully its something they'll fix in the future as I know its not ideal to be able to use VUM to distributed VIBs/offline bundles.
I just wanted to drop a quick thank you. I had been trying to find a solution to a non-impacting / non-distructive root password recovery of some old, legacy ESXi hosts I inherited that were luckily still managed by a vCenter I was able to gain access to. Your article here was a Eureka moment and your other articles on building VIBs as well as the docker image were helpful time savers.
If you have any interest in my password recovery solution I just posted it on GitLab today. https://gitlab.com/meyeaard/ESXi_5-6_root_Recovery
Many thanks for your ongoing contributions to the community!
Hi William,
Thanks for this very useful post.
I have created vib with payload etc/init.d/test-special-vib
I am using following acceptance level in the descriptor.xml file
community
also using same level on the ESXi host.
[[email protected]:~] esxcli software acceptance get
CommunitySupported
Getting following "Permission denied" error while installation.
What could be the reason of this error and how it can be fixed.
[[email protected]:~] esxcli software vib install -v /tmp/test.vib -f
[LiveInstallationError]
Error running command '['/etc/init.d/tes-special-vib', 'start', 'install']': [Errno 13] Permission denied
It is not safe to continue. Please reboot the host immediately to discard the unfinished update.
Please refer to the log file for more details.
Regards,
Vijay
This issue has been resolved.
There was some permission issue in the etc/init.d/test-special-vib while including into the payload.
After this permission issue fix, VIB installation is working but prints from script is not coming during installation/removal. What could be the issue ?
While manual execution of /etc/init.d/test-special-vib is working.
[1] How this script would be selected for execution during VIB installation ?
- Since it is placed in the init.d and part of the VIB being installed. Is it correct ?