How to compile Busybox for ESXi ... kind of Part 1

I had initially researched this topic a few months back and decided not to write about it since it did not yield the results I was expecting. Some recent twitter conversations with Carter Shanklin and Didier Pironet this morning motivated me to dig up this content and share it with the community.

Before I get started, a word of caution:
!!!! THIS IS NOT SUPPORTED BY VMWARE - PLEASE USE AT YOUR OWN RISK !!!!

As you all know, ESXi does not have a Service Console like the classic ESX, but it does have a limited console based on Busybox, specifically v1.9.1 which is still the current version even with the latest release vSphere 4.1 Update 1. Busybox is heavlily customized by VMware and you will find a limited set of POSIX-like utilities, it is no where feature complete to your traditional POSIX environment and there is good reason for this.

If you have ever downloaded vSphere Hypervisor (formally known as ESXi), you may have noticed an "Open Source" tab at the top. This page contains some of the open source licensing information that VMware has used and incorporated into their products, most of these are just text files listing the various packages. One file that is of interest is the "OSS Source Code for ESX, ESXi and ESXi Embedded 4" which actually includes some of the open source RPMs and packages used to build Busybox for both classic ESX and ESXi (Yes, classic ESX also includes a busybox binary).

One interesting thing to note, is the date of this package, it has not been updated for almost two years. I am not sure if this is done intentionally, but I doubt that would be the case. It is also odd that VMware is still using an old release of Busybox (1.9.1), the latest release is at 1.18.3 and there have been quite a few updates between 1.9.1 and the current release. There has been speculation and rumors in the past that VMware choose a particular version of Busybox to get around the licensing of Busybox with ESXi. I do not know if this is true, but I do know, there have been quite a few bug fixes since incarnation of ESXi and you may hit some of those if you venture into Busybox console also known as Tech Support Mode (formally known as Unsupported Console)

The build environment that I used is running the latest version of CentOS 5.5 64bit which can be downloaded here.

Download VMware-esx-public-source-4.0-162945.tar.gz (565MB) from from VMware's website and SCP it to your build system

You will also need to install the following packages, you can do so using yum if you are using CentOS or RHEL and have a proper repository configured. You can use the following:

yum install -y gcc flex bison texinfo ncurses-devel libselinux-devel.x86_64 pam-devel.x86_64

Create a temporarily directory and move the tar.gz file into the directory and then extract the contents using the following command:

tar -zxvf VMware-esx-public-source-4.0-162945.tar.gz

You now should have a bunch of *.rpm files that have been extracted. There are two interesting RPM packages that we are interested that have "busybox" in the name, one of classic ESX (vmware-esx-busybox-1.9.1-1.624.vmw.src.rpm) and one for ESXi (vmware-visor-busybox-1.9.1-1.654.vmw.src.rpm)

You will need to create the following directory before installing the ESXi RPM package. Use the following command:

mkdir -p /usr/src/redhat/SOURCES

Next, you will install the vmware-visor-busybox-1.9.1-1.654.vmw.src.rpm by using the following command:

rpm -ivh vmware-visor-busybox-1.9.1-1.654.vmw.src.rpm

Note: Don't worry about the warnings regarding the user/groups that do not exists, these are most likely a default from VMware's environment that expects the mts user and group.

Now you will change into the /usr/src/redhat/SOURCES directory and you should see three files:

• busybox-1.9.1.tar.bz2 (Busybox v1.9.1)
• config-vmware-visor (Busybox config file from VMware)
• vmware.patch (VMware patch for building Busybox)

You will now extract the contents of the Busybox archive by using the following command:

tar -xvjpf busybox-1.9.1.tar.bz2

Next you will copy the vmware.patch into the busybox-1.9.1 directory by using the "cp" command and then change busybox-1.9.1 directory. If you do a "ls" you should see the following files:

We are now going to apply the patch provided by VMware which will update some of the C source code found in default Busybox 1.9.1 along with setting up a build environment. We go through a dryrun process to see what files will be patched, use the following command:

patch --dry-run -p1 < vmware.patch

To apply the patch, you will run the following command:

patch -p1 < vmware.patch

After the patch has been applied, there are two interesting files that will be created in addition to the source code updates:

• README.vmware (Pretty well documented set of changes + how to build Busybox)
• vmware-env.sh (environment script that is sourced to build Busybox)

Now we are ready to build Busbox, the exact instructions are specified towards the end of the README.vmware file

Basically you will be executing the following commands:

VMWARE_TARGET=visor
source vmware-env.sh
cp ../config-vmware-visor .config
make oldconfig
busybox-build

Now if you executed the above, you will find that you quickly run into some build errors.

This is where the "kind of" part of the blog post comes in. If you recall, we sourced the vmware-env.sh script just prior to building, this is a script that VMware wrote to quickly setup the build environment for compiling Busybox for either ESX or ESXi.

For this particular build and release, they are using specific versions of GCC, Binutils and PAM. You might ask why not substitute these with the versions found on system? I have actually gone through that, but ran into other compliation issues which maybe due to fact that GCC and other components are much newer and not compatible with older versions of Busybox. I have also tried to match the build environment by using buildroot but continue to hit other compilation issues. The only way I can see building this particular patched version of Busybox is to get a build environment that matches what VMware uses.

In Part2, I will go into details on building your own Busybox binary for ESXi which I promise will be much more fruitful than this. Stay tuned ...

Update: It looks like there is a slightly more recent version of ESX(i) OSS released on 07/13/2010, for whatever reason, it was filed under VMware View section. The source RPM for ESXi still looks like the same one released back in 2009, but there some other packages that differ from the one found on vSphere Hypervisor page.