One really cool feature of Tanzu Kubernetes Grid (TKG) is the ability to bring your custom images (BYOI) which can then be used to deploy TKG Workload Clusters. To do so, customers will need to use Kubernetes (K8s) Image Builder tool to author new OVA images and then make TKG aware by updating the Tanzu Kubernetes Release (TKR) Build of Materials (BOM) configuration.
I had played around with Image Builder awhile back during the TKG 1.2 release and it definitely was not very easy to use. I have been meaning to kick the tires on Image Builder again as I know with the latest 1.3.x release, there have been a number of improvements. This week I saw an inquiry from my buddy Alan Renouf who was looking to see if there was a way to use the new Photon OS Real Time Kernel as a base image for a K8s-based application that he was working with that had requirements for the real time kernel.
Interestingly enough, there was another inquiry with a similiar customer request for their edge deployment and I thought this would be a good opportunity to try out Image Builder again, which has been overhauled and the build process can be completely consumed as a Docker container, which definitely made things much easier than before. I also had never played with real time version of Photon OS, so this gave me a reason to try that out which was initially introduced with Photon OS 4.0 but it also looks like real time kernel was added to 3.0 recently, which is the version I had used to test.
Note: vSphere with Tanzu currently does not support the ability to bring your own image like TKG, I know this is something that has been asked about and is being considered in the future.
The BYOI process for TKG is comprised of two steps:
- Create Custom TKG OVA
- Update TKG with new TKR BOM
Although there are detailed documentation for this process, I still ran into a number of issues which I think the documentation could be improved with a complete working example rather than using generic values which lead to some interpretation, which I did not interpret correctly the first time through. After posting some questions in the Image Builder Slack Channel, I was able to finally connect the dots with the help from Scott Rosenberg, who I also knew, as a customer of our VMware Event Broker Appliance (VEBA) Fling. Putting everything together, I figure it would be useful to document the process I took and hopefully this can benefit other customers looking to build and consume their own OVA images with TKG.