WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Hardware Options
    • Hardware Reviews
    • Lab Deployment Scripts
    • Nested Virtualization
    • Homelab Podcasts
  • VMware Nostalgia
  • Apple

Cluster API BYOH Provider on Photon OS (Arm) with Tanzu Community Edition (TCE) and ESXi-Arm

11.22.2021 by William Lam // Leave a Comment

Last week I demonstrated how to take advantage of the new Kubernetes Cluster API Bring Your Own Host (BYOH) Provider with a VM running on ESXi-Arm and managed with Tanzu Community Edition (TCE). The Cluster API BYOH Provider is currently only tested and supported with an Ubuntu OS, but since the only requirements for a linux host was simply: kubeadm, kubelet and containerd, I figured it should also be possible with VMware's Photon OS which also has an Arm edition.

With a TON of trial/error and reverting snapshots, I was able to finally get Cluster API BYOH Provider to successful run on Photon OS as shared in a recent tweet.

πŸ‘Š
🎀

πŸ”₯ Uber Hybrid TCE Workload Cluster πŸ”₯

βœ… ESXi-Arm
βœ… ESXi-x86
βœ… Ubuntu Arm
βœ… Photon Arm
βœ… Ubuntu x86
❔ Photon x86 (should work but I'm lazy now haha) pic.twitter.com/dkPXSl4vLB

— William Lam (@lamw.bsky.social | @*protected email*) (@lamw) November 21, 2021

What actually made this possible was actually the work I had done with VMware Event Broker Appliance (VEBA) project which also involves Photon OS and Kubernetes. More specifically, I had recently worked on porting VEBA from using the Docker runtime to Containerd with Kubernetes and that prior experience was invaluable while figuring out how to do this with Photon OS (Arm) which also had its own challenges. The instructions below will help setup a Photon OS (Arm) VM that can then be used with Cluster API BOYH Provider and the previous article will still need to be reference for the complete setup.

[Read more...]

Categories // ESXi-Arm, Kubernetes, VMware Tanzu Tags // Arm, ESXi, Photon, Raspberry Pi, Tanzu Community Edition, Tanzu Kubernetes Grid, TKG

Hybrid (x86 and Arm) Kubernetes clusters using Tanzu Community Edition (TCE) and ESXi-Arm

11.19.2021 by William Lam // Leave a Comment

With the recent introduction of Tanzu Community Edition (TCE), users can now easily get first hand experience across VMware's Tanzu portfolio, including VMware's Enterprise Kubernetes (K8s) runtime called Tanzu Kubernetes Grid (TKG), all completely for free. One popular request that frequently comes up from our community is the ability to use TCE with the ESXi-Arm Fling.

Currently, TCE is only supported with x86 hardware platforms which includes ESXi-x86 and there is certainly a desire to be able to use TCE with Arm-based hardware running on top of ESXi-Arm, especially with inexpensive Raspberry Pi for learning and exploration purposes.

I recently came to learn about a really cool project that is being developed as part of VMware's Office of the CTO (OCTO) for a new Cluster API (CAPI) provider where you can Bring Your own Host (BYOH) that is already running Linux. What really intrigued me about their project was not the fact that they could create a TCE Workload Cluster that comprised of physical hosts but the fact that they were actually running on Arm hardware! 🀩

My immediate reaction was to see if this would also work with just Linux VMs? With some trial/error and help from Jixing Jia, one of the project maintainers, I was able to confirm that this indeed does works using Ubuntu VMs running on ESXi-Arm. What was even more impressive was the realization that this not only works for both physical and virtual Arm Linux systems, but that users could also create a hybrid TCE Workload Cluster that consists of BOTH x86 and Arm nodes! 🀯

I can only imagine the possibilities that this could enable in the future where application(s) could potentially span across CPU architecture, virtual and physical worker nodes which exposes different capabilities that can then be delivered based on the requirements of the application such as GPU as an example. It will be interesting to see the types of use cases the BYOH Cluster API Provider will help enable, especially pertaining to Edge computing.

If you are interested in playing with the BYOH Cluster API Provider, check out the detailed instructions below on how to get started. Since this is still currently in Alpha development, there are still a few manual steps and currently there is no native TCE integration. If this is something that is interesting to you, feel free to leave any feedback or better yet, leave comments directly on the Github repo asking for feature enhancements that you would like to see such as native support for TCE πŸ˜€

[Read more...]

Categories // ESXi-Arm, Kubernetes, VMware Tanzu Tags // Arm, ESXi, Raspberry Pi, Tanzu Community Edition, Tanzu Kubernetes Grid, TKG

VEBA + Knative + k3s on ESXi-Arm

01.26.2021 by William Lam // Leave a Comment

In response to a customer request to add Arm64 support for our VMware Event Router, I have been spending some more time playing with k3s (lightweight Kubernetes distribution for Arm) running on ESXi-Arm using a Raspberry Pi. Not only was this a good learning experience that exposed to me to the broader Arm ecosystem, which is still maturing but it also took me down several πŸ°πŸ•³οΈ which got me exploring new tools that I had never used before such as Buildpacks and Docker buildx to name a few.

This past weekend, I was finally successful in setting up our VMware Event Router for Arm using the Knative processor on a k3s cluster using ESXi-Arm running on a Raspberry Pi 4b 8GB model! As of writing this, the following versions were used:

  • Knative Serving v0.20.0
  • Knative Net Contour v0.20.0
  • Knative Eventing v0.20.1
  • RabbitMQ Cluster Operator v0.5.0

Made some more progress w/@KnativeProject + @VMWEventBroker on k3s on @esxi_arm

βœ… Knative Serving & Eventing
βœ… @RabbitMQ Operator & Eventing
βœ… @projectcontour
βœ… @VMware Event Router

Just need to figure out @buildpacks_io for Arm64 - https://t.co/ChdkMLSXMp looks promising pic.twitter.com/XFWDiGONSB

— William Lam (@lamw.bsky.social | @*protected email*) (@lamw) January 24, 2021

In addition, I was able to also convert the Knative python echo function that was originally created by my colleague Michael Gasch and build an Arm64 version of the Knative python echo function which demonstrates the integration of VEBA with the Knative processor connected to a vCenter Server as my event source.

πŸ₯³ Successfully deployed & verified my arm64 python echo func w/@VMWEventBroker (Event Router) using the @KnativeProject processor!

Awesome for lightweight testing/development purposes on small VM w/k3s on @esxi_arm

Heck, don’t even need real vCenter, can run vcsim locally! pic.twitter.com/DuI16fvXfs

— William Lam (@lamw.bsky.social | @*protected email*) (@lamw) January 24, 2021

For those interested in just the VMware Event Router Arm64 image, you can access it here and we plan to make that an official image shortly. For those interested in setting up a fully functional Arm deployment of VEBA and Knative processor, you can find the detailed instructions below.

[Read more...]

Categories // Automation, ESXi-Arm, Kubernetes Tags // Arm, k3s, Knative, Kubernetes, Raspberry Pi, VEBA

  • 1
  • 2
  • 3
  • …
  • 5
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Ultimate Lab Resource for VCF 9.0 06/25/2025
  • VMware Cloud Foundation (VCF) on ASUS NUC 15 Pro (Cyber Canyon) 06/25/2025
  • VMware Cloud Foundation (VCF) on Minisforum MS-A2 06/25/2025
  • VCF 9.0 Offline Depot using Synology 06/25/2025
  • Deploying VCF 9.0 on a single ESXi host? 06/24/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025