Since the release of vSphere 7.0 Update 1, the demand and interests from the community on getting hands on with vSphere with Tanzu and the new simplified networking solution, has been non-stop. Most folks are either upgrading their existing homelab or looking to purchase new hardware that can better support the new features of the vSphere 7.0 release.
Although vSphere with Tanzu now has a flavor that does not require NSX-T which helps reduces the barrier on getting started, it still has some networking requirements which may not be easily met in for all lab environments. In fact, this was actually the primary reason I had started to look into this since my personal homelab network is very basic and I do not have nor want a switch that can support multiple VLANs, which is one of the requirements for vSphere with Tanzu.
While investigating for a potential solution, which included way too MANY hours of debugging and troubleshooting, I also thought about the absolute minimal amount of resources I could get away with after put everything together. To be clear, my homelab is comprised of a single Supermicro E200-8D which has 128GB of memory and that has served me well over the years and I highly recommend it for anyone that can fit that into their budget. With that said, I did set out with a pretty aggressive goal of using something that is pretty common in VMware homelabs which is an Intel NUC and with just 32GB of memory.
UPDATE (09/17/22) - The steps outlined in this blog post is also applicable for running vSphere with Tanzu using vSphere 8.
Sweet! Looks like running a complete vSphere w/Tanzu lab with with just 32GB of memory (https://t.co/jisSFCTYKM) still works with #vSphere8 🙌
I'm using my latest Intel NUC 12 Pro https://t.co/DEgOQnF3zr but any prior generation works too pic.twitter.com/oGPtzHzJhC
— William Lam (@lamw) September 17, 2022
Here is the hardware BOM (similar hardware should also work):
- Intel NUC 10i7FNH
- 32GB memory
- Single 250GB M.2 NVMe SSD
- NUC can support two SSD (M.2 + SATA), you can always go larger
Here is the software BOM:
- vCenter Server Appliance 7.0 Update 1 Build 16860138
- ESXi 7.0 Update 1 Build 16850804
- HAProxy v0.1.8 OVA
- Photon OS 3.0 OVA
Note: The Intel NUCs (Gen 6 to 10) can all support up to 64GB of memory and this is one of the best upgrades you can give yourself, but if you only have 32GB of memory, this will also work.
The final solution will comprise of the following:
- 1 x vCenter Server Appliance (VCSA) running on the Intel NUC self-managing the ESXi host
- VMFS storage will be used instead of vSAN to reduce memory footprint (If you have 64GB of memory, recommend using vSAN)
- Onboard NIC will be used for all traffic and will be attached to a Distributed Virtual Switch (VDS)
- 3 x Distributed Portgroups will be configured on top of your existing LAN network, the latter two will be routed through our Photon OS Router VM
- Management - Existing LAN network
- Frontend - 10.10.0/24
- Workload - 10.20.0.0/24
- 3 x Distributed Portgroups will be configured on top of your existing LAN network, the latter two will be routed through our Photon OS Router VM
- 1 x vSphere with Tanzu Cluster enabled with Workload Management
- 1 x HAProxy VM deployed using 3-NIC configuration
- 1 x Photon OS Linux VM used as a Router for IP forwarding and optionally, a DNS server if you do not already have one
- 9 x IP Addresses in total will be required from your local LAN network
- 4 x IP Addresses which should map to following hostnames or similiar
- esxi-01.tanzu.local
- vcsa.tanzu.local
- router.tanzu.local
- haproxy.tanzu.local
- 5 x IP Addresses in a consecutive block (e.g. 192.168.30.20-192.168.30.25) will be needed for the Supervisor Control Plane VMs
- 4 x IP Addresses which should map to following hostnames or similiar
As part of this solution, I have automated as much of the tasks as possible and all scripts used for this solution can be found at https://github.com/lamw/vsphere-with-tanzu-homelab-scripts which I will be referencing throughout the instructions. There are also a number of techniques and tricks I am using to be able to reduce the overall memory footprint for setting up vSphere with Tanzu, obviously these should not be used in a Production grade environment.
I also want to give a huge thanks to Timo Sugliani for all of his help with the networking question/challenges and Mayank B. from the vSphere with Tanzu Engineering team who helped with the debugging and ultimately making this solution a possibility.
[Read more...]