With the release of the highly anticipated ESXi-Arm Fling, I thought it would be useful to share my hardware build-of-materials that I am currently using with ESXi-Arm Fling, especially as folks have been asking about what is possible or things to be aware of before purchasing a Raspberry Pi.
I do want to stress that the components listed below is just one of many options, it is highly recommend folks carefully review the ESXi-Arm Fling documentation to understand which accessories are supported along with some of their constraints prior to making a purchase.
JMicron M.2 to USB 3.0 Enclosure - Although you can use a standard USB 3.0 storage device for a local VMFS volume, they are usually not the most reliable. I prefer to use a USB 3.0 enclosure that supports standard M.2 NVMe device, which can then be used for both VMFS as well as vSAN, which I have previously blogged about before.
RSHTECH Powered USB 3.0 Hub - USB devices can consume a significant amount of power and can put additional stress on the rPI and cause unstable behaviors. If you are going to use a USB to M.2 enclosure, it is highly recommend that you do not connect it directly to the rPI but rather through a USB-powered hub. Another benefit of having a USB hub is that if you are going to setup vSAN Witness, you will need additional USB slots for both the caching and capacity USB devices and having extra USB slots will be required.
Canakit Raspberry Pi 4b (8GB) - It is highly recommended that folks invest in the rPI 4b (8GB) model so that you can do more when running ESXi-Arm, especially if you intend to setup a vSAN Witness or run additional workloads. There are a number of online vendors where you can purchase rPI 4b, Canakit is just one such vendor and what I like about them is a complete "kit" including a case and fan.
Note: I can NOT stress enough the importance of having proper cooling (passive or active) when purchasing an rPI. These devices do run hot, even under light load and you should have proper cooling to ensure you do not have unstable behavior or worse cause damage to the device. In fact, I had to learn this the hard way with my first rPI 4b purchase which was the 4GB model and the stock heatsink and case did NOTHING. I ended up purchasing a new case + fan which really helped and you can get more details below.
MazerPi Raspberry Pi 4 Case + Fan - I was an earlier adopter of the rPI 4b (4GB) model and assumed the stock case with heatsinks would be sufficient. Not only was the device too hot to the touch but the high temperature also affect my ESXi-Arm usage (there is a blinking thermometer icon on the upper right corner of the ESXi DUCI if you have a monitor connected). There are many rPI cases with fans, I highly recommend folks do their research but just make sure you have proper cooling whether you are using either models of the rPI 4b.
Realtek RTL8153 USB Gigabit Ethernet - In early development of ESXi-Arm, only USB network adapters were originally supported. At launch, we now fully support the on-board network adapter which will provide the best experience compared to USB networking. For those that wish to connect to additional networks and/or create network isolation, additional USB NICs can be added. I have two of these connected up as shown in the diagram and can be used for basic vSphere Management networks to even isolating vSAN traffic (though you are probably better off using the on-board NICs for that type of traffic). Today, we only support RTL8153 USB NICs and specifically those with VID of 0bda, so not all RTL8153 will function. In the future, the ESXi-Arm team would like to integrate the popular USB Network Native Driver for ESXi Fling which would allow for additional USB NICs to be used.
RAVPower 4-Port USB Charger 40W - Just purchased this recently. It came as a recommendation by fellow colleague from the ESXi-Arm team (Cyprien) which be used to power multiple rPI which will be handy for anyone planning to have a few rPI and not have to deal with running out of power outlets.
4-Pack USB-C to USB Cables - Perfect length to go with the RAVPower USB charger and also recommended by Cyprien.
Additional Notes
- Network (NFS/iSCSI) based storage is another alternative for running your ESXi-Arm workloads, not only is this easier to maintain but shared storage is also required if you plan to have multiple rPI and wish to take advantage of vMotion. For my setup, I also have an NFS volume that is shared between both the 4GB & 8GB rPI. In fact, if you have vSphere 7.0 or greater, you can leverage vSAN File Services to export an NFS volume for this very use 🙂
- Fellow colleague Andrei is also using iSCSI to boot ESXi-Arm over the network, so instead of installing it onto a USB storage device, you can boot over the network
- If you have access to Power over Ethernet (PoE), you could reduce the amount of cabling to the rPI with single Ethernet cable. I knew few folks have done in their setup
Knox says
Can you run Tanzu on it?
William Lam says
Not currently. All the Tanzu offerings deploy some form of x86 VMs (whether thats the Control Plane for vSphere w/Tanzu or K8s VM for Tanzu Kubernetes Grid / Tanzu Kubernetes Grid Multi-Cloud). Was there a specific use case you were hoping to achieve with using Tanzu?
Richard Hughes says
I have ESXi running on my Pi 4s I cant figure out how to boot a VM, is there any guide or help for that? I'd be happy to do a how-to video on ARM VMs https://youtu.be/wVCp5I7BVtI
William Lam says
Richard,
Please take a look at the documentation (I believe you may have already asked this in Slack channel). It has detailed instructions on how to install GuestOS which is by uploading the desired ISO, mounting it to VM and powering it on and using the VM Console to perform installation.
Richard Hughes says
TY! I see the procedure in slack. Will give it a try!
bradlay2 says
This seems like a good solution to keep things nice, neat and compact?
https://www.argon40.com/argon-one-m-2-case-for-raspberry-pi-4.html
ETA prime did a good video on it also - https://www.youtube.com/watch?v=LLc4pF3jZQg
barthedv says
It seems that the RavPower Power Supply delivers 2.5 A per port, the official Power Supply delivers 3 A. Did you have any issues with that?
Richard Hughes says
The RAVPOWER Powervault is my FAVORITE! It can be used as a UPS for the Pi as it can be charged at the same time it is discharged. If you are not sucking power from the USB ports with spinning disks & just using thumb drives, 2.5 amps will be fine!
Bas says
The Raspberry Pi firmware will display a warning icon on the upper right corner of the ESXi DUCI, to indicate an issue with Undervoltage and Over Temperature.
Are these (important) warnings also noticed bij ESXi ?
Nick fusco says
I bought one of those Cable Matters USB adapters and it only seems to work at 10/100
it is obda:
Bus 001 Device 004: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
Shows up as:
vusb0
Link speed:100 Mbps
Driver: cdce
Kinda disappointed 🙁
William Lam says
The link speed is actually a cosmetic issue (hopefully it'll be fixed in a future update). You definitely will get more than 100Mbps but there are still some constraints with USB on Pi, so you'll still have best performance when using the onboard NIC
oildiuelt says
for 'usb ethernet adapter' i mean those with RTL8153 chip.
Tim Sheppard says
Hi William,
Have been running the ESXi ARM fling for a while now. Have recently tried adding a second NIC (same Cable Matters / Realtek USB NIC you are using) but it seems to be stuck running at 100Mbs. Have tried forcing it to run at 1000Mbs with...
esxcli network nic set -n vusb0 -S 1000 -D full
And this shows up in the GUI as configured for 1Gbs but still runs at 100Mbs.
Is there some additional configuration I need to do in order to get this NIC running full speed or does your USB NIC only run at 100Mbs too?
Tim Sheppard says
Have just found Cyprien's reply from over a year ago...
"Hi! We're happy you are loving it 🙂 For the USB NIC, yes it shows up at 100Mbit, but can go higher than that. However due to the USB hardware and software constraints you may not get much more from it. We probably need to list that in the known issues."
Quoting it here so that hopefully anyone searching in the future can find it.
flyingRich (@FlyingRich) says
I am working on a case for my Pi Cluster. https://www.youtube.com/watch?v=UwOYFt0BL7U Would any of you have interest in the case? I may just release the plans.