WilliamLam.com

Two methods to network boot Raspberry Pi 4

by // 27 Comments

My Raspberry Pi 4 (8GB) model just came last week and after completing my RADIO (VMware's R&D Innovation Offsite) session recording, I wanted to setup my new rPI so I can start playing with it when I had some spare time. I also have the 4GB model but it was running quite hot as I was using the default case (do not recommend) and decided to put that aside for now. I ended up purchasing the 8GB model from Canakit which includes additional heatsinks and nice built-in fan with their custom case.

After spending some time setting up the rPI OS and applying the various updates, I was not too keen on using the SD Card, especially as some folks on forums mentioned that it can easily be worn out depending on the type of workload. While researching online and some references pointed out by colleagues, I came to learn that in addition to booting from USB which is now possible with rPI, you can also network boot the rPI without an SD Card or any storage for that matter! This immediately resonated with me, especially as I am a huge fan of scripted installations such as Kickstart/Jumpstart.

Here are all the resources that I had used that provided all the heavy lifting. I have optimized some of the commands to make it easy for anyone to simply copy and past who is new to setting up an rPI. The authors below did a fantastic job in outlining all the details, so I highly recommend a read if you would like to get more information. I also will be going over an alternative method at the end for booting the rPI over the network which is not really talked about.

One really cool thing that I came to learn while setting up the infrastructure to network boot an rPI was the use of dnsmasq, which I have used in the past but I did not realize it could do so much more. I may have spent more time playing with dnsmasq than with the rPI itself and I will probably cover this in another blog post on how you can easily setup a PXE/gPXE/iPXE system to enable automated OS installation (e.g. Kickstart) that can be used with ESXi or any other OS for that support network installations via BIOS/UEFI.

[Read more...]

Configure NSX-T Edge to run on AMD Ryzen CPU

by // 13 Comments

The vast majority of VMware Homelabs is still Intel-based today but I have been seeing a slow rise of AMD-based kits being adopted, especially with AMD's desktop line of CPUs known as Ryzen. One of the considerations on whether you could use an AMD processor was whether you were planning to deploy NSX-T and in earlier releases, only Intel was supported as the NSX-T Edge required support for Data Plane Development Kit (DPDK) and this was only supported with Intel-based processors.

With the latest NSX-T 3.0 release, AMD-based processors are now supported and per the release notes, the following CPUs can be used:

  • AMD EPYC 7xx1 Series (Naples)
  • AMD EPYC 3000 Embedded Family and newer
  • AMD EPYC 7xx2 Series (Rome)

You will notice that only AMD's server line of CPUs known as EPYC are currently supported, which makes sense for running Production workloads. If you attempt to deploy an NSX-T Edge Node running on a non-EPYC platform, you will get an error message stating the CPU is not supported and I figured this was probably due to the lack of DPDK support in the consumer CPUs.

Yesterday, in our internal "Homelab" Slack channel, I came across an interesting tidbit from Andrea Spagnolo, a Sr. Field Engineer in our Cloud Native Business Unit who shared a pretty neat trick on how to get latest NSX-T 3.0 release to work with a Ryzen-based CPU.

Disclaimer: This is not officially supported by VMware. The behaviors described here can change in the future

First off, I want to thank Andrea for sharing but also credit to Beniamino Guarnaschelli and his blog post here which actually gave Andrea the idea to take a closer look as he was trying to get this setup in his own personal homelab.

[Read more...]

Changing the default size of the ESX-OSData volume in ESXi 7.0

by // 29 Comments

In ESXi 7.0, a new partition scheme was introduced which also brings along a new set of storage requirements. These changes are explained in the official documentation here and the following VMware KB 77009 also contains some additional info which can be helpful. Storage changes are not easy but this was necessary to not only better support some of the current capabilities but more importantly, it setups the foundation for future ESXi capabilities.

The biggest change to the partition layout is the consolidation of VMware Tools Locker, Core Dump and Scratch partitions into a new ESX-OSData volume (based on VMFS-L). This new volume can vary in size (up to 138GB) depending on a number of factors including the current ESXi boot media (USB SD-Card, Local Disk) but also the size of the device itself, which is explained in the official documentation.

From some of the comments on Twitter, Reddit and the direct inquiries that I have received, this new behavior seems to be most impactful to smaller homelabs where a fresh install of ESXi 7.0 has been performed. Folks have shared that their ESX-OSData volume has taken up 120GB which can be quite significant if you have a smaller disk which can be quite common. I normally install ESXi on a USB device and I also use vSAN, which has a different behavior and I have also not upgraded my physical ESXi host (E200-8D) to 7.0 yet.

I performed a fresh installation of ESXi 7.0 (running as Nested ESXi VM) that was configured with 1TB of storage and here is what the filesystem layout now looks:


We can see that the ESX-OSData volume takes up ~119.75GB, which is not too bad for 1TB volume but I can understand this may not be ideal if you have something smaller such as 250GB to 512GB disk. Due to the size of the local device, the boot options mentioned in the KB would not be helpful and I was curious myself if this ESX-OSData volume size could be configurable. In doing some research it looks like the size of the ESX-OSData can be specified using the following ESXi boot option (SHIFT+O during the initial boot) called autoPartitionOSDataSize

UPDATE (12/17/20) - Official support for specifying the size of ESX-OSData has been added to the release of ESXi 7.0 Update 1c with a new ESXi kernel boot option called systemMediaSize which takes one of four values:

  • min = 25GB
  • small = 55GB
  • default = 138GB (default behavior)
  • max = Consumes all available space

If you do not require or have 138GB for the ESX-OSData, you can override the default behavior by appending this option with the specified value (e.g. systemMediaSize=min). It is worth noting that by using this setting, the smallest ESX-OSData volume you can configure is 25GB. For homelabs or environment which require less than this, you would have to use the unsupported autoPartitionOSDataSize parameter , which is not officially supported as mentioned below.

Disclaimer: This may not be officially supported by VMware as it deviates from the system defaults and can have other unintended behaviors. Use at your own risk.

[Read more...]