WilliamLam.com

VMware Cloud Foundation 5.0 running on Intel NUC

by // 10 Comments

Interested in trying out the latest release of VMware Cloud Foundation (VCF) 5.0? Don't have some beefy hardware to meet all the requirements, not to worry! Did you know you can actually deploy a VCF Management Domain using just a single Intel NUC or simliar small form factor system? This is exactly how I kicked the tires with the latest VCF 5.0 release 😎


Disclaimer: This is not officially supported by VMware, please use at your own risk.

Requirements:

  • VMware Cloud Builder 5.0 OVA (Build 21822418)
  • VCF 5.0 Licenses
  • Intel NUC configured with
    • 64GB of memory or more
    • Dual onboard networking ("Tall" NUC like Intel NUC 11 Pro, which is what I used) OR add additional NICs with these Thunderbolt 3 Networking options (no USB NIC)
    • 2 x SSD that are empty for use for vSAN bootstrap (500GB+ for capacity)
  • ESXi 8.0 Update 1a installed on the Intel NUC using USB device
  • Ability to deploy and run the VMware Cloud Builder (CB) Appliance in a separate environment (ESXi/Fusion/Workstation)

Note: While my experiment used an Intel NUC, any system that meets the basic requirements above should also work.

[Read more...]

Frigate NVR with Coral TPU & iGPU passthrough using ESXi on Intel NUC

by // 8 Comments

For the past couple of weeks, I have been investigating some issues reported by the community when using ESXi with the popular Google Coral Edge TPU for accelerating machine learning (ML) inferencing. Fortunately, with the help from one of our engineers, Songtao, we were able to find a solution! You can find the complete write-up HERE and it also works with the latest ESXi 8.0 Update 1 release.

I was actually surprised at how popular the combination of the Google TPU and ESXi was from the community, which I guess should not come as a surprise, especially with the capabilities of ESXi coupled with all the interests in AI/ML these days.


Another popular use case of the Google TPU, which I had recently learned about is for real-time AI object detection using the Frigate NVR (Network Video Recorder) software, which is a commonly deployed solution that enable various home security and automation capabilities.

In fact, during a conversation with my buddy Alan Renouf, who is a Product Manager focused on running modern Edge workloads and is also a Frigate user, I discovered that the Frigate stack, which encompasses inferencing, video decoding, and the integration of cameras and sensors, closely resembles the components that you would find in many Edge deployments with simliar set of use cases.

Funny enough, I ended up leveraging a lot of my existing work with running ESXi on Intel NUCs and iGPU passthrough, while learning about and setting up Frigate! This was definitely an interesting project to explore and as shared, I now have a complete working setup with the full setup and write-up below.

JFYI - I have already submitted a PR 6576 to update the Frigate ESXi documentation as it is severely out of date and help folks quickly find the latest setup instructions.

[Read more...]

GPU Passthrough with Nested ESXi

by // 9 Comments

Advancements in ESXi Nested Virtualization have given us the ability to run ESXi inside of a VM (Nested ESXi) and has allowed us to do just about anything you would with a physical ESXi host for development, testing and learning purposes. In fact, I have shared many tips and tricks for using Nested ESXi and Nested Virtualization over the years on my blog, which is worth bookmarking in case you are trying to do something and run into an issue which more than likely, I have come across.

Today, there is very little you can not do using Nested ESXi and is typically limited to a physical device that can not be virtualized and/or emulated in software.

I bring this up because I recently had a chat with Frank Denneman on an unrelated topic and he brought up the question about being able to double passthrough of a GPU from a physical ESXi host into a Nested ESXi VM which would then be passthrough'ed again to a VM running on that Nested ESXi system. While this was not the first time that I had heard of such a request, it does not come up often, this has only been the second time I have heard of this request. For context, his use case was for testing purposes and I can certainly see some interesting scenarios where you want to run vSphere in a Nested environment and still access all the vSphere capabilities including leveraging a physical GPU within that environment, whether that is AI/ML or other graphics process requirements.

My response to Frank was this will not work for a few reasons, one of which is that the use of Virtual Hardware-Assisted Virtualization (VHV) is not supported with DirectPath I/O and if the GPU is passthrough to a VM, even if it was running ESXi, it would be in control of the GPU, so how could one passthrough it again?

My curiosity got the better of me and given this was the second time I had ever been asked about this, I figured maybe it was worth exploring but before I go down anymore 🐇🕳️, I wanted to get quick sanity check from one of our graphics engineers on the remote feasibility of this ask.

[Read more...]