WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Resources
    • Nested Virtualization
  • VMware Nostalgia
  • Apple

Frigate NVR with Coral TPU & iGPU passthrough using ESXi on Intel NUC

05.22.2023 by William Lam // 8 Comments

For the past couple of weeks, I have been investigating some issues reported by the community when using ESXi with the popular Google Coral Edge TPU for accelerating machine learning (ML) inferencing. Fortunately, with the help from one of our engineers, Songtao, we were able to find a solution! You can find the complete write-up HERE and it also works with the latest ESXi 8.0 Update 1 release.

I was actually surprised at how popular the combination of the Google TPU and ESXi was from the community, which I guess should not come as a surprise, especially with the capabilities of ESXi coupled with all the interests in AI/ML these days.


Another popular use case of the Google TPU, which I had recently learned about is for real-time AI object detection using the Frigate NVR (Network Video Recorder) software, which is a commonly deployed solution that enable various home security and automation capabilities.

In fact, during a conversation with my buddy Alan Renouf, who is a Product Manager focused on running modern Edge workloads and is also a Frigate user, I discovered that the Frigate stack, which encompasses inferencing, video decoding, and the integration of cameras and sensors, closely resembles the components that you would find in many Edge deployments with simliar set of use cases.

Funny enough, I ended up leveraging a lot of my existing work with running ESXi on Intel NUCs and iGPU passthrough, while learning about and setting up Frigate! This was definitely an interesting project to explore and as shared, I now have a complete working setup with the full setup and write-up below.

JFYI - I have already submitted a PR 6576 to update the Frigate ESXi documentation as it is severely out of date and help folks quickly find the latest setup instructions.

Earlier this week I had no idea what Frigate NVR was ...

Today, full setup w/ESXi on Intel NUC (this thing is amazing, SO many use cases) 🥳

✅ Passthrough Google Coral USB TPU (inferencing)
✅ Passthrough Intel iGPU (vid
decoding)
✅ RTSP enabled camera #AlwaysBeLearning pic.twitter.com/Qghj7qwOFp

— William Lam (@lamw.bsky.social | @*protected email*) (@lamw) May 18, 2023

[Read more...]

Categories // Automation, ESXi, vSphere Tags // Coral, ESXi, Frigate, Intel NUC

Google Coral USB Edge TPU Accelerator on ESXi

05.10.2023 by William Lam // 58 Comments

Several weeks back, I came across a really strange post on the VMTN communities asking how to change the Device ID (DID) and Vendor ID (VID) for a USB Device that has been passthrough to a VM from ESXi? The device in question is the Google Coral USB Edge TPU (Tensor Processing Unit) Accelerator, which is a relatively in-expensive device that can help accelerate machine learning (ML) inferencing. With all the buzz these days with Generative AI and ChatGPT, I can only imagine its popularity has grown even further but I did not realize how popular this device has been in the community, especially for those wanting to use it with ESXi.

The initial observation reported by this user and also by many others in the Coral community was that ESXi was showing the incorrect VID/DID for the Coral USB device and because of this, it was not working correctly when passthrough'ed to a VM and they were looking for a way to change the DID/VID value from 1a6e:089a (Global Unichip Corp.) to 18d1:9302 (Google Inc.).

Interestingly enough, a couple of weeks ago, my buddy Alan Renouf had also shared that he recently purchased the Coral USB device, so I figured I would check with him first to see if he was observing the same behavior that was being reported, which he was. I had been going through the Github reports to try better understand the issue and some of the previous workarounds that users had done including disabling the vmkusb module, which I definitely not recommended, especially for more recent releases of ESXi where that will simply disable all USB functionality to your ESXi host.

I still could not wrap my head around the issue as the reports did not make any sense in terms of the DID/VID not being claimed correctly or that it needed to change to properly function. This also did not make sense when speaking with our USB expert (Songtao who also developed our USB Network Native Driver for ESXi), so I decided to bite the bullet and purchase the Coral USB device, which apparently is difficult to obtain unless you overpay on Amazon, which I did.

[Read more...]

Categories // ESXi, vSphere 7.0, vSphere 8.0 Tags // AI, Coral, ESXi 7.0, ESXi 8.0, ESXi 8.0 Update 1, TPU, usb

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • VMware Flings is now available in Free Downloads of Broadcom Support Portal (BSP) 05/19/2025
  • VMUG Connect 2025 - Minimal VMware Cloud Foundation (VCF) 5.x in a Box  05/15/2025
  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025
  • Quick Tip - Validating Broadcom Download Token  05/01/2025
  • Supported chipsets for the USB Network Native Driver for ESXi Fling 04/23/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...