WilliamLam.com

Frigate NVR with Coral TPU & iGPU passthrough using ESXi on Intel NUC

by // 8 Comments

For the past couple of weeks, I have been investigating some issues reported by the community when using ESXi with the popular Google Coral Edge TPU for accelerating machine learning (ML) inferencing. Fortunately, with the help from one of our engineers, Songtao, we were able to find a solution! You can find the complete write-up HERE and it also works with the latest ESXi 8.0 Update 1 release.

I was actually surprised at how popular the combination of the Google TPU and ESXi was from the community, which I guess should not come as a surprise, especially with the capabilities of ESXi coupled with all the interests in AI/ML these days.


Another popular use case of the Google TPU, which I had recently learned about is for real-time AI object detection using the Frigate NVR (Network Video Recorder) software, which is a commonly deployed solution that enable various home security and automation capabilities.

In fact, during a conversation with my buddy Alan Renouf, who is a Product Manager focused on running modern Edge workloads and is also a Frigate user, I discovered that the Frigate stack, which encompasses inferencing, video decoding, and the integration of cameras and sensors, closely resembles the components that you would find in many Edge deployments with simliar set of use cases.

Funny enough, I ended up leveraging a lot of my existing work with running ESXi on Intel NUCs and iGPU passthrough, while learning about and setting up Frigate! This was definitely an interesting project to explore and as shared, I now have a complete working setup with the full setup and write-up below.

JFYI - I have already submitted a PR 6576 to update the Frigate ESXi documentation as it is severely out of date and help folks quickly find the latest setup instructions.

[Read more...]

96GB SODIMM memory for DDR5 system with ESXi

by // 26 Comments

Back in February of this year, I had shared that non-binary 24GB and 48GB SODIMM memory was finally available, but I quickly realized that the news from Samsung was only for traditional memory modules and not the laptop SODIMM memory, which is also commonly used in small form factor systems like an Intel NUC.

Three months later, we still have no word from either Samsung or Crucial, but recently Mushkin came out of no where and released their 48GB DDR5 SODIMM modules which I had also shared the news on my blog HERE. While DDR5 adoption is slowly increasing, there are not many systems out there right now that currently supports DDR5 and you will need a DDR5 capable system to use DDR5 memory.


I recently got my hands on a new DDR5 system, which I will share more details in a future blog post, but the maximum supported memory listed for the system is still 64GB. I was curious on whether these new 48GB SODIMM would actually work with this system? If we go back to 2019, when 32GB SODIMM was first released, it was not clear whether these would work with systems that listed 32GB as their maximum supported memory?

I was the first to confirm 32GB modules worked with the popular Intel NUCs back in 2019 and that I could even use these new SODIMM modules going all the way back to an Intel 6th Gen NUC, which was released back in 2016! So while the officially tested memory limit was only 32GB, the CPU actually supported more than that!

🙏🤞I decided to take another chance and I purchased the Mushkin 96GB DDR5 SODIMM 4800mhz kit which has just arrived and I have put my theory to the test again ...

[Read more...]

Extending ESXCLI commands

by // 2 Comments

Back on the first of April, I had shared the following video:


While many thought it was an April Fools's joke, that was certainly the original intention, but I also wanted the joke to be based on something realistic and not made up. Everything that you saw in that video is completely real, this was a real ESXCLI command that integrated with ChatGPT and was running on an ESXi 8.0 Update 1 release! 😊

So how did I do it?

[Read more...]