WilliamLam.com

Search Results for: NUC

Modding an NVIDIA RTX 4000 Ada (20GB VRAM) into Minisforum MS-A2

by // 5 Comments

The Minisforum MS-A2 has been my go to platform for running an optimized and fully functional VMware Cloud Foundation (VCF) 9.0 lab environment. From vSAN ESA to NSX VPC to vSphere Kubernetes Service to VCF Automation to Data Services Manager, it runs it all!

Over the past couple of weeks, I had also been experimenting with running VMware Private AI Services (PAIS) using the MS-A2. For those with an eye for detail, may have noticed that the NVIDIA GPU requirement was actually being satisfied by an ASUS NUC 14 Performance, which includes an NVIDIA GeForce RTX 4070 (8GB VRAM). The ASUS NUC was purely for validation and prototyping purposes to ensure that I could actually run PAIS before investing in a more capable and certainly more expensive NVIDIA GPU 🤑

One of the lesser known capabilities of the MS-A2 is the ability to add a half-height PCIe device and from the very beginning when I had first heard about the MS-A2, my plan and hope was to be able to add an NVIDIA GPU to the system!

[Read more...]

MS-A2 VCF 9.0 Lab: Deploying Model Endpoint with DirectPath I/O using VMware for Private AI Services (PAIS)

by // Leave a Comment

In this final blog post, we will now deploy several AI model endpoints (downloaded from Hugging Face), configure our private data source which can be a shared location (Google Drive, Confluence, Microsoft Sharepoint, or S3-compatible endpoint) or using local files and then consuming them using an AI Agent that is built using VMware for Private AI Services (PAIS).

As mentioned in the very first blog post of this mini-series, my goal was to get hands experience with PAIS but without the need to have an NVIDIA GPU capable of vGPU, which would also require an NVIDIA AI for Enterprise (NVAIE) license.

Luckily, we can use an NVIDIA GPU via DirectPath I/O, thanks to the backend plumbing the PAIS Engineering team have built and had shared with me 😊

For my proof of concept, I am using an ASUS NUC 14 Performance, which has an NVIDIA GeForce RTX 4070 mobile GPU (8GB VRAM). The ASUS NUC 14 is running alongside my Minisforum MS-A2 setup, is only used to deploy the completions model endpoint. The use of the ASUS NUC 14 is purely for prototyping and experimentation purposes to demonstrate that anyone can play with PAIS within their lab environment. I plan to use a more powerful NVIDIA GPU setup, which I will share more details at a later point for those interested.

References:

Requirements:

[Read more...]

Ultimate Lab Resource for VCF 9.0

by // 12 Comments

Since the release of VMware Cloud Foundation (VCF) 9.0, you may have seen a few blogs from me ?

To help folks digest all the resources for deploying and running VMware Cloud Foundation (VCF) 9.0 in a lab environment, I have put this blog post together to summarize the various considerations and/or tweaks that will allow you to get the full experience of VCF 9.0 for non-production/commercial usage.

Make sure you to bookmark this page as I will continue to add new updates, I still have a number of backlog items to share ... but for now that will have to wait as I am in need of some PTO, which starts tomorrow! ?

[Read more...]