WilliamLam.com

VCF 9.1.1 - OCuLink External Graphics (eGPU) Dock with integrated PSU for GPU Passthrough with VKS

by // 2 Comments

For those who have been following my VCF Private AI Services (PAIS) exploration with deploying my own AI models and using them with Coding Agents running on VCF, I have found that these AI workloads can really push the thermals on my NVIDIA RTX 4000 running inside my Minisforum MS-A2, eventually causing the GPU to start throttling.

Earlier this year, I had shared a solution from a colleague who wanted to leverage a more capable GPU with vSphere Kubernetes Service (VKS) and was able to externalize the GPU using an OCuLink-based eGPU dock that is also compatible with the MS-A2.

I was initially going to use the same eGPU solution, which would have also required purchasing a separate PSU to power the GPU. While looking at my options, I came across another OCuLink eGPU dock (AG01) from Aoostar that had an integrated PSU (up to 800W), was plug-and-play ready, and certainly had a much cleaner aesthetic design.

[Read more...]

VCF 9.1.1 - Using VCF Private AI Services (PAIS) Models with AI Assistant for VCF

by // Leave a Comment

For those following along, earlier this week I demonstrated how to configure the new AI Assistant (Tech Preview) for VCF in VMware Cloud Foundation (VCF) 9.1.1 using Google Gemini as a cloud-based LLM provider, which is a quick way to get started and explore the new capability.

Alongside VCF 9.1.1, we also released VCF Private AI Services (PAIS) 3.0, which enables users to deploy and run their own AI models locally within VCF. In my follow-up articles, I walked through setting up PAIS for an air-gapped environment and then deploying an AI model downloaded from Hugging Face, which PAIS serves through an OpenAI-compatible model endpoint.

In this final blog post, we will bring everything together by connecting AI Assistant for VCF to our PAIS Model Endpoint, allowing the AI Assistant to use an AI model running locally within our VCF environment.

[Read more...]

VCF 9.1.1 - Deploying Your Own AI Models with VCF Private AI Services (PAIS) 3.0

by // 4 Comments

My recent blog post on the new AI Assistant for VCF in VCF Operations 9.1.1 received a lot of positive feedback from both internal and external readers, especially from those who were not aware that this capability was available and were excited to try it out. The excitement is understandable, as this opens up many interesting possibilities for infrastructure administrators to operate and manage their environments more efficiently.

In that article, I focused on how to enable AI Assistant for VCF and quickly get started using Google Gemini as a cloud-based LLM provider. While that is certainly the fastest way to start exploring the capability, AI Assistant for VCF also supports local AI models deployed using VCF Private AI Services (PAIS) 3.0, enabling customers to take advantage of powerful AI models running securely and more cost-effectively within their own on-premises environment.

In this blog post, we will now focus on deploying and consuming local AI models with PAIS. This builds on another recent article where I walked through the setup of VCF Private AI Services (PAIS) using the Artifact Mirroring Tool (AMT) for air-gapped environments. 

[Read more...]