WilliamLam.com

For those following along, earlier this week I demonstrated how to configure the new AI Assistant for VCF in VMware Cloud Foundation (VCF) 9.1.1 using Google Gemini as a cloud-based LLM provider, which is a quick way to get started and explore the new capability.

Alongside VCF 9.1.1, we also released VCF Private AI Services (PAIS) 3.0, which enables users to deploy and run their own AI models locally within VCF. In my follow-up articles, I walked through setting up PAIS for an air-gapped environment and then deploying an AI model downloaded from Hugging Face, which PAIS serves through an OpenAI-compatible model endpoint.

In this final blog post, we will bring everything together by connecting AI Assistant for VCF to our PAIS Model Endpoint, allowing the AI Assistant to use an AI model running locally within our VCF environment.

[Read more...]

My recent blog post on the new AI Assistant for VCF in VCF Operations 9.1.1 received a lot of positive feedback from both internal and external readers, especially from those who were not aware that this capability was available and were excited to try it out. The excitement is understandable, as this opens up many interesting possibilities for infrastructure administrators to operate and manage their environments more efficiently.

In that article, I focused on how to enable AI Assistant for VCF and quickly get started using Google Gemini as a cloud-based LLM provider. While that is certainly the fastest way to start exploring the capability, AI Assistant for VCF also supports local AI models deployed using VCF Private AI Services (PAIS) 3.0, enabling customers to take advantage of powerful AI models running securely and more cost-effectively within their own on-premises environment.

In this blog post, we will now focus on deploying and consuming local AI models with PAIS. This builds on another recent article where I walked through the setup of VCF Private AI Services (PAIS) using the Artifact Mirroring Tool (AMT) for air-gapped environments. 

[Read more...]

With the rising cost of consuming AI services in the public cloud, the availability of powerful open-weight models, and growing concerns around security and data sovereignty, there was certainly a lot of interest from attendees at last week’s VMware Explore in running their own AI models within their VCF Private Cloud environments.

The timing could not have been better with the release of VMware Cloud Foundation (VCF) Private AI Services (PAIS) 3.0, which is part of the broader VCF Private AI Cloud offering and perfectly complements the latest VCF 9.1.1 release. VCF 9.1.1 also introduces a number of new AI capabilities for infrastructure administrators, including the new AI Assistant for VCF, which can leverage the same PAIS infrastructure to consume AI models running directly within your VCF Private Cloud environment.

Some key highlights in PAIS 3.0 include the new Model-as-a-Service capability, which can now be shared across different vSphere Namespaces to better utilize precious GPU resources, the ability to consume PAIS services directly from vCenter Server in addition to VCF Automation (VCFA), and support for built-in local identity for streamlined proof-of-concepts (POC)/lab deployments.

While not new, the Private AI Services (PAIS) Artifact Mirroring Tool (AMT) provides an easy way to download all the required container images for non-internet-connected or air-gapped environments. A typical workflow would involve inspecting the various deployment YAML manifests, using the imgpkg utility to export the required container images, and then manually updating the deployment manifests to reference your local container registry, hoping that everything works as expected. Since this was my first time using PAIS AMT, I thought it would be useful to document the workflow for those interested in trying out PAIS in their lab environment.

[Read more...]