With the rising cost of consuming AI services in the public cloud, the availability of powerful open-weight models, and growing concerns around security and data sovereignty, there was certainly a lot of interest from attendees at last week’s VMware Explore in running their own AI models within their VCF Private Cloud environments.
The timing could not have been better with the release of VMware Cloud Foundation (VCF) Private AI Services (PAIS) 3.0, which is part of the broader VCF Private AI Cloud offering and perfectly complements the latest VCF 9.1.1 release. VCF 9.1.1 also introduces a number of new AI capabilities for infrastructure administrators, including the new AI Assistant for VCF, which can leverage the same PAIS infrastructure to consume AI models running directly within your VCF Private Cloud environment.
Some key highlights in PAIS 3.0 include the new Model-as-a-Service capability, which can now be shared across different vSphere Namespaces to better utilize precious GPU resources, the ability to consume PAIS services directly from vCenter Server in addition to VCF Automation (VCFA), and support for built-in local identity for streamlined proof-of-concepts (POC)/lab deployments.
While not new, the Private AI Services (PAIS) Artifact Mirroring Tool (AMT) provides an easy way to download all the required container images for non-internet-connected or air-gapped environments. A typical workflow would involve inspecting the various deployment YAML manifests, using the imgpkg utility to export the required container images, and then manually updating the deployment manifests to reference your local container registry, hoping that everything works as expected. Since this was my first time using PAIS AMT, I thought it would be useful to document the workflow for those interested in trying out PAIS in their lab environment.
