For those following along, earlier this week I demonstrated how to configure the new AI Assistant for VCF in VMware Cloud Foundation (VCF) 9.1.1 using Google Gemini as a cloud-based LLM provider, which is a quick way to get started and explore the new capability.
Alongside VCF 9.1.1, we also released VCF Private AI Services (PAIS) 3.0, which enables users to deploy and run their own AI models locally within VCF. In my follow-up articles, I walked through setting up PAIS for an air-gapped environment and then deploying an AI model downloaded from Hugging Face, which PAIS serves through an OpenAI-compatible model endpoint.
In this final blog post, we will bring everything together by connecting AI Assistant for VCF to our PAIS Model Endpoint, allowing the AI Assistant to use an AI model running locally within our VCF environment.
Requirements:
Step 1 - Log in to vSphere UI and navigate to your vSphere Namespace that has PAIS Model Endpoint running and under Service Configuration tab, make a note of the OpenAI base URL endpoint.
Note: AI Assistant for VCF does not support IP address only base URL, you will need to configure an FQDN when deploying PAIS service
Step 2 - Log in to VCF Operations and navigate to Extensions > Advanced Configuration > LLM Providers, then click Add Provider. Select VCF Private AI as the provider type and enter the PAIS Model Endpoint URL from the previous step along with the API token for authorization.
After accepting the self-signed TLS certificate generated by PAIS, you should see that AI Assistant for VCF has successfully connected to your PAIS Model Endpoint!
Step 3 - Finally, navigate to the Models tab, where you will see all available models running within PAIS. Depending on your environment, you may have one or more models that you can select for use with AI Assistant for VCF. You can easily switch between models, which can be useful for baselining and comparing the capabilities of different AI models.
I have already received questions about the types of interactions you can have with AI Assistant for VCF, such as whether it can generate a document like a PDF. In this first release, AI Assistant for VCF operates in read-only mode, so these types of actions are not currently possible. However, it includes 25 built-in tools, commonly referred to as skills, which you can view under the Tools tab.
In a future release, users will have the ability to create their own skills, and I can even envision a marketplace where Broadcom and the broader community can develop, contribute, and share skills that benefit all users. There are definitely a lot of exciting possibilities, especially as the system evolves beyond read-only interactions to support read-write capabilities with the appropriate access controls and guardrails in place, of course.
Lastly, you can also view your AI Assistant for VCF usage by navigating to the AI Observability tab, which provides insights into token usage, tokens per second, tools used, request latency, and basic audit logs for model configuration changes. There is certainly more that can be done in this area, so if you have any feedback as you start using AI Assistant for VCF, feel free to leave a comment and I will make sure our Product Management team is aware.
I am definitely looking forward to seeing what our users will do with this new capability in VCF Operations!
Thanks for the comment!