One feature in VMware Cloud Foundation (VCF) 9.1.1 that I am personally excited about is the new AI Assistant for VCF. It is great to see AI being integrated directly into VCF for infrastructure teams, and while this is just the first release, I think it will open up many interesting possibilities for troubleshooting, reporting, operational insights, and ultimately improving operational efficiency by giving infrastructure teams valuable time back.
More importantly, as we enter a world where AI agents can operate at a scale and speed that humans simply cannot match, these capabilities will become increasingly important. AI will not only help infrastructure teams scale their operations, but also better prepare them to defend against and respond to threats operating at machine speed.
AI Assistant for VCF currently supports both cloud-based and VCF Private AI Services (PAIS) LLM model providers. For ease of setup and to quickly get started with the new capability, I will walk through the configuration using Google Gemini, which is currently the only supported cloud-based LLM provider.
Requirements:
- Google Gemini API Key (https://aistudio.google.com/api-keys)
- VCF 9.1.1 deployment
Step 1 - Enable Log Management by logging in to VCF Operations and navigating to Build > Lifecycle > VCF Management > Add Component to begin the deployment.
Step 2 - In VCF Operations UI navigate to Extensions > Extension Management and activate both VMware Intelligent Assist and Advanced Configuration extensions.
Note: When VCF SSO is configured, the required permission to view the Extensions tile is not enabled by default. You can either log in using the local admin account to view, configure, and use AI Assistant for VCF, or update the Administrator role by navigating to Operate > Administration > Control Panel > Access Control > Roles > Administrator and enabling the Extensions permission.
Step 3 - Navigate to Extensions > Advanced Configuration to enable VMware Intelligent Assist service
Step 4 - Navigate to Extensions > Advanced Configuration > LLM Providers to add an LLM provider, which can be either PAIS or Google Gemini. For this example, we will use Google Gemini. Select Google Gemini as the provider and enter your Gemini API key to complete the configuration.
Step 5 - Navigate to Extensions > Advanced Configuration > Models to select your desired Google Gemini Model.
Note: While the model provider will list all available models to choose from, not all models may actually be usable. I ran into this issue after selecting the first model in the list, only to later discover that it had been deprecated. Unfortunately, no error was surfaced by the service, which made troubleshooting the issue less obvious.
We are now ready to play with VMware Intelligent Assist! 🤖
Navigate to Extensions > VMware Intelligent Assist and start talking to your VCF Infrastructure. You list the available skills (which can also be seen by navigating to Extensions > Advanced Configuration > Tools) or use some of the pre-canned prompt as shown in the example below by summarizing all vCenter Server issues.
We can also get creative with our prompts and do something like the following:
Provide a table summarizing all VMs by Name, vCPU, Memory, Storage Provisioned and use an emoji to represent each VM based on what you think their functions are
As you can see from the output, VMware Intelligent Assist not only provided the response in the requested format, but also included contextual links to each resource found within VCF Operations. This makes the experience extremely useful for troubleshooting, investigating issues, or generating reports.
Note: The VMware Intelligent Assist chat session is currently not persisted. If you navigate to another view and return, the previous chat history will be lost. This is something I have already shared with the Engineering team and hopefully in a future update, users will be able to see all prior chats similar to how you interact with other AI Agent platforms.
You can also view a number of statistics about your model usage by navigating to Extensions > Advanced Configuration > AI Observability. This includes metrics such as token usage, average tokens per second (TPS), and average request latency, along with insights into the types of skills being used and audit logs for the service.
Very cool for Internet connected VCF instances!
Stay tuned, I've got a blog post for deploying your own models which can also power AI Assistant for VCF, which I suspect most users would be interested in that can not use Cloud LLM models
Do we need full stack VCF suite to use this or can be done in VVF as well.
I just checked in my VVF 9.1.1 instance and the option to add this is not present.
Hi William,
While adding the gemini API key, I get the error :"The API key is invalid or has insufficient permissions."
The same key works using a curl command via ssh in vcfops, and can display the available models, Cant figure out what Im doing wrong here. Only issue, the env is running on an eval until all testing can be completed.
Ive tried 3 different keys just to be sure...
That's strange ... I believe there are "different" Gemini systems, do you happen to know if you're using Gemini Enterprise or some other variation? In my example, I am simply using my personal Google Gemini account and given the error throws insufficient permissions, I'm wondering if we need specific permissions based on your Gemini instance
Same, personal Gemini account. 0 restrictions on the key. It is however an isolated environment , with only vcfops and the sddc manager having internet access. Does anything else need internet access here?
Or any suggestion on a log file I could check for what’s going wrong ?
Any help is much appreciated 😀
Ah! When you enable AI Assistant for VCF, it deploys the service on VCFMS, which means all worker nodes must have outbound connectivity to Gemini API endpoint
100 %, that was the issue, all good now, thank you 🙂