vSphere 7.0 Update 2a was just released yesterday and it introduces the highly anticipated vSphere Virtual Machine Service which is part of the vSphere with Tanzu solution. Customers will now be able to get the exact same Kubernetes-native provisioning and management experience for deploying Virtual Machine based workloads along side their Kubernetes-based applications.
I was definitely interested in checking out this new feature and quickly navigated to my VCSA to start the upgrade and to my surprise, I noticed that both the Stage Only and Stage and Install buttons were disabled (grayed out) and preventing me from updating to the latest release!
I had filed an issue internally and after some troubleshooting, it looks like this may have been remnants caused by the previous Exception occurred in install precheck phase issue, which I had ran into during the update to 7.0 Update 2. Although I was initially pointed by Engineering to VMware KB 83145, it did not resolve the issue and I ran into a couple more errors like "Previous patching unsuccessful" and "Downloading RPM ...." messages within the VAMI UI as shown in the screenshots below
It turns out there were some additional files that also needed to be cleaned up and also navigate away from the Updates tab, before you clean up and re-attempt the Stage operation or you will run into a race/caching issue that we had observed. Lastly, I did run into another issue that prevented me from upgrading where the "Downloading RPM ..." message kept showing up even after following the instructions below, this particular problem is still being triaged.
Note: VMware Engineering is currently working on a simplified clean up script that will be published with an updated KB outlining the symptoms and the script usage. The following commands below was used to remediate my environment and it is also a subset of the commands outlined in KB 83145. If you prefer to wait for official guidance, I will update this blog post when that KB is published.
For those interested, below are the commands provided by Engineering to resolve my issue, which I have now successfully upgraded to vSphere 7.0 Update 2a!
Step 1 - Navigate away from the Updates tab
Step 2 - SSH to the VCSA and run the following commands:
rm -rf /storage/core/software-update/updates
rm -rf /storage/updatemgr/software-*
rm /etc/applmgmt/appliance/software_update_state.conf
rm /storage/db/patching.db*
rm -r /storage/core/software-update/*
Step 3 - Click on the Updates tab and then click on "Check Updates". Once the the update shows up, you can then click on the Stage operation and that should now complete successfully and allowing you to install. If you do run into any issues, it is recommended that you file a ticket with VMware Support.
Argh! William love your work and yes I fuddled thru the previous issue as well. Again? Hate to be negative but I wonder if I can trust the complex coding for Tanzu if they haven't tested the upgrade process which to a customer seems "basic"?
Hi Tom,
Appreciate the note and it's a fair comment. All I can say is that the VC Product/Engr team are fully engaged in resolving these various issues. To be transparent, we may see some of these issues again until there's a version that contains the fixes AND then you would not benefit until the next update/upgrade after that. Its certainly not an ideal experience but I'm asking the teams to make sure we fully document these issues in release notes, rather than having to learn about it after the fact
An abnormally high amount if bugs recently... I've got one where I can't upgrade to update 2, still have a ticket open.
try https://kb.vmware.com/s/article/83145
Perfectly Done
Thanks William!
You safed my day with this tipp!