WilliamLam.com

You are here: Home / VCSA / VCSA 7.0 Update 2 Upgrade Issue - Exception occurred in install precheck phase

VCSA 7.0 Update 2 Upgrade Issue - Exception occurred in install precheck phase

by // 34 Comments

Like most folks, I was excited about the release of vSphere 7.0 Update 2 and I was ready to upgrade my personal homelab, which was running on vSphere 7.0 Update 1c. However, after starting my VCSA upgrade in the VAMI UI, it quickly failed with the following error message: Exception occurred in install precheck phase

Given the release had just GA'ed less than an hour ago and everyone was probably hammering the site, I figured I would wait and then try again.

Issue

I normally use the "Stage and Install" option and after accepting the EULA and providing the required credentials, the upgrade began.


Almost immediately, it failed with the same error and the only option was to click on the Resume button which I did.

The upgrade seemed to have resumed and pre-check results passed as you can see from the screenshot below. However, after hitting continue, I would be taken right back to the pre-check error as noted earlier. I repeated this a few times and realize it was stuck in this loop.


I even attempted to reboot the VCSA, maybe my environment had some issue but the results were the same. There were also a couple of suggestions on removing the following state file /etc/applmgmt/appliance/software_update_state.conf on the VCSA filesystem which apparently was a solution that had worked. However, in this case it did not do anything and I still ran into the issue.


At this point, I was not interested poking around or randomly removing files as I heard few other folks were running into this problem. I generated a support bundle and file an internal bug to have Engineering take a look.

Disclaimer: If you are currently facing this problem in a Production environment, please stop and file a VMware Support Request so that this can be officially tracked and you can have GSS reference PR 2732490. The workaround below has only been confirmed by the community and should only be attempted on non-production or homelabs environment where you are comfortable with the workaround.

Workaround

Several folks have shared with me a workaround that they have verified which requires the upgrading the VCSA using the VCSA Shell. I have personally not applied this workaround as I wanted to leave my environment in its existing state, in case VMware Engineering need to troubleshoot live. For those interested, below are the instructions to apply the workaround.

Step 1 - Login to the VAMI UI and under Updates, only select the "Stage" option to download the 7.0 Update 2 updates.

Step 2 - SSH to the VCSA and remove /etc/applmgmt/appliance/software_update_state.conf file

Step 3 - Run the following command to install the staged 7.0 Update 2 software:

software-packages install --url --acceptEulas

Once this operation completes, you should now have a successfully upgraded VCSA to 7.0 Update 2. I will update this blog post as I learn more from Engineering and if this workaround is official, I will ask the team to publish a KB and also mention this in the 7.0 Update 2 release notes.

More from my site

Comments

  1. I ran into this today. The fix for me was staging and then upgrading in two steps, rather than one. The install then proceeded until 80% when it threw an error that I was able to continue past. No issues with my second vCenter.

    Two vCenters have been upgraded today. It should be easier than this. Don't they test?!

    Reply

  2. William - It would be helpful to know what version your VCSA was at prior to the upgrade attempt to 7.0.2? I recently ran into this same issue when upgrading VCSA 7.0.1a to 7.0.1d. I'll refer readers to your Disclaimer above - STOP and call GSS if you're experiencing this on a production system! In my case, GSS had me follow the procedure in https://kb.vmware.com/s/article/67179 to remove the software_update_state.conf file, then mount the full patch -fp.iso to my VCSA vm appliance, then staging the packages and upgrading in two separate steps via CLI as described here: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vcenter.upgrade.doc/GUID-5FCA78EC-8637-43A4-8B28-24624E4D5EBA.html . One thing not mentioned is that you will get prompted to enter SSO credentials during the upgrade process, but the CLI upgrade went smoothly. GSS indicated to me that this was caused a bug in the source VCSA bits of my previous 7.0.1a install, and that this bug was resolved in the newer versions - so future upgrades from VAMI GUI should work as normal. I have NOT tried this 7.0.2 upgrade yet in that environment.

    Reply

      • My apologies - my eyes skimmed right over it. Reading Is Fundamental! Now I need to test from 7.0.1d and see if I run into the same issue 😉

        Reply

  4. I've tried the workaround to update from 7.0 Update 1d. The update ends with the following error: Mismatch:
    summary: PostgreSQL failed to run upgrade command. So even it's my lab, I decide to wait for an update of vCSA 7.0 Update2. 🙂

    Reply

  5. Hit this exact issue twice on the last two times I patched the VCSA in our lab.
    The same workaround (patch ISO) worked each time, and appears to be working this time.

    Reply

  6. I wanted to update my lab yesterday to 7.0 U2. from 7.0 U1a

    First I tried via VAMI and got the same error like above.
    Removing the file etc/applmgmt/appliance/software_update_state.conf did not resolve the issue.

    But also via CLI I could not update, if I use the --ulr switch I got:
    "Answers for following questions have to be provided to install phase:
    Question:
    ID: vmdir.password
    Text: Single Sign-On administrator password
    Description: For the first instance of the identity domain, this is the password given to the Administrator account. Otherwise, this is the password of the Administrator account of the replication partner.
    Allowed values:
    Default value:
    "
    I do not get a prompt to enter something.
    (Command> software-packages stage --url https://vapp-updates.vmware.com/vai-catalog/valm/vmw/8d167796-34d5-4899-be0a-6daade4005a3/7.0.1.00100.latest/ --acceptEulas
    )

    Then I downloaded the 7.0 U2 iso and mounted it to VCSA and run the update via CLI with the --iso switch, I got:
    "ISO unmounted successfully
    CD drives do not have valid patch iso."

    Reply

  8. There has been a running bug in 7.0 since GA just in a slightly different form. If you install any updates using the CLI then VAMI will no longer install updates at all failing with errors or not showing any updates available.

    I had cases open and was told this would be fixed in U2 sounds like fixes for it may have broken something else.

    Reply

  10. Thank you William, the workaround worked as a charm. In my case I stumbled on some other issues like:

    "Error in method invocation {'id': 'com.vmware.appliance.update.checksum_verification_failed', 'default_message': 'Checksum verification failed', 'args': []}"; so changing the URL at update Section overcomes this one: https://10.15.9.4:5480/#/ui/update/availableUpdates
    https://vapp-updates.vmware.com/vai-catalog/valm/vmw/8d167796-34d5-4899-be0a-6daade4005a3/7.0.1.00100.latest/

    Afterwards, once staged and prechecked/validated the Install process starts to never end. After an snapshot rollback the only way to resolve it so far is to follow on the posted solution at the article.

    Reply

  12. Get the same issue but still no luck with your instruction, always failed in the "copying software packages" stage when it comes to 66/67 package out of 150.

    Reply

  13. Had an odd experience with this issue just now. I got the error as everyone else here. Then started to google the issue, and of course stumbed upon this post.

    Then after a while, without doing anything,i noticed i got tossed out into the login screen of the VCSA, and when trying to log in again, it failed.
    I then tried opening vcenter normally, and got a message about upstream issue.

    This reminded me a bit about what happens while an update is going on, so i let it rest for a while, and god different errors when trying to enter vcenter.

    Left it for about an hour, and when i tried again, it was updated! So even though i got the error, and didn't try any of the workarounds, it ended up updating to Update 2 after all!

    Reply

  14. I tried a couple times using the Management site Update (including separately pre-checking, staging, installing... precheck always passed, staging always succeeded but install always hung up and I had to recover by deleting the update state conf file). Now I'm using the cli and software-packages command, it had a couple separate steps but I did get it to install successfully that way.

    Reply

  18. I've hit this issue 5-6 times already, had to reverse the order of the steps in the workaround.

    Delete that file first
    then pre-check (this was a required step for me)
    then stage
    then install

    the above worked okay for me

    Reply

  19. I used command line and iso
    Update went fine and after reboot
    vmdir.password is not set ; aborting installation

    SAD SAD
    any idea for this one ?

    Reply

  20. Had the same issues on two separate vcenters. I had to restart the VAMI because it was stuck in a failure loop, and changing the address to get out of the loop worked but the staging option would fail.

    After restarting the VAMI the staging option worked on both servers, and then was able to procceed with the workaround.

    Thank You!

    Reply

  22. I had a similar problem when upgrading my test installation from 7.0.3.00000 (7.0U3) to 7.0.3.00100 (7.0U3a).
    I tried all the root shell tricks mentioned above regarding software_update_state.conf, result was always: "Stage path file doesn't exist".
    This command helped:
    software-packages unstage

    After that, this worked:
    software-packages stage --iso --acceptEula

    and thus this as well:
    software-packages install --staged

    Reply

  24. Seems like this is an on going issue that vmware has not fixed yet.
    I faced the issue in my upgrade from 7.0 U1 to U2 and today again from 7.0U2b to 7.0U2d

    the fix is as below.

    At around 75 % of the install ( as shown in the GUI), keep checking the permission of the below file.

    It is supposed to look like this:
    root@HRC-1C-VC01 [ /usr/lib ]# ls -al /usr/lib/libsas*
    -rwxr-xr-x 1 root root 112240 May 12 2020 /usr/lib/libsasl2.so.3.0.0

    At some point (75 % of the installer) , the permission will change as below:
    root@HRC-1C-VC01 [ /usr/lib ]# ls -al /usr/lib/libsas*
    -rw-r----- 1 root root 112240 May 12 2020 /usr/lib/libsasl2.so.3.0.0

    Immediately run the below command to correct the permission:
    chmod 755 /usr/lib/libsasl2.so.3.0.0

    Let the installer run.

    This worked for me on 2 different versions.

    Reply

  26. I tried to run the command "software-packages install --url --acceptEulas" but I received the following error:

    -bash: software-packages: command not found

    Reply

  27. I know this is an old thread, however this may help other searching for the same fix.
    I am patching a vCenter 7.0u3h appliance to 7.0u3p and keep getting the error "Installation failed due to an exception.". I installed JExplorer and connected to the vCenter server and found 2 issues:
    1. An old PSC was still listed under Domain Controllers.
    2. A non-existent site was listed under Configuration \ Sites
    I deleted both of these and the vCenter can now patch to 7.0u3p.

    Reply

Thanks for the comment!

This site uses Akismet to reduce spam. Learn how your comment data is processed.