WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud
  • Tanzu
    • Application Modernization
    • Tanzu services
    • Tanzu Community Edition
    • Tanzu Kubernetes Grid
    • vSphere with Tanzu
  • Home Lab
  • Nested Virtualization
  • Apple
You are here: Home / ESXCLI / Important - NVMe SSD not found after upgrading to ESXi 7.0

Important - NVMe SSD not found after upgrading to ESXi 7.0

04.16.2020 by William Lam // 17 Comments

Several folks in the community had reported issues that after upgrading from ESXi 6.7 to 7.0, their Samsung NVMe PCIe SSDs were no longer showing up. The first report of this was from Ivo Beerens and eventually found that reinstalling ESXi from scratch works but certainly that was not ideal. Just yesterday, I saw that Jeffrey Kusters also shared a similiar issue and used a different workaround which allowed him to upgrade. I also reached out to VMware Engineering as they thought this was a strange behavior but needed to see the logs to understand what was actually going on. Since Jeffrey's setup was an upgrade, I was able to get a copy of his vm-support bundle to provide to Engineering.

Within minutes of looking at the support bundle, they quickly identified the issue and this was caused by using the incorrect ESXCLI command to upgrade a standalone ESXi host from 6.7 to 7.0. Instead of using "esxcli software vib update" command, folks should be using "esxcli software profile update" which has always been the correct command to use when upgrading an ESXi image. In fact, this has been in the vSphere documentation for quite some time and here is the ESXi 7.0 version of that documentation. More importantly, the incorrect command only upgrades the ESXi 6.7 VIBs that exists and does not install any of the ESXi 7.0 VIBs, which means after the upgrade, you are not only missing the nvme-pcie VIB but many other ESXi 7.0 VIBs!

tl;dr - If you are going to use ESXCLI to upgrade your standalone ESXi host, please make sure to use the correct command or you will have issues. Below are the two commands you will need to determine which ESXi Image Profiles are available given an offline bundle and then updating to a specific image profile.

List Image Profiles from ESXi 7.0 Offline Bundle:

[[email protected]:~] esxcli software sources profile list -d /vmfs/volumes/e200-8d-local-datastore/VMware-ESXi-7.0.0-15843807-depot.zip
Name Vendor Acceptance Level Creation Time Modification Time
---------------------------- ------------ ---------------- ------------------- -------------------
ESXi-7.0.0-15843807-standard VMware, Inc. PartnerSupported 2020-03-16T10:48:54 2020-03-16T10:48:54
ESXi-7.0.0-15843807-no-tools VMware, Inc. PartnerSupported 2020-03-16T10:48:54 2020-03-16T10:48:54

Upgrade to a specific Image Profile from ESXi 7.0 Offline Bundle:

esxcli software profile update -d /vmfs/volumes/e200-8d-local-datastore/VMware-ESXi-7.0.0-15843807-depot.zip -p ESXi-7.0.0-15843807-standard

More from my site

  • ESXi Advanced & Kernel Settings Reference
  • ESXi with Intel Arc 750 / 770 GPU
  • How to recover ESXi installed on USB device after disabling vmkusb module?
  • Applying additional security hardening enhancements in ESXi 8.0
  • Video of ESXi install workaround for Fatal CPU mismatch on feature for Intel 12th Gen CPUs and newer

Categories // ESXCLI, ESXi, vSphere 7.0 Tags // esxcli, ESXi 7.0

Comments

  1. Steve Ballmers says

    04/16/2020 at 7:00 am

    Hello William,
    Great blog. However many folks have been using the "incorrect" command for years and it worked fine.
    You can also look at the KB below and the "incorrect" command is used in examples there as well

    https://kb.vmware.com/s/article/2008939

    Reply
    • William Lam says

      04/16/2020 at 7:33 am

      That KB talks about "patchig" ... not upgrading

      Reply
      • Mike Jones says

        04/16/2020 at 11:44 am

        You are correct. It gets very confusing as some people used that command to upgrade as well. It worked until now. Vmware should have one command and to confuse with upgrading vs updating. The updating command section is under upgrading esxcli documentation on the vmware docs website.

        Reply
  2. None says

    04/16/2020 at 3:54 pm

    Hello,
    Your update command is wrong, it should be "esxcli software profile update"

    Reply
    • William Lam says

      04/17/2020 at 4:59 am

      Thanks for the catch. Its been fixed

      Reply
  3. DM says

    04/21/2020 at 5:53 pm

    Hello William,
    I read your article about some NVMe not being recognized, and since I have 2 of them, I decided to try it out and installed ESXi 7.0 on my SYS-E300-9D-8C8NTP system using bootable USB instead of upgrade my existing 6.7.
    Onboard NVMe (Samsung) was recognized without problem. However, my ADATA XPG SX8200 Pro NVMe installed on 90-degree angled bracket inserted into PCIe slot - did not 🙁
    I got it installed few months ago on 6.7 and it was recognized immediately. but since this fresh install on USB, it didn't recognize it. I guess I won't upgrade my system for now. Any tips how to identify if there is a problem with this driver on 7.0?

    Reply
    • Алексей Кознов says

      05/05/2020 at 4:13 am

      Same thing here. Have you fix this or till now facing this issue with 8200?

      Reply
      • JV says

        05/19/2020 at 10:04 am

        Exact same problem over here too with same NVMe board.

        Reply
    • dave says

      05/25/2020 at 1:38 pm

      Just adding my two cents here as well. 6.7 works great for NVMe. New install of 7.0 doesn't who my 1 TB NVMe drive. Hopefully able to find some work arounds as I was still have to do the nvme.v00 driver copy to make 6.7 work.

      Reply
      • dave says

        05/25/2020 at 1:41 pm

        Actually - the same work around worked for me just now. However - the file names are different...

        mv nvme_pci.v00 nvme_pci.v00.old
        mv nvme.v00 nvme_pci.v00

        from the VMW_bootbank_nvme_1.2.1.34-1vmw.650.2.50.8294253 patch

        Reply
        • Jonathan A Colon says

          05/29/2020 at 6:56 am

          Can you explains more of this workaround?

          Reply
        • evan says

          06/12/2020 at 7:00 pm

          This worked great for me! It's a life saver! You might want to make note of the file rename above.

          Reply
          • evan says

            06/12/2020 at 7:03 pm

            I should say the author of the article should mention the file rename up above.

          • William Lam says

            06/13/2020 at 6:26 am

            Thanks for sharing the updated instructions Dave and the confirmation from Evan. However, this blog pos is really about incorrectly upgrading to ESXi 7.0. The details have been updated in i https://www.williamlam.com/2019/05/quick-tip-crucial-nvme-ssd-not-recognized-by-esxi-6-7.html which is the blog post that details about NVMe devices that used to work and copying 6.5 NVMe VIB

  4. Ballmer says

    04/27/2020 at 10:26 am

    Would these Cisco instructions be incorrect as well? They seem to go back and forth with update and upgrade between 5.5 to 6.x

    https://www.cisco.com/c/en/us/support/docs/unified-communications/unified-communications-manager-callmanager/213689-upgrade-vmware-esxi-from-version-5-5-to.html#anc6

    Reply
  5. Chris Busch says

    08/30/2021 at 2:50 pm

    The same seems to happen with ESXi-7.0U2c-18426014-standard (Build 18426014). I lost access to my Samsung 970 after upgrade. Offline bundle for U2c doesnt seem to be out. So for now I am stuck without my samsung...

    Reply
    • Chris Busch says

      08/30/2021 at 2:58 pm

      I was able to rollback with Shift-R during boot and the samsung 970 showed up again.

      Reply

Thanks for the comment! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search

Author

William Lam is a Senior Staff Solution Architect working in the VMware Cloud team within the Cloud Infrastructure Business Group (CIBG) at VMware. He focuses on Cloud Native technologies, Automation, Integration and Operation for the VMware Cloud based Software Defined Datacenters (SDDC)

Connect

  • Email
  • GitHub
  • LinkedIn
  • RSS
  • Twitter
  • Vimeo

Recent

  • vSphere with Tanzu using Intel Arc GPU 01/26/2023
  • Quick Tip - Automating allowed and not allowed Datastores for use with vSphere Cluster Services (vCLS) 01/25/2023
  • ESXi with Intel Arc 750 / 770 GPU 01/24/2023
  • How to bootstrap vSAN Express Storage Architecture (ESA) on unsupported hardware? 01/19/2023
  • Automating Virtual Machine screenshots in vSphere 01/18/2023

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2023