VMware recently published a new knowledge base (KB) article 52085 that outlines instructions for enabling the Hypervisor-Assisted Guest Mitigation (CVE-2017-5715), also known as the Spectre vulnerability. This KB also provides steps to verify the updated microcode (included in the ESXi patch) has been applied along with Virtual Machine verification (those applicable) to ensure that they are seeing the new CPU features. While following the KB and patching one of my vSphere environments, I had noticed the verification steps was not only manual but it also to difficult to scale beyond a few VMs as it required customers to look for a specific set of strings from within the vmware.log file which is generated for each powered on VM, which could easily be several hundreds or thousands of VMs.
I figured there had to be a better way to help customers automate this at scale and remove the human factor. The other reason I was not fond of the current method is that it would require customers to either enable ESXi Shell/SSH access or to manually or through automation to download every single vmware.log file to inspect for specific log entries which can take quite a bit of time for any sizable environment. I had an idea on how this could be done without having to look at the vmware.log file while leveraging our vSphere APIs and did some investigation. Before proceeding further, please familiarize yourself with KB 52085. For complete background on both Spectre (CVE-2017-5753 & CVE-2017-5715) and Meltdown (CVE-2017-5754) as it relates to all VMware products, please carefully read through this top level KB 52245 which is being updated as new information is available, so you may want to subscribe to the KB for all the latest updates.
UPDATE 4 (01/23/18) - VMware has just updated KB 52345 with the current list of Intel CPUs affected by Intel Sightings. I have also updated my script to reflect all these changes. Make sure to download the latest version to ensure you have the latest changes.
UPDATE 3 (01/16/18) - I have just enhanced the script further to also collect the current microcode version running on a given ESXi host. Unfortunately, this information can only be collected when SSH is enabled and is something a user must explicitly allow. The benefit here is that Intel Sighting impact reporting is more robust as the code now checks for both impacted CPU signature as well as the microcode affected by Intel Sighting as outline in KB 52345. See below for more details.
UPDATE 2 (01/14/18) - As mentioned in the last update, I have been working on a Intel Sighting remediation script which can help customers automate the temporary workaround as recommended in KB 52345. Please see this blog post for complete details.
UPDATE 1 (01/13/18) - VMware just published a new KB 52345 outlining certain Intel Broadwell and Haswell CPUs being affected by Intel Sightings after applying latest microcode updates. Please refer to the KB for the complete details. I am currently working on a script to help with the remediation as the remediation method outlined in the KB would require customers to enable SSH access and manually update /etc/vmware/config. In the meantime, I wanted to provide a way for customers to easily identify whether their system are affected by Intel Sightings and thanks to community member Adam Robinson who just submitted a patch this morning to update my existing script to include these details. I have also added the CPU model to the output as additional information.
Note: This script only provides information about your existing vSphere environment, it does not make any changes or provides any remediation steps, please follow the KB for those instructions.
The PowerCLI script is called VerifyESXiMicrocodePatch.ps1 and it contains the following two functions:
- Verify-ESXiMicrocodePatchAndVM
- Verify-ESXiMicrocodePatch
The first function verifies both ESXi hosts and VMs and provides the output from the VM's point of view. The script will ensure VMs are at least vHW9+ (as older vHW are not applicable), powered on and that one of the three new CPU features are available as outlined in the KB. If the HypervisorAssistedGuestAffected column for a given VM row shows false, it means BOTH the microcode + ESXi patch has been applied and that the VM has gone through a complete power cycle to see the new CPU features. The second function only verifies that the ESXi microcode has been applied (this could have come a hardware vendor BIOS/Firmware update) but as mentioned earlier, it is also bundled within the ESXi patch. Similar to the first function, it also verifies that one of the three new CPU features are exposed to the ESXi host and you will also see an HypervisorAssistedGuestAffected column for each ESXi host. For columns which show true, please refer back to the KB above and follow the remediation procedure. In addition to information related to KB 52085, the second function has also been recently enhanced (see Update #1, 2 & 3 above) which include details about Intel Sighting. You can find more details below.
The Verify-ESXiMicrocodePatchAndVM function can run in one of three ways. If you do not pass in any parameters, then it will query all VMs for a given vSphere environment whether that is connecting to a vCenter Server or directly to a specific ESXi host. You have the ability to limit the scope by providing a specific vSphere Cluster by using the -ClusterName parameter or you can check a particular VM by using the -VMName parameter. Below is a screenshot exercising the three different methods:
The Verify-ESXiMicrocodePatch function can also be run in one of three ways. If you do not pass in any parameters, then it will query all ESXi hosts for a given vSphere environment when connected to a vCenter Server or a specific ESXi host if you are directly connected to a host. You have the ability to limit the scope by providing a specific vSphere Cluster by using the -ClusterName parameter or you can check a particular ESXi host by using the -VMhostName parameter. Below is a screenshot exercising the three different methods:
As mentioned earlier, the second function also contains additional information about the recent Intel Sighting issue which is represented by the IntelSighting column as seen in the screenshot above.
The value of IntelSighting can contain four potential values:
- True = ESXi host contains microcode update is affected by Intel Sighting, you will need to apply the workaround as outlined in KB 52345
- AffectedOncePatched = CPU is affected by Intel Sighting, but does not need the work around unless it has been patched or has a BIOS update that includes the fault microcode
- False = CPU not affected by Intel Sighting it is currently recommended to only apply one of the ESXi patches (until Intel provides a microcode update fix), please refer to KB 52345 for full details
- N/A = CPU is not Intel
The "default" method in how the script identifies whether an ESXi host is affected by IntelSighting is by comparing against the known CPU signatures as outlined in KB 52345 which can be done so using the vSphere API. Although this method is sufficient at this time since no updated microcodes have not been released (as far as I know), the results may be incorrect when new microcodes have been released and applied to remediate against Intel Sighting. A more robust solution would be to verify BOTH the affected CPU signature and the microcode version that are known to be impacted by Intel Sighting which is also available in the KB. The main issue from an Automation standpoint is that to be able to retrieve the microcode version that is currently deployed on an ESXi host, you would need to either enable ESXi Shell and/or SSH to system and use the vsish interface as this information is currently not available through a remote vSphere API.
Here is the vsish command you can manually run to retrieve the current microcode version:
vsish -e cat /hardware/cpu/cpuList/0 | grep "Current Revision:"
Since information can not be retrieved through a remote API, I have decided to enhance the function for those customers who wish to collect this information and have SSH enabled (can just be for the duration of running the script). To do so, I have introduced four optional parameters that must be provided if you wish to retrieve the microcode version:
- IncludeMicrocodeVerCheck - By default, this is set to $false, you will need to set to $true
- PlinkPath - You will need to provide the full path to the Plink executable which I use as an SSH client to perform the remote operation
- ESXiUsername - The username to login to ESXi host and perform the operation, this would normally be root
- ESXiPassword - The password for the ESXi username provided
Note: If you enable microcode version retrieval and SSH is disabled on the ESXi host, then you will see the microcode column have a value of SSHNeedsToBeEnabled. You can easily mass enable/disable SSH access for your ESXi hosts using PowerCLI. Here is an example blog post which shows you how.
Here is a screenshot of an ESXi host which is impacted by IntelSighting which the script has been able to determine by checking BOTH the CPU signature and the microcode version (which is now available as part of the output) as seen below.
Excellent. Was waiting for some kind of VM-verification tool/script!
I ran the updated version of the script and now my fully patched 6.5 U1 hosts are showing as affected. I thought the CPU microcode was part of the ESXi patch. What am I missing? HP ProLiant BL460c Gen8, Intel Xeon E5-2600
Could the ESXi patch not have the microcode update for my specific CPU? Hmm...
HPE hasn't released a new BIOS for that machine yet, but they will in the future. I suspect that means the microcode doesn't exist yet.
https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-a00039267en_us
FYI - There's a bug in the Verify-ESXiMicrocodePatch function with the incorrect variable $STIBPPass/$STIBPass, so you may get inconsistent results.
Yup. Someone just sent me pull request and its been fixed.
Yeah, that was me 🙂 Thanks for the script, saved us a lot of time and effort!
Thank you! Can you explain the difference between the Hypervisor only patch (201711101-sg) and the Hypervisor-Assisted Guest Mitigation patch? Should you be installing both or one or the other?
Please refer to the KB for specific guidance
Will,
I need your help. Why the script is not working for me when I try to run it for an esxi or even through vcenter?
I get the following:
C:\Users\irving\Desktop> .\Verify-ESXiMicrocodePatchAndVM.ps1 Verify-ESXiMicrocodePatch
C:\Users\irving\Desktop>
It returns nothing.
same all here, dosent do anything... news from your end?
no news here.. i am just getting ignored left and right...
I'm a Powershell noob as well, but try this:
Import-Module Verify-ESXiMicrocodePatchAndVM.ps1
Verify-ESXiMicrocodePatch
Marc is correct, but just to correct the filenames used here:
Import-Module .\VerifyESXiMicrocodePatch.ps1
Then use either:
Verify-ESXiMicrocodePatchAndVM
Verify-ESXiMicrocodePatch
Another way to run the script, which is nothing more than functions, is to simply "sourcing" the script by running: . ./VerifyESXiMicrocodePatch.ps1
Great, this will help me a lot.. thanks as always
Thanks William!
One thing I came across is that Powered Off VMs are listed as affected, since Runtime.FeatureRequirement returns null for them.
I added Runtime.PowerState to the get-view and a few other lines to note them.
Yea, this was something that had crossed my mind as well for handling poweredOff VMs ... I could filter for only poweredOn VMs to avoid this and even VMs suspended. I may update the script to add that logic
I've just updated the script to now only process powered on VMs. Thanks for the suggestion
YOU ARE A F'KIN SAVIOR!! I MEAN THAT WITH EVERYTHING!! THANK YOU SOO MUCH. I am a noob in powershell i will admit it but you have saved me a ton of work and I appreciate it!
Something seems off. I updated my Cisco UCS B200 M3 and M4 blades with the latest VMware patches. When manually checking the hosts based on the VMware KB, I do not see that the M3 blades received a microcode update, as the values are all 0, but the M4 blades do update
VM with hardware v10 on M3 after patches
2018-01-11T22:13:38.039Z| vmx| I120: hostCPUID level 00000007, 0: 0x00000000 0x00000000 0x00000000 0x00000000
When I run your powershell script to check just the hosts in this cluster, it shows the M4 blades that have been patched as accurately not being affected, and the M3 blade that has not been patched yet it shows as affected, but on the M3 that has received all of the patches, it shows all 3 as present and affected as false which the line in the vmware.log of a VM doesn't support. The CPU in this host shows up as:
Intel(R) Xeon(R) CPU E5-2665 0 @ 2.40GH
It appears that after the patches are installed, then the host includes the featurecapability for cpuid.IBRS, cpuid.IBPB, and cpuid.STIBP, but there isn't any verifying that the value is set to 1 in this table.
I think this line in Verify-ESXiMicrocodePatch:
$cpuFeatures = $vmhost.Config.FeatureCapability
Should be replaced with
$cpufeatures = $vmhost.Config.FeatureCapability | where {$_.value -eq 1}
A similar change needed for the VM check.
I just hit this myself, and I submitted a pull request with the fix 🙂
Thanks. I'm not that familiar with Git. I did an edit to update my proposed change, but not sure if I did it correctly.
The $vm.Runtime.FeatureRequirement command appears to only include the features that are enabled by default, so no change needed there.
I've taken the pull request, please download the new version and give it a try now
I applied esxi6.0 ESXi600-201801401-BG & ESXi600-201801402-BG patches to few of my Hosts and when i ran this script, only for few host it showed cpuid.IBRS, cpuid.IBPB, and cpuid.STIBP as "true" and other as "false". these are Gen8 360. any idea why there am seeing different values?
Intel has probably not released new microcode for all of your CPU's yet. See https://www.bleepingcomputer.com/news/security/intel-releases-linux-cpu-microcodes-to-fix-meltdown-and-spectre-bugs/ for a list of CPU's that have been patched already.
Just submitted a pull request. Verify-ESXiMicrocodePatch is only looking for the cpuid key names and not the values of them.
This resulted in my hosts that do not have microcode updates showing that they had the new instructions when they in-fact do not.
Adam,
Thanks for the pull request. I thought it was sufficient to just check for the CPU feature but didn't realize the value needed to be checked. I just approved and hopefully its showing the correct results now
For Verify-ESXiMicrocodePatchAndVM with -ClusterName argument, I get this error message:
Get-View : Cannot validate argument on parameter 'VIObject'. The argument is null, empty, or an element of the argument collection contains a null value. Supply a collection that does not contain any null values and then
try the command again.
At *****
+ $vms = Get-View ((Get-View $cluster.ResourcePool).VM)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Get-View], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInterop.GetVIView
Any ideas how to fix?
You probably miss-typed the name of your vSphere Cluster, make sure you spell it correctly and if there's spaces, use a single tick mark between the string
Simply replace:
$cluster = Get-View -ViewType ClusterComputeResource -Property Name,ResourcePool -Filter @{"name"=$ClusterName}
$vms = Get-View ((Get-View $cluster.ResourcePool).VM) -Property Name,Config.Version,Runtime.PowerState,Runtime.FeatureRequirement
with the following code:
$cluster = get-view -ViewType ClusterComputeResource -Property Name -Filter @{"Name" = $ClusterName } | select -ExpandProperty MoRef
$vms = Get-View -ViewType VirtualMachine -Property Name,Config.Version,Runtime.PowerState,Runtime.FeatureRequirement -SearchRoot $Cluster
@William Lam: Hi William, on my production cluster (6.5) the script works flawlessly, on my Home Lab (6.7) I also get the error reported by @Yuri:
Get-View : Cannot validate argument on parameter 'VIObject'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.
VerifyESXiMicrocodePatch.ps1:32 char:25
I have verified the vSphere Cluster and it is entered correctly. I tried both with hostname and IP. Neither one worked for me.
@ujan, this does not work either outputting the following error:
Get-View : Cannot convert 'System.Object[]' to the type 'VMware.Vim.ManagedObjectReference' required by parameter 'SearchRoot'. Specified method is not supported.
At VerifyESXiMicrocodePatch.ps1:34 char:138
The script will ensure VMs are at least vHW9+ (as older vHW are not applicable) and that one of the three new CPU features are available as outlined by the KB.
Wanted to confirm this, is it true that vms with hw version 9 and above are the only ones affected by CVE-2017-5715, anything below are not affected by CVE-2017-5715 ?
vm's with a hw version lower than 9 ARE affected, you have to update the vHW to at least vHW9 in order to be able to patch them.
We ran this script for 12 ESXi which are at 6.0 build 7504637 (patched with ESXi600-201801401-BG & ESXi600-201801402-BG ).
when I ran this report, I saw that few host reported with IBRPresent, IBPBPresent, STIBPresent & Affected as "True" and few came with "False". All the Host are same H/w version (gen8, with no h/w patch for spectra) and are single host in Cluster. any reason am seeing this mismatch.
I get a bunch of error when trying to run the script. I am using vm powercli and connect to vcenter but as soon as i run the script i get a bunch of invalid char lines is this script bad?
Do note of the latest updated KB on for certain processor to use the older patches https://kb.vmware.com/s/article/52345.
can someone help me out why i cant run this script? i get a bunch of err char codes when running it.
this is a function, you need to add this to your profile and then run.
How do i add it to my profile?
Refer link:
https://www.jonathanmedd.net/2015/01/how-to-make-use-of-functions-in-powershell.html
did this but the script still doesnt work
You're using PowerCLI and have connected to your VI-Server right?
Yes Sir I am using VMware PowerCLI and have connected to my vcenter server through it using the Connect-VIserver Command
Paste here the command you are using after you open powercli And the errors.
Security warning
Run only scripts that you trust. While scripts from the internet can be useful,
this script can potentially harm your computer. If you trust this script, use
the Unblock-File cmdlet to allow the script to run without this warning
message. Do you want to run
C:\Users\isantos\Desktop\VerifyESXiMicrocodePatch.ps1?
[D] Do not run [R] Run once [S] Suspend [?] Help (default is "D"): R
At C:\Users\isantos\Desktop\VerifyESXiMicrocodePatch.ps1:129 char:19
+
+ ~
The '<' operator is reserved for future use.
At C:\Users\isantos\Desktop\VerifyESXiMicrocodePatch.ps1:133 char:19
+
+ ~
The '<' operator is reserved for future use.
At C:\Users\isantos\Desktop\VerifyESXiMicrocodePatch.ps1:1326 char:42
+ ... 2018 GitHubGitHub' in expression or statement.
Not all parse errors were reported. Correct the reported errors and try again.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : RedirectionNotSupported
Sorry I had to send the error in chops because this site doesnt allow me to paste it all.
Ok so i got it to go but when i enter the command Verify-ESXiMicrocodePatchAndVM i dont get a response back. its not spitting any information back into the cli. am i missing something?
When I run verify-esximicrocodepatch -clustername I do not see the column intelsighting.
This is what I see:
VMHost CPU IBRPresent IBPBPresent STIBPresent HypervisorAssistedGuestAffec
ted
------ --- ---------- ----------- ----------- ----------------------------
192.168.97.10 Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz True True True False
192.168.97.16 Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz True True True False
192.168.97.30 Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz True True True False
192.168.97.34 Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz True True True False
192.168.97.40 Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz True True True False
192.168.97.42 Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz True True True False
192.168.97.43 Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz True True True False
Somehow it now works showing intelsighting...
Thanks for the great info and script. Much appreciated!!
Anyone please help me. I do not get any response back when i run the script. I am doing everything the right way. but I dont know why i am not getting a response! Please help!
how are you getting the text to format correctly? when I run the function I just get line after line, it's not in columns
Maybe you can add the server vendor + model with bios version to quickly check for new BIOS updates. Something like: Dell Inc. PowerEdge R620;2.5.4
It should be noted that Intel came out today, and now recommends users and vendors do not retract their microcode.
See this:
https://security-center.intel.com/advisory.aspx?intelid=INTEL-SA-00088&languageid=en-fr
And the section under January 17:
Intel recommends that these partners maintain availability of existing microcode updates already released to end users. Intel does not recommend pulling back any updates already made available to end users
NEW - Intel recommends that these partners, at their discretion, continue development and release of updates with existing microcode to provide protection against these exploits, understanding that the current versions may introduce issues such as reboot in some configurations
NEW - We further recommend that OEMs, Cloud service providers, system manufacturers and software vendors begin evaluation of Intel beta microcode update releases in anticipation of definitive root cause and subsequent production releases suitable for end users
Jeff -From the same 1/18 statement:
"For PCs and Data Center infrastructure, Intel recommends that patches be applied as soon as they are available from your system manufacturer, and software vendors
For data center infrastructure, Intel additionally recommends that IT administrators evaluate potential impacts from the reboot issue and make decisions based on the security profile of the infrastructure"
So as an IT/VMware administrator, how can Intel reasonably ask me to "evaluate potential impacts from the reboot issue and make decisions" when there is NO public data describing the precise reboot findings, as best I can tell? For example, the phrase "in some configurations" appears three times in that statement without any indication of which specific configurations fit the reboot risk profile. Unless of course it is as I suspect to eventually be revealed - that the answer to that question is "all configurations using the specified processors which run User mode programs".
Can anyone point out a good technical resource with specific details of the Intel Sightings reboot issue? Specifically - the "some configurations" under which these reboots have been observed?
Folks - VMware is aware is of Intel's recent statement and https://kb.vmware.com/s/article/52345 **is** fully update to date, please follow the current guidance. If there are required, they will be made and if you wish to get notified of any changes, please subscribe to the KB article.
William - I a a bit confused by your definition of the value "AffectedOncePatched = CPU is affected by Intel Sighting, but does not , need the work around unless it is patched or has a BIOS update." If my understanding of this issue is correct, a more complete definition might be "CPU is affected by Intel Sighting, but the hypervisor does not need this work around unless it has been patched to include the now-recalled patch levels specified in VMSA-2018-0004.2, regardless of whether or not the underlying host BIOS has also been updated with a version containing the new faulty microcode."
I am thinking specifically of the case where a customer may have updated the host BIOS from their vendor but has NOT yet applied the now-withdrawn ESXi patches. In this case even guest vm's to which mitigating OS patches have been applied (and have also been subsequently powercycled) will NOT be able to use those new CPU instructions IBRSPresent, IBPBPresent, and STIBPPresent (and thus will not be at risk of reboot per Intel Sightings) because those new instructions are not present in the vCPU presented to those guests. Those new instructions will not be exposed to guest vCPUs for use by guest OS's until the ESXi build has been patched with builds newer than those specified in VMSA-2018-0002.2 AND that guest has subsequently power-cycled to see the newly-exposed instructions in the vCPU..
That is a long-winded way of simply asking -is an ESXi host or guest vm at risk of Intel Sightings reboots if It has a patched host BIOS and patched guest vm OS (including power cycle) but HAS NOT patched ESXi with anything newer than the VMSA-2018-0002.2 recommended versions?
Not exactly. The ESXi patches isn't the issue but rather the fault microcode that was provided by Intel which is bundled with the ESXi patch. This faulty microcode could have also been delivered as part of a BIOS update from your HW vendors or through another means. The "AffectedOncePatched" is merely stating that because the CPU signature was identified to be on the affected list AND it currently does not have the faulty microcode, that if you did happen to pull the patch before VMware recalled it, then you would be affected by Intel Sighting, if not, you are fine for now and you can follow the rest of the existing guidance
Hi,
If hosts were patched using VMSA-2018-0004.2 before the patches were withdrawn and they're not affected by sighting (E5-2600 v2), shouldn't 'Verify-ESXiMicrocodePatch' show True for the new CPU features (IBRS,STIBP,IBPB)?
Thanks for the explanation.
Hi After executing the script, i dont see Intel sighting column on the output:
Verify-ESXiMicrocodePatch -VMHostName Hostname
VMHost CPU Microcode IBRSPresent IBPBPresent STIBPPresent HypervisorAssistedGuestAffect
ed
------ --- --------- ----------- ----------- ------------ -----------------------------
Hostname Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz N/A False False False True
Hi!
Thank you for this post. But I dont understand the values. Is this result bad?
CPU is not effected, but still get Guestaffected. I have read the KB, but I don´t understand
PowerCLI C:\Script> Verify-ESXiMicrocodePatch
VMHost CPU Microcode IBRSPresent IBPBPresent STIBPPresent HypervisorAssistedGuestAffected IntelSighting
------ --- --------- ----------- ----------- ------------ ------------------------------- --------
esx50 Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz N/A False False False True False
PowerCLI C:\Script> Verify-ESXiMicrocodePatchAndVM
VM IBRSPresent IBPBPresent STIBPPresent vHW HypervisorAssistedGuestAffected
-- ----------- ----------- ------------ --- -------------------------------
SERVER False False False vmx-09 True
Does the hypervisor-assisted-guest mitigation require that the guest operating system also include the latest security fixes that mitigates Spectre and Meltdown?
i must be missing something. I have applied ESXi550-201709101-SG and ESXi550-201801301-BG to my esxi hosts. I run this script and and everthing still comes back false. HypervisorGuestAffect is True. The build i show i am runing 5.5u3f at this time after the updates are applied.
Gidday William, I think that you may need to update your scripts now. I have updated a server to the new HPE Proliant 460c Gen9 updates, and it has updated the Microcode to 0x0b00002a, ( which is higher than 0x0B000025 for the Intel Xeon E5-2600-v4 ). I think that this will mean that you might need to check if the microcode is higher than the blacklisted, so that it reports that it has been update.
I think the script needs an update now that new microcode has been released, as the "IntelSighting" now may be a false positive
How to export the result to csv using this command Verify-ESXiMicrocodePatchAndVM
Verify-ESXiMicrocodePatchAndVM | Out-File -Filepath c:\temp\hostvm.txt
Thank you this resolve the problem.
Has anyone been in a situation where they see "True True True False False" for Verify-ESXiMicrocodePatch, but when they try Verify-ESXiMicrocodePatchandVM it shows all "False" for all the VMs? I tried powering off and powering on VMs... verified they have the windows patches and additional configuration registry setting..vmx version 11... Kind of running out of ideas
Hi John,
I have the same problem on ESX 6.7.
I verified that I have the microcode version listed in the vmware KB and have the same outputs.
I manually verified in the vmware.log of some VMs that the new CPU features are not listed.
Inside a VM I ran Get-SpeculationControlSettings and it showed the VM to be vulnarable.
I don't know what to do anymore.
VM hardware version is upgraded to 14.
Some additional infos:
I have a E3-1230 V2 processor.
[root@esxi01:/vmfs/volumes/514bd719-ce84ff86-d474-002590a96828/FreeNAS02] vsish -e cat /hardware/cpu/cpuList/0 | grep "Current Revision:"
Current Revision:0x0000001f
wich is the version suggested in KB52085 for my CPU.
I repeatedly restarted VMs and the host.
C:\> Verify-ESXiMicrocodePatch
VMHost CPU Model Name Family Model Stepping Microcode procSig IBRSPresent IBPBPresent STIBPPresent HypervisorAssistedGuestAffected Good Microcode IntelSighting
------ -------------- ------ ----- -------- --------- ------- ----------- ----------- ------------ ------------------------------- -------------- -------------
esxi01.benni.local Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz 6 58 9 N/A 0x000306A9 True True True False N/A N/A
C:\> Verify-ESXiMicrocodePatchAndVM
VM IBRSPresent IBPBPresent STIBPPresent vHW HypervisorAssistedGuestAffect
ed
-- ----------- ----------- ------------ --- -----------------------------
DC01 False False False vmx-14 True
DC02 False False False vmx-14 True
Hi John,
I resolved the issue at least for me.
I had to delete EVC masking entries from /etc/vmware/config as my host was a EVC member before.
Now it's working.
@Benjamin: great suggestion.. we did apply some masking entries in the past! I tried to reset one VM's CPUID mask to default just now but no improvement. I'm going to try raising the EVC compatibility mode for the cluster first, then failing that I'll try to disable EVC, and failing that I'll try directly editing /etc/vmware/config as you have done. If it works I'll reply to this comment. Any hint as to which specific lines you might have to remove? Cheers!
I raised the EVC compatibility mode to the highest level supported by my cluster and now the capabilities are showing up in the vmware.log files for VMs! There is a note that "Nehalem" generation exposes additional CPU features... and I'm guessing the level I was running at was insufficient.
I have updated our hosts, they show patched, but the VMs are not showing the entires that they are updated from the script output or even looking at the vmware.log file. The VMs have been power cycled, OS updated and they are at HW ver 11, any ideals?
Apart from Benjamin Koch suggestion I would add that the VMware cpu micro code patch released on the 20/03/2018 is not fully reliable in the sense that often it simply does not patch some hosts. Same type of hosts with same identical cpus with the same patch applied. Some get patched and some not. And William Lam script confirms. We have HP servers and by applying the HP firmware patch I fixed this inconsistency. I ran again Lam's script and hosts are now all compliant. At least that is my experience. I hope it helps someone.
Hi William, will you be updated for the latest 2 vulnerabilities? Microsoft has already updated the Windows Guest speculationcontrol powershell. Thanks!
Hi Jay,
Please see https://kb.vmware.com/s/article/56931 which includes a set of PowerShell functions to help enable the new Side-Channel Aware Scheduler as well as retrieving information (e.g. if its enabled/disabled)
I am revisiting the script in patching. I have my vendor firmware updated to the latest code of which spectre/meldown was fixed a few revisions ago. I have updated my vcsa and hosts to 6.0u3h but i still show the IntelSighting column as true. Before updating my hosts it said it would be "affected when patched" so once i patched it changed to true. That makes sense. If i have updated my hardware and vmware am i getting a false positive here? My guest vms show HypervisorAssitedGuestAffected is False after running all updates in the guest, powering down and powering back up. Am i missing something here. Also.....is there a place that explains what IBRSPresent, IBPBPresent and STRIBPresent columns mean. On my patched hosts those are showing as true.