Since this question has come up a few times this week, I thought it is worth a quick blog post on how to enable the new vGPU vMotion feature which is now available in latest vSphere 6.7 Update 1 release. If you try to vMotion a VM that has been configured with a vGPU, you see the following message stating vGPU hot migration is not enabled.
To enable vGPU vMotion, you just need to update the following vCenter Server Advanced Setting vgpu.hotmigrate.enabled to true using the vSphere UI. The change will go into effect immediately and you will now be able to vMotion a VM configured with vGPU. This setting is actually documented in the official vSphere documentation here, but from all the folks I spoke with, it looks like it never came up or it must have been missed.
In addition to vMotion support, you can also perform Storage vMotion & Cross vMotion (Compute & Storage) for vGPU enabled VMs. Make sure that both your vCenter Server and ESXi hosts have been upgraded to vSphere 6.7 Update 1 and that you have NVIDIA GRID hardware and VIB installed on ESXi host. For folks interested in learning more about the new vMotion features in vSphere 6.7 Update 1, be sure to check out the VMworld 2018 session What's New in vMotion Technical Deep Dive.
Lastly, for those that prefer to automate this configuration change, here is a quick PowerCLI snippet for enabling vGPU vMotion:
Get-AdvancedSetting -Entity $global:DefaultVIServer -Name vgpu.hotmigrate.enabled | Set-AdvancedSetting -Value $true -Confirm:$false
I have install the nv driver(410.68) on esxi host, but i still can't find the setting on advanced setting
As the article stated, the setting is a vCenter Server setting and NOT an ESXi host setting 🙂
Thanks for support me , It works!
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.68 Driver Version: 410.68 CUDA Version: N/A |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P4 On | 00000000:AF:00.0 Off | Off |
| N/A 32C P8 11W / 75W | 27MiB / 8191MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
I updated the driver to version 410.68, enabled the 'vgpu.hotmigrate.enabled' option, but I keep getting "A required migration feature is not supported on the "Source" host"
I have other non-vGPU vms in this cluster that work fine with vmotion, so it's not a network issue. I have M10's in all 5 of my hosts. Just checking to see if there's something else that I might be missing...
Thanks!
Sorry...meant to post as a new thread. Didn't mean to hijack yours.
I found the issue. I accidentally used the "security only" update profile when upgrading from 6.5 to 6.7 up1. I reapplied the correct profile and it's working now.
This is a great enhancement, cool tech! I wonder however, since I could not find it in the documentation: would it be possible to vMotion a VM with vGPU between ESXi hosting different GPU cards? (like migrate from an NVIDIA M6 to an NVIDIA P6).
It is not possible to migrate VMs between hosts with different GPU types
HI,
Can any one assist me to get drivers for vsphere 6.7 NVidia GP 104GL tesla p6