There was an excellent blog post from Duncan a couple of weeks back going over the new vMotion capabilities in vSphere 6.0 which includes: Cross vSwitch vMotion, Cross vCenter vMotion (xVC-vMotion) and Long Distance vMotion (LD-vMotion). If you have not checked out his article, I highly recommend you give it a read before proceeding further. After reading through Duncan's article, I noticed he had missed out on one additional vMotion capability which might not be obvious as the option is no where to be found in the vSphere Web Client UI. In fact, I was only aware of this additional capability after hearing about it from Engineering during the development of vSphere 6.
The additional vMotion capability actually extends the Cross vCenter Server vMotion (xVC-vMotion) workflow which allows an administrator to live migrate a running virtual machine between two vCenter Servers that are part of the same SSO Domain. By virtue of being in the same SSO Domain using the new Enhanced Linked Mode feature, both vCenter Servers will be visible in the vSphere Web Client and will be available to be selected either as a source or destination for a vMotion operation.
This extended Cross vCenter Server vMotion capability (unofficially I am calling it ExVC-vMotion) allows an administrator to live migrate a running virtual machine between two vCenter Servers which are NOT part of the same SSO Domain. How cool is that!? In my opinion, this is actually a pretty big deal because I think it truly removes any boundaries for a vSphere virtual machine and will open up an entire new class of mobility use cases that were never thought possible before. This will definitely make it interesting for customers who wish to migrate workloads from their on-premises datacenter into a completely different vSphere environment or even one that is hosted by a service provider or maybe even vCloud Air?
The ExVC-vMotion operation is currently only available today using the vSphere API, not because it is a private API but because there is no UI wizard for this operation. The reason the current xVC-vMotion is so seamless today is that both your source and destination vCenter Server is visible by being part of the same SSO Domain. If you have two completely different vCenter Servers which are not joined to the same SSO Domain or have completely different SSO Domains, then you will need to use the vSphere API to perform this operation.
All vMotion operations including vMotion without shared storage uses the vSphere API RelocateVM_Task() method. In vSphere 6.0, the method has been enhanced to accept a new property called ServiceLocator which provides a service endpoint to a vCenter Server where a VM can be migrated to. One important thing to note is that if you wish to migrate a VM between two vCenter Servers located in the same SSO Domain, there is an sslThumbprint property that is not required to be set. However, if the two vCenter Servers are NOT part of the same SSO Domain, then you need to set that property. In addition, if the VM is migrated to a different vCenter Server, additional properties such as the ESXi host, vSphere Cluster/Resource Pool and Datastore must be specified as part of the migration spec.
UPDATE (05/25/16) - I have re-factored and simplified my xVC-vMotion script which supports additional capabilities. Please have a look at the blog post here for more details.
To demonstrate this awesome ExVC-vMotion operation, I have created a simple PowerCLI script called run-cool-ExVC-vMotion.ps1 which accepts 12 command-line parameters which are described in more detail below:
| Variable | Description |
|---|---|
| sourceVC | The hostname or IP Address of the source vCenter Server |
| sourceVCUsername | The username to connect to source vCenter Server |
| sourceVCPassword | The password to connect to source vCenter Server |
| destVC | The hostname or IP Address of the destination vCenter Server |
| destVCUsername | The username to connect to the destination vCenter Server |
| destVCPassword | The password to connect to the destination vCenter Server |
| destVCThumbprint | The SSL Thumbprint (SHA1) of the destination vCenter Server (can be retrieved using either this or this) |
| datastorename | The destination vSphere Datastore where the VM will be migrated to |
| clustername | The destination vSphere Cluster where the VM will be migrated to |
| vmhostname | The destination vSphere ESXi host where the VM will be migrated to |
| vmnetworkname | The destination vSphere VM Portgroup where the VM will be migrated to |
| vmname | The name of the source VM to be migrated |
In my lab environment, I have configured two vCenter Server's which are part of two different SSO Domains as seen in the screenshot below:
I have tiny Linux VM (vMA) that I am using which I will be migrating from vcenter60-4 to vcenter60-5 which has a completely different datastore and VM portgroup (if you have stretched/extended L2, then the VM would stay online during this migration). I then execute the script using the following parameters based on my own environment and we can see the migration is kicking off:
.\run-cool-ExVC-vMotion.ps1 vcenter60-4.primp-industries.com *protected email* VMware1! vcenter60-5.primp-industries.com *protected email* VMware1! 82:D0:CF:B5:CC:EA:FE:AE:03:BE:E9:4B:AC:A2:B0:AB:2F:E3:87:49 vesxi60-8-local-storage NY-Cluster vesxi60-8.primp-industries.com NY-VM-Network vMA
One the migration has completed, if we now take a look at our vSphere Web Client, we can see the VM has now been migrated to the other vCenter Server.
I really hope to see the vSphere Web Client get enhanced to support this cool vMotion capability, but in the mean time you can easily perform this operation using the above PowerCLI script or any other scripting/programming language calling into the vSphere API. Set your VM free and let it migrate where your heart desires 🙂
excellent
Amazing explanation. One question though, does this mean that a VM which is not on an extended L2 network can live migrate between vCenter servers on same SSO domain?
Correct, if you don't have a stretched L2/overlay network, then of course your VM will not be available from network standpoint as the IP would have to be updated once the migration has completed on the destination site.
Thanks William!
Maybe I missed it, how does this deal with potential duplicate MACs going forward ?
vCenter 1 now thinks the MACs are free and may give them to another VM.
Troy,
Great question, actually this won't be a problem as there's a special MAC Address "blacklist" which is maintained by source VC. This list is updated with the VM's MAC Address(s) once it's been successfully migrated to another VC to ensures that the source VC will not hand out that MAC Address again
Thanks William! from Russia
Nice script, thanks for sharing it
But it does not work for me (using the RC). It ask for the VM Folder destination and stop the migration.
If you add this line in the # Source VM to migrate part:
$vmfolder = (Get-Folder -Server $destVCConn -Name "VM Folder Name...")
And then this line in the # Relocate Spec for Migration part:
$spec.folder = $vmfolder.Id
Then it works (at least for me)
Strange, this worked for me on a much newer build, perhaps this changed from RC. Definitely recommend giving this a try when we GA vSphere 6 🙂
I've tested only with the RC... For sure I will try it again on the GA 🙂
Very informative
Hi William, thanks for sharing.
By the way, any idea why it fails when machines are connected to vDS? and another one, on the script comments you say that its only looking for one network card, can you share the modifications for VM's with 2 or more vnics.
Thanks a lot for you help.
Regards
Paulo José
That's an exercise I've left for the reader 🙂 You can always post in the PowerCLI community forums if you're unable to figure it out but the script is provided as a sample
Problem solved, got help from a colleague of mine.
So our "enhanced" version of your script now migrates machines with more then one nic, and also migrates machines connected to vDS.
Thanks any way.
Regards
Anyway to make your enhanced script available? I am stuck on the moving vm's with vDS and have asked the PowerCli community if they can help.
Thank you
not a PowerCLI guru yet so I'd love to ge my hands on your version that works with dVS, Thanks in advance for sharing!
Add me to the list of needing the script that works with a VMs connected to a vDS. It would be greatly appreciated!
pjose77 - Any chance you'd be willing to share your code?
I was thinking of using this script to migrate to a new vSphere 6 environment. I was going to present the same datastores to both clusters and do an xVC-vMotion across. I would like to do a vMotion without moving the VMDK's. From the above notes its not clear if I could set the datastorename on the source and destination the same to achieve this?
It sounds like you're wanting to do just a vMotion (where the VMDKs don't change) but across two different vCenter Server 6.0 env? You could try specifying the source/destination datastore to be the same name, not sure if it'll work but worth a try.
That's right. I was able to set this up and it works a treat. Thanks
When we run this script in our environment, we are getting the following error:
Wait-Task : 7/29/2015 10:41:48 AM Wait-Task The operation for the
entity "KVTEST1" failed with the following message: "A specified parameter was
not correct: ServiceLocator.instanceUuid"
At c:\xvc_migrate.ps1:159 char:10
+ $task1 | Wait-Task -Verbose
+ ~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Wait-Task], InvalidArgument
+ FullyQualifiedErrorId : Client20_TaskServiceImpl_CheckServerSideTaskUpda
tes_OperationFailed,VMware.VimAutomation.ViCore.Cmdlets.Commands.WaitTask
I am getting the instanceUUID from the dest vcenter, so I don't think that that is the problem.
Any ideas?
We figured it out. Looks like the $service.instanceUuid needs to be uppercase.
Could you explain more clear, please? Which characters needs to be uppercase? I have the same error. 🙂
I know this is old, but in case anyone is curious what the fix was..
Line 101: $service.instanceUuid = $destVCConn.InstanceUuid
add .ToUpper() to the end.
so it'd be $service.instanceUuid = $destVCConn.InstanceUuid.ToUpper()
This got my script going.
Which constrains have Cross VC migration for data relocate DRS/HA and Tasks of VM? Located in the same SSO domain in VCs, maybe?
Hi William,
is it possible to run such vMotion if the Source VM runs on an ESXi 5.5 and vDS 5.5 connected to a 6.0 vCenter? Or must all be on version 6.0 (ESXi host and vDS)
do we have a full fledged working way of setting l3 vmotion on vsphere 6 yet? it s a pain to configure it manually on 100s of blades every day when you are a consultant.
Yes, L3 is fully supported. You can create a new Network Stack in ESXi which will allow you to use different networks to route out versus old method of adding static routes. Please refer to the vSphere documentation for more details
Hi William,
Awesome script, one thing though - it doesn't work with Distributed Switches.
I have a solution for anyone wanting to use Williams script, please see below.
1) Open the script and locate the line that reads "$Spec.DeviceChange[0].Device.Backing.DeviceName = $vmnetworkname"
2) Comment out that line, and add the following directly below it.
################################################################################################
$vm2 = Get-View -ViewType VirtualMachine -Filter @{"Name" = $VMname}
$vmhostView = Get-View -ID $vm2.Runtime.Host
$esx = Get-View -Id $vmview.Runtime.Host
foreach($netMoRef in $esx.Network){
if($netMoRef.Type -eq "DistributedVirtualPortGroup"){
$net = Get-View -Id $netMoRef
if($net.Name -eq $destdvPortGroup){
$dvPgKey = $net.MoRef.Value
$dvSwitchUuid = (Get-View -Id $net.Config.DistributedVirtualSwitch).Summary.Uuid
} # end if
} # end if
} # end foreach loop
$spec.deviceChange[0].device.backing = New-Object VMware.Vim.VirtualEthernetCardDistributedVirtualPortBackingInfo
$spec.deviceChange[0].device.backing.port = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection
$spec.deviceChange[0].device.backing.port.switchUuid = $dvSwitchUuid
$spec.deviceChange[0].device.backing.port.portgroupKey = $dvPgKey
################################################################################################
3) Scroll back up to the 'debugging' section of the script which is commented out, and add the following line, where $destdvPortGroup is the name of the destination port group you want to vMotion to.
################################################################################################
$destdvPortGroup = "DPG - VLAN 77"
################################################################################################
4) Done! It's a bit of a hardcoded change but it works. Someone with better scripting skills could easily update Williams script to dynamically tell if it should be a normal switch or a distributed switch.
Don't normally reply to blog posts but this took a little while to figure out and couldn't find it posted anywhere else! Hope it helps someone
Rumpole,
First of all, thank you for posting code to help with a vDS Switch. I am running into an issue with your additions.
Is this line correct?
$esx = Get-View -Id $vmview.Runtime.Host
I get the following error:
Get-View : Cannot validate argument on parameter 'Id'. The argument is null or empty. Supply an argument that is not null or empty and then try the command again.
At D:\Scripts\0 Automate\run-cool-ExVC-vMotion.ps1:176 char:21
+ $esx = Get-View -Id $vmview.Runtime.Host
+ ~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Get-View], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.DotNetInterop.GetVIView
Thanks
Fred
Hi Fred,
Did you ever get the answer to this or get this working? We are trying to accomplish the same thing.
Thanks!
Wes
Great posting guys thanks so much. Meanwhile I'm having a hard time finding information pertaining to non-federated cross vCenter vMotion. Mr Google knows very little. The only reference I'm finding points to the SDK and mentions using additional (unspecified) parameters. Can anyone offer suggestions. Our objective is to collapse 4 vCenters with 4 separate SSO domains into 2 vCenters with a single SSO domain running in enhanced linked mode.
Ray,
I'm a bit confused about your question, did you go through the blog article? What you're asking for is exactly what I've blogged about which is be able to migrate a VM across two vCenter Server 6.0 that are not part of the same SSO Domain or as you called it "non-federated"
Cross vCenter vmotion where venter do not belong to same SSO domain, maybe I'm using the term non federated incorrectly.
Yes, that's what the article is about 🙂 Please have another look
Hi Lam,
I am getting the error below.
Exception calling "RelocateVM_Task" with "2" argument(s): "
Required property instanceUuid is missing from data object of type ServiceLocat
or
while parsing serialized DataObject of type vim.ServiceLocator
at line 1, column 301
while parsing property "service" of static type ServiceLocator
while parsing serialized DataObject of type vim.vm.RelocateSpec
at line 1, column 295
while parsing call information for method RelocateVM_Task
Are you specifying the instanceUUID property? That seems to be the error that's indicated ... If you are, can you try both all lower and all upper case, I remember someone was having an issue and was related to the "case" of this property
Any commentors willing to share their code to vMotion with Distributed Switches? I'm still struggling to get it to work.
Were you able to get it to work?
I have posted the the changed for DV Switches as new comment
Hi William,
I was trying to use your script to do a vMotion within the same SSO - Enhanced Linked Mode - but as I do not want to change datastores (stretched metro cluster with SRM 6.1) I don't think it is appropriate. I'm running into an issue where I am unable to vMotion any VM within the GUI that has vmdks in more than one datastore. A single datastore VM works fine across vCenters but once I add another datastore to the VM, it will only show me the local hosts in the vCenter. I've never heard of such a restriction so I was trying to script the move to see if I could get a warning message as to why it is not displaying the other hosts. Is there another method like MigrateVM_Task that allows cross-vCenter with the same SSO? Thanks.
Drew,
You're not forced to change the datastore 🙂 Although most customers who may have a use for xVC-vMotion would more than likely change the various compute/storage resources as they're usually not shared amongst VC. However, as I said this is not necessary required and what is possible is all documented in the vSphere API which you can see from the "spec" http://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.wssdk.apiref.doc%2Fvim.vm.RelocateSpec.html you do not need to relocate the disks. You should be able to just leave that blank, if it throws an error, I'm curious if you've tried just specifying the exact same datastore (e.g. no change).
MigrateVM_Task() is only for traditional vMotion which does not cross vCenter Server boundary, for all other types of migrations which included Storage vMotion/etc. you will need to use the RelocateVM_Task() as mentioned in the documentation
So this turned out to be 2 bugs in the VMware code concerning shared storage. The first is you cannot cross-vMotion with a VM that has multiple datastores. The second is you cannot cross-vMotion a VM if there is not 2x vmdk free space in the datastore - basically enough so that if you did a Storage vMotion it would succeed. VMware is working on bugs and hopefully a KB.
Hi Drew (and anyone that stumbles across this),
I submitted a pull request to update William's script (https://github.com/lamw/vghetto-scripts/pulls). I wrote in some logic to handle multi-nic VMs where the NICs are on different portgroups as well as VMDKs on different datastores. The 2x space requirement still exists (doh!) but I was able to move around 3k VMs between vCenters with the same shared storage. Hopefully others can make use of it too
-Alex
If you use a DVswitch on the target vcenter you have to change a little bit in your script, i have hardcoded my dvswitch name:
I have added in Line 134:
$destvds = Get-view (Get-VDSwitch -Server $destVCConn -name mydvswitch) -Property uuid
$destpg = Get-View (Get-VDPortgroup -name $vmnetworkname) -Property key
and changed line 165 from
$spec.deviceChange[0].Device.backing.NetworkName = $vmnetworkname
to
$spec.deviceChange[0].Device.backing = new-object VMware.Vim.VirtualEthernetCardDistributedVirtualPortBackingInfo
$spec.deviceChange[0].Device.backing.port = New-Object VMware.Vim.DistributedVirtualSwitchPortConnection
$spec.deviceChange[0].Device.backing.port.portgroupkey = $destpg.key
$spec.deviceChange[0].Device.backing.port.switchuuid = $destvds.uuid
Typo 🙁 Forgot the -Server $destVCConn Parameter
$destpg = Get-View (Get-VDPortgroup -Server $destVCConn -name $vmnetworkname) -Property key
Thank you very much, that works!
Also very interested in your "enhanced" version ! thanks
Has anyone been able to get multiple nics with different port groups working?
Hi William,
I added some logic to handle multi-nic setups with different port groups. Check out my pull request against William's script (https://github.com/thomsonac/vghetto-scripts/blob/master/powershell/xMove-VM.ps1) for the latest.
Still a handful of edge-case bugs but I moved ~3k VMs between vCenters with it.
Hi folks,
Given the interests for this script that includes both XvC-vMotion within the same SSO Domain (which the script does do today) as well as between different SSO Domain + variations between VSS/VDS and supporting multiple vNICs. I've actually re-written the script to easily support all these use cases. I'm doing some finalize testing but I will post an update which will be a new blog so that folks can easily find it as I know the title does not lend itself to Google searches 🙂
Appreciate all the feedback
Folks,
The new script and blog article with the details have been posted here http://www.virtuallyghetto.com/2016/05/automating-cross-vcenter-vmotion-xvc-vmotion-between-the-same-different-sso-domain.html
This is awesome to hear! Do you know this will work while migrating from a vCenter 5.1 installation to a completely separate vCenter 6 installation?
No, Inter-vCenter vMotion only works between two vCenter 6 with ESXi 6 Hosts connected.
Hi William! We had with migrating VMs between vCenters not in the same SSO domain. Will the instanceUUID change for that VM when moved to the new vCenter? We're thinking it will but thought you may be able to confirm.
Thanks!
Hi William, I just wanted to check back on this question. Is this something you could help answer? Thanks!
Very simplified video to understand vMotion in vSphere 6.0
Great script. we used it to move over a bunch of VMs. We noticed the migrations are going over the 1GB management interfaces. We have 10GB dedicated vmotion interfaces. Is there a way to force the migrations over the 10GB? Thanks in advance.