WilliamLam.com

You are here: Home / Automation / How to Copy VMs Directly Between ESXi Hosts Without Shared Storage

How to Copy VMs Directly Between ESXi Hosts Without Shared Storage

by // 93 Comments

There were some discussions earlier this week about copying Virtual Machines from one ESXi host to another ESXi host that reminded me of a very cool feature in the ovftool that could help with this task(which I thought I had written about already). As you probably have guessed by now, I am a big fan of the ovftool and have written several articles here, here and here. It still surprises me with the amount of features this little utility contains and this particular one is definitely a cool one!

If you have ever needed to copy a Virtual Machine from one host to another, it can be a challenge sometimes, especially if you do not have shared storage. You can of course leverage tools like VMware Converter or exporting the VM to a "middle man" system and then re-importing that VM into the destination host but it could take awhile or you have to run a Windows system. Well, if you are looking for a quick and easy way to copy a VM from one host to another, try using the ovftool.

In this example, I have two ESXi hosts called vESXi-03 and vESXi-04 and they both contain a single local datastore (no shared storage). I have a VM called vMA5 that is located on vESXi-03 and I would like to copy that directly to vESXi-04 without needing any additional storage.

Here is an example of using ovftool to probe the ESXi host to see the list of available VMs:

Note: A VM must be powered off for ovftool to transfered or exported.

Now that we have identified our VM, we just need to specify the source ESXi host and the destination ESXi host as well as the datastore using the -ds option. Here is an example of using ovftool to export the VM from one ESXi host to another ESXi host:

There are also other options that you can specify such as the network configuration and power options, please refer to the ovftool documentation for more details.

If you open up a vSphere Client connection to each of your ESXi hosts, you will see that the source host will have an export task and the destination host will have an import task as shown in the screenshot below:

Pretty nifty huh? 🙂

If anyone is interested in how this works, the system that is running ovftool acts as a proxy between your source and destination. The system running ovftool IS in the data path during the transfer but its only for the data to stream from source->client->destination. Nothing is stored on the client system and you do not need to have the storage capacity of what you are transferring. This is very nifty little feature that many people are not aware of with ovftool.

More from my site

Comments

  2. Thanks, nice tip to know!

    BTW, free edition of Veeam Backup and Replication tool can copy/migrate live VMs from one ESXi host/datastore to another.

    Reply

    • Yep, I recommended that in my blog as well for those wanting a bit easier of administration. http://wp.me/p14UMO-ba

      So if you are importing/exporting VM's using the OVF how does that stack up to using CP or the HTTP get/put from a performance standpoint? Obviously the OVF tool is made for virtual machine related functions so maybe the performance is better? Sounds like I will need to do some further testing...

      Reply

  3. I'm trying to do the same here, wanna move a VM called 'centos2' (it's powerd off) from 10.101.102.13 to 10.101.102.12

    [root@centos1 downloads]# ovftool vi://[email protected]
    Please enter login information for source vi://10.101.102.13/
    Username: root
    Password: *********
    Error: Found wrong kind of object (ResourcePool). Possible completions are:
    TSMSRV1
    TSMSRV2
    centos2
    centos3
    openfiler1
    vma

    [root@centos1 downloads]# ovftool -ds=ds_12 vi://[email protected]/centos2 vi://[email protected]
    Please enter login information for source vi://10.101.102.13/
    Username: root
    Password: *********
    Opening VI source: vi://[email protected]:443/centos2
    Error: Could not open URL https://10.101.102.13/ha-nfc/52c5119b-f877-91c7-dc66-0eadbf657559/disk-1 (response code: 500)
    Completed with errors

    Reply

  4. hi, nice info.
    if the OVF tool is on a vMA box does the traffic goes over the vMA box or directly host to host? Also any info on performance?

    Thanks
    Tilo

    Reply

    • Hi, the traffic goes from source host to OVFtool machine and then to destination host.

      - Steffen

      Reply

  6. Some nice tools, thanks. I've always stuck with just SSH to the ESXi server and perform a SCP on the files to the other ESXi server.

    This is a nice shortcut for people though, thanks for the find Mr Ghetto 🙂

    Reply

  7. Here some rounded numbers of my (not representative) tests:

    oopy with ovftool: ~ 80-100 Mbps

    backup with ghettoVCB.sh to NFS: ~ 500-700 Mbps
    restore with ghettoVCB-restore.sh from NFS: ~ 500-700 Mbps

    backup with ghettoVCB.sh to iSCSI: ~ 800-950 Mbps
    restore with ghettoVCB-restore.sh from iSCSI: ~ 800-950 Mbps

    So, if you have a shared storage it is faster to backup with ghettoVCB.sh and restore with ghettoVCB-restore.sh instead of useing ovftool like above - at least for larger VMs.

    - Steffen

    Reply

  8. Hello William

    For some reason,We have a vCenter 5.0 and would be replaced by several single ESXi 5.1 hosts. The VMs on vCenter 5.0 should be migrated and distributed to 6 ESXi hosts' local data-storage. And the snapshot along with VM is required to be kept during migration. I searched the some solutions as below. It seems all existing solution has some limitation so that can not meet our requirement well.

    P.S Because some reason, we are not allowed to add new ESXi 5.1 hosts to the existing vCenter 5.0 which would be retired.

    1, Veeam B&R, by using Quick Migration Feature
    A,data moving performance is very bad
    B, Snapshot can not be kept
    C, Do not support THIN HDD well. For example, a VM with 200GB provisioned sized and with 13GB actual size, B&R seems moving 200GB data during quick migration and caused about 90 mins.

    2, OVFtool, hard to implement because of we have more than 100 VM to be migrated. and it can not keep snapshot.

    3, vCenter Converter Standalone, hard to implement because of we have more than 100 VM to be migrated. and it can not keep snapshot.

    4, Download/Upload VMs' data in DataStore Browser interface, though it can keep snapshot, it's hard to implement for so much VMs

    Could please help me and get me any suggestion or better solution?

    thanks !!

    Reply

  9. Hey Will,

    Well, that was super embarrassing. I told a network dude to use the "middle-man" approach as you described in the article. He went off and found ovftool and copied over 200GB in 10 minutes when my suggestion took many hours (he actually cancelled it before it completed to find a faster way). So off the google-izer I go to find out about ovftool copying VMs. Thanks for sharing.

    - Mike

    http://VirtuallyMikeBrown.com
    http://twitter.com/VirtuallyMikeB

    Reply

  10. Hello William.

    Is this solution allow to copy VMs with snapshots? If not, is there any other solution? I guess Sean Lv wrote exactly about that.

    IB.

    Reply

  11. Great solution using it to copy VMs out of a VC to a standalone Host.

    @Anonymous from 22. May
    It should be possible to export a VM with snapshots to a OVF, but i would recommend to try and see what happens... At leas the source VM will be untouched.

    Reply

  14. GSCopy Pro v6.0 (RoboCopy Alternative) with Open File Agent
    GSCopyPro is a single command-line tool (CLI) that can copy, replicate and move files from one folder to another. This folder can be on the same machine/ server or another server elsewhere. What makes GSCopyPro stand out from other competitors is the fact it works on 32-bit as well as 64-bit systems and has no restrictions. It can easily be scheduled to run as a scheduled task and fully automated. GSCopyPro also comes with an open file agent which can copy files that are locked/ opened by other processes. This feature is supported in all windows vSCersions from widows XP/ 2003 and later.
    Go To:>> http://www.gurusquad.com/GOPYPRO

    Reply

  15. What I required was a space office that made me feel that my stuff might be secured. What I got was a clean and well-run office, that feeling that all is well with the world, additionally a cordial staff to make my comings and goings a charming background.
    self storage in Glen Waverley

    Reply

      • Here's my example for others to use for ESXi to a specific host and SAN datastore on vCenter server. Also mapping networks of the multihomed VM.

        ovftool -ds=data1 --net:"VM Network"="VM Network" --net:"Network 2"="other network" vi://root@/ vi://root@/?ip=

        After ovftool ran successfully, I had to right click on the VM in vCenter and select "Move To..." to assign the vm a specific compute node of our vCenter cluster.

        Cheers,

        Reply

        • Looks like wordpress stipped out my values between \

          ovftool -ds=data1 --net:”VM Network”=”VM Network” --net:”Network 2″=”other network” vi://root@"ip of esxi host"/"source vm name" vi://root@"ip of vcenter server"/?ip="ip of host in cluster to handle the job"

          Reply

  18. I running the OVFTOOL and received the Error:
    - The operation is not supported on the object.
    Completed with errors

    I have two ESXI 5.5.

    Can you help me?

    Reply

  20. Looking for a solution or a trick that I would be able to import license to the specific virtual machine then export it to OVF/OVA file, after that I want to deploy that OVF file on different ESX without losing the license of virtual machine.
    By default when you get an OVF of a machine from an ESX and deploy it on another ESX you will lose your license that imported before.

    Reply

  21. FYI, ovftool has a very annoying bug, in that if you have certain "bad" characters in the password, the command fails with:

    Error: cURL error: URL using bad/illegal format or missing URL

    to date, the characters i have found you can't have in a password and use that password with ovftool are: ? +

    it's a bit frustrating

    Reply

      • I know this is a very old topic, but I figured out why the password doesn't work. According to the `ovftool` debug logs (and some strace dumps I did), ovftool forms a URL that includes the userid and password - even if you entered the password at the prompt. The URL looks like this:

        vi://user:password@server

        Passwords in URLs must be urlencoded, otherwise they don't get interpreted correctly.

        ovftool, for some bizarre reason, even in version 4.2, does NOT internally urlencode the password, so the login fails. This IMO is an annoying and unnecessary bug.

        A workaround (not great, but it will allow you to use your password as is) is to urlencode the password on the command line and give that to ovftool to use. I've tried it and it actually works.

        Let's say that your password is (I hope this doesn't get shredded by the blog software...)

        my@random pass/+?$#%^&*()-_+={}[]\|;:'",/?

        This is guaranteed to upset many things, not just URLs 🙂

        Probably the most portable way you can do this is using a short Python or Perl snippet.

        $ python -c 'import urllib; import getpass; print(urllib.quote_plus(getpass.getpass()))'
        Password:
        my%40random+pass%2F%2B%3F%24%23%25%5E%26%2A%28%29-_%2B%3D%7B%7D%5B%5D%5C%7C%3B%3A%27%22%2C%3C.%3E%2F%3F

        Copy the urlencoded output, and when ovftool asks for the password, paste it.

        Reply

  22. This is so cool. In the past i was migratin with vdr - Backup. This is so much faster... From hours for a 200GB thick Windowsmachine to under 14 minutes.
    Thx William, you'r awesome!

    Reply

  23. I want to migrate the files from geo seperated ESX hosts. Can I use rsync. I tried to accees Damiens page but it looks I need to get permission.

    Reply

  26. Thanks for the post this worked great except for one thing. I had to log in and reset the NIC settings once migrating.

    Reply

  27. Have you seen the error

    Error: cURL error: Server returned nothing (no headers, no data)
    Completed with errors

    When performing a copy ?

    Reply

    • Usually this happens because ovftool can't perform a lookup of the hostname/IP, least that's resolution when I've seen folks his this problem when using ovftool.

      Reply

      • Kudos... This tip helped me resolved a vcsa 5.5 to 6 upgrade problem I was having. Found out that my DNS was only somewhat routable from my vcsa client making the system error out when trying to resolve names.

        Reply

  28. Good info, only problem I've run into is that it is KILLER slow! Is there any way to speed it up? something along the lines of the VM Converter encryption setting that slowed that down? I started a move of a 750g VM over night and after 10 hours it was at 5%. That seems really slow to me. That would take 9 days to transfer! Any suggestiosn are MUCH appreciated!

    Reply

    • I think if you run the ovftool on your workstation it will pump traffic trough it, not directly between hosts. It is true, that it will not use any local storage, but the data will cross your PC. So if you try to move VMs between two ESXi hosts which are located in the same but remote to you network, you will act as a proxy and send all the data accross the WAN link. This can be really slow...

      At least this is what I observed using windows based ovftool. In some other artikle I found a recommendation to setup a little Linux VM on one of the ESXi hosts and run ovftool there. This should greatly speed up the process.

      Reply

  29. I'm getting an error when I run the following command:

    C:\Program Files\VMware\VMware OVF Tool>ovftool -ds=SRVHS-Servers-VNX vi://root@srvhs-esxi-old/SRVHS-SCCM vi://root@srvhs-esxi
    Error: Unexpected option: vi://root@srvhs-esxi
    Completed with errors

    I have tried to move the -ds option to the end and also in between the source and destination but what ever is the last string of the command I get an Unexpected option error.

    Reply

  30. Whatever I try, I can't seem to get the syntax for the datastore switch right. I've tried William's example and several others.
    Source ESX = 5.1, target = 5.5

    C:\Program Files\VMware\VMware OVF Tool>ovftool -ds=10.12.1.99-datastore vi://ow
    -vmhost1.ontwikkel.loc/OW-VIRW3SS vi://[email protected]
    Enter login information for source vi://ow-vmhost1.ontwikkel.loc/
    Username: root
    Password: *******
    Opening VI source: vi://*protected email*:443/OW-VIRW3SS
    Enter login information for target vi://10.12.1.99/
    Username: root
    Password: *******
    Opening VI target: vi://[email protected]:443/
    Error: Invalid target datastore specified (10.12.1.99-datastore): Possible completions:
    datastore1
    Completed with errors

    Reply

  31. I am finding that while in some cases (like when I was trying to transfer some CentOS VMs with ovftool) the network configuration was not being ported. The resulting IP configuration is effectively blank on the target VM. Do you know what could cause this?

    Reply

    • I ran into this too. I think it's a difference in the network hardware (Nic cards) in the different esx hosts. On linux I ran ifconfig -a (or ip addr) to get the interface names (eno1, ens132, eth0, etc...) then updated my network configuration scripts (debian = /etc/network/interfaces. Redhat/CentOS= /etc/sysconfig/network-scripts/ifcfg-eth0 or /etc/sysconfig/network-scripts/ifcfg-ens0 or /etc/sysconfig/network-scripts/ifcfg-ens132, etc etc etc).

      Then brought up the interfaces with "ifup eth0" (replace eth0 with correct interface)

      Reply

  32. Hi,
    Thanks for the tip ! It worked great with a small VM, but now I would like to use it with a critical VM that can't be turned off for long. The problem is that this time it concerns the copy of 2 datastores (ssd and sata). How can I put an option to say datastore1 (source) goes to datastore1 (target) and datastore2 (source) goes to datastore2 (target)? Is it even possible to do that with OVF tool?
    Thanks

    Reply

  33. Hi William,
    how to force ovftool to move VM from one ESXi to another using VMKernel port? I've connected 2 ESXi servers directly using patch cord and assigned address 10.10.1.10 and 10.10.1.11 respectively. Should I expect faster transfer speed?
    Thank you,
    Vadim

    Reply

  34. Hi William,
    Q1: I have problem, error if I use space in name VM. Any idea? Help me, please.
    esxi4 - VMware ESXi 4.0
    esxi6 - VMware ESXi 6.0
    ovftool - VMware OVF Tool 4.1 for Win 64-bit
    nameVM - vSphere MA (vMA6)

    C:\Program Files\VMware\VMware OVF Tool>ovftool.exe -ds=Local vi:"//root@esxi4/vSphere MA (vMA6)" vi://root@esxi6
    Enter login information for source vi://esxi4/
    Username: root
    Password: *********
    Opening VI source: vi://root@esxi4:443/vSphere%20MA%20(vMA6)
    Error: vim.fault.TaskInProgress
    Completed with errors

    C:\Program Files\VMware\VMware OVF Tool>ovftool.exe -ds=Local vi://root@esxi4/"vSphere MA (vMA6)" vi://root@esxi6
    Enter login information for source vi://esxi4/
    Username: root
    Password: *********
    Opening VI source: vi://root@esxi4:443/vSphere%20MA%20(vMA6)
    Error: vim.fault.TaskInProgress
    Completed with errors

    C:\Program Files\VMware\VMware OVF Tool>ovftool.exe -ds=Local "vi://root@esxi4/vSphere MA (vMA6)" vi://root@esxi6
    Enter login information for source vi://esxi4/
    Username: root
    Password: *********
    Opening VI source: vi://root@esxi4:443/vSphere%20MA%20(vMA6)
    Error: vim.fault.InvalidState
    Completed with errors

    Q2: ovftool.exe --compress=9 -ds=Local vi://root@esxi4/vMA6 vi://root@esxi6
    Option: --compress or --compress=9 with the above command, has no effect. Any idea?

    Reply

    • Paul,

      Try changing quotes in your command to

      ovftool.exe -ds=Local vi://root@esxi4/"vSphere MA (vMA6)” vi://root@esxi6

      Reply

      • C:\Program Files\VMware\VMware OVF Tool>ovftool.exe -ds=Local vi://root@esxi4/”vSphere MA (vMA6)” vi://root@esxi6
        Enter login information for source vi://esxi4/
        Username: root
        Password: *********
        Opening VI source: vi://root@esxi4:443/vSphere%20MA%20(vMA6)
        Error: vim.fault.TaskInProgress
        Completed with errors

        Reply

  35. Hi William,

    Good Day! we are currently in the process of consolidating our data centers. We are about to move all of our Esxi host from our Amsterdam data center to our Chicago data center. We are looking for ways and means to move it in a fast and efficient manner. is the ovftool will help? and what type of approach you can recommend?

    Reply

    • It really depends on what you need/care about. ovftool allows you to export/import offline VMs. If you care about speed, you may consider just storing all the VMs on an external drive and shipping it to your final destination and then re-importing them in. There are other tools like VMware Converter that can do V2V migration or if you're running vSphere 6.0, you could even leverage XvC-vMotion feature to migrate a VM (online or offline) between vCenter Servers, of course if VM networking isn't stretched or running with an overlay, VM will still see downtime. It sounds like you're after speed, so the first recommendation is what I would do and its something I've done as a customer 6yrs ago where we had acquired a company and need to quickly consolidate. We just had everything exported to external disk and re-imported once it hit our datacenter.

      Reply

  37. Thank you sir. I had to move a large VM to a dedicated standalone host and first tried it with Veeam, but that gave me some random error. This works perfectly. Time to clean up that Veeam installation and go to bed, while my VM moves overnight 🙂

    Reply

  39. Still very usefull!
    Is there a way to keep the MAC address of the original VM in the cloned one?

    I got problems on some moved VMs with licenses and interface nameing.

    Thanks,
    Steffen

    Reply

    • you can manually specify the MAC in the VM configuration. just make sure you stop using it on the old host or it will break things.

      Reply

      • You mean, changing the MAC address in the .vmx file?
        I had hoped, there is a more easy way.
        And it does no work with generated MAC addresses. VMware will change it upon startup. 🙁

        Reply

        • Steffen, just don't clone the VM - move it instead! Veeam and Extrasphere will help you.

          Reply

  40. If you don't have a shared storage you can create it in temporary Linux VM and run storage migration twice with Extrasphere solution.
    It will be very similar to ovf trick but need less downtime.
    Or use Veeam - it's free and easy to use )

    Reply

  41. Can you run multiple instances of the ovftool from the same PC to copy 2 or more VMs at the time? Or will this cause an issue/congestion?

    Reply

  43. Nice article!

    We use vSphere Replication for these scenarios, but good to have as another option.

    Thanks!

    Reply

  44. Thanks, unfortunately, after finally getting the right parameters (paths and datastore/cluster/pool names), I got the «ovftool does not support this server».

    Reply

  48. Veeam B&R is the BEST CHOICE if your V.E Virtual Environment is properly licensed not free license applied on it
    it will give you 6-35mb/s and thin provisioning also applied
    easy to use

    Reply

  49. Could they be on different subnets, connected directly from one physical nic to another? The create vmkern/vmotion designated vnics on each?
    I have a lab I would like to push things to, while keeping it segregated.

    Reply

  52. For transferring files faster, I prefer using GS Richcopy 360. It has multi threaded feature so its faster than the rest. Kinda cool and is all in one package!

    Reply

  53. Hi!

    I am experiencing a very strange issue with OVF exporting.
    With somes VMs, after a lot of tuning on my script (involving OVFtool), I receive the following result:

    Writing OVA package: /data/Site_Mariage.ova
    Transfer Completed
    Error: vim.fault.InvalidState
    Error: vim.fault.InvalidState
    Completed with errors

    However, the OVA file is well created. When I extract it (with tar), I have the 3 files but it seems that there are some errors:
    - the mf file is wrong (the sha1 hash is not generated. The hash is replaced by "xxxxxxxxxxxx")
    - the ovf file seems weird: for example, I can see this line "ovf:size=9223372036854775807". The size seem sto be overestimated.
    - the vmdk seems to be good

    When I try to export the OVA file with vSphere client 6.0, the problem is the same. I also tried to export as an OVF, but still the same errors.

    Important: When I try with a different VM, there is no problem.

    I really don't understand what's happening...
    Do you have any idea?

    Thank you very much for your help!

    Reply

  56. William Lam I am your fan.

    The article is from 2012 and still served me today.

    I thought a lot about a way to migrate / copy my vms between esxi without storage.
    This function of ovftools, this for me was very cool. I had thought of creating an NFS in Esxi, to do this task, looking for this article, I previously did the export / import process using a PC between ESXi, but it directly reduced my time in half.

    When running I came across this error:
    line 26 unsupported family 14
    I saw some shapes around, but I stopped thinking and did like this:

    1. unregistered in vm
    1. download .vmx
    2. I edited and changed to vm_version13
    3. upload .vmx
    4. I registered to vm (keeping it off)
    5. and performed the process

    that simple.

    Thanks dude

    Reply

Trackbacks

Thanks for the comment!

This site uses Akismet to reduce spam. Learn how your comment data is processed.