WilliamLam.com

Handy tidbits & workarounds for the VCS to VCSA Migration Fling

by //

The VCS (Windows VC) to VCSA Migration Fling has been out for a little over 6 months and the response from customers thus far has simply been phenomenal. We have also received some great feedback (200+ comments) from customers who have tried out the Fling in either a Dev/Test environment and some even in their production environment for those that are a bit more on the adventurous side. I have also had the pleasure in talking to some of these customers who have been successful in migrating off of their Windows vCenter Server (both large and small) and onto the vCenter Server Appliance (VCSA) and sharing additional feedback they may have about the Fling and how we can further improve.

Given the popularity of this topic, I thought it would be useful to aggregate some of the learning's, tidbits and workarounds that have been discovered in the past 6months to help any new or even existing users who might be interested in trying out the Fling. We really do appreciate all the feedback that everyone has given in the various forms and in fact, several of the workarounds were ones provided by our customers. As you know, the Fling today is not currently officially supported, however the feedback has really helped our PM/Engineering team. In fact, you can even get a sneak peak at an early Tech Preview we did at VMworld here to give you an idea on how some of your feedback has influenced a feature that may or may not be out in the near future 😉

Tidbit 1 Microsoft Windows 2012 is currently not supported.
Additional Info There is a known winexe bug which is affects migrating from this specific OS platform.
Workaround Engineering has a fix for this and is currently in the process of testing the fix along with legal review. There is not an ETA due to the review but we hope to release an update to Fling that includes this fix very soon. Stay tuned!This has been resolved with v0.9.1 of the Migration Appliance and for more details please take a look here.
Tidbit 2 Use of non-default (custom) ports on Microsoft SQL Server Database is not supported
Additional Info The Fling currently assumes the SQL Server Database is running on port 1433
Workaround Engineering has a fix for this and is currently in the process of testing the fix along with legal review. There is not an ETA due to the review but we hope to release an update to Fling that includes this fix very soon. Stay tuned!
Tidbit 3 Use of an Embedded Microsoft SQL Server or Microsoft SQL Express Database on the vCenter Server is not supported
Additional Info Since the source Windows vCenter Server must be powered off during the database migration; running the database on the same source vCenter Server is not possible.
Workaround One option is to re-ip the source Windows vCenter Server and ensuring the vCenter Server service is completely disable which would allow the Migration Appliance to communicate with the database. This is not ideal as you are modifying the source Windows vCenter Server but has worked in our testing. Second option that several other customers have recommended instead is to export the vCenter Server Database to a single instance of a Microsoft SQL Server or Microsoft SQL Express and that has worked really well.
Tidbit 4 Clustered database such as Microsoft Clustering Services (MSCS) is not supported
Additional Info There have been issues from some customers when trying to connect to an instance of the vCenter Server Database behind an MSCS Cluster.
Workaround Exporting the vCenter Server Database to a single instance of a Microsoft SQL Server or Microsoft SQL Express and then using the Fling has worked for several customers.
Tidbit 5 Issues connecting to a non-default named instance (e.g. SERVERNAME\VCENTER) of the vCenter Server Database.
Additional Info Some customers have had issues with the connection string to a non-default named instance of the vCenter Server Database during the database migration portion of the Fling.
Workaround A solution that was identified by a customer used the following: http://stackoverflow.com/a/11921896/2668394
Tidbit 6 Upgrade to VCSA 6.0 after migrating from Windows vCenter Server 5.5 to VCSA 5.5 fails
Additional Info You see the following error "Extra sequences: vpx_host_cnx_seq;" in /var/log/vmware/upgrade/vcdb_req.err during the upgrade to VCSA 6.0. These sequences are only found and valid in a Microsoft SQL Server Database and are not relevant in an vPostgres Database and just simply need to be dropped as they are not used at all.
Workaround Login to the VCSA 6.0 appliance as root and run the following command: /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "drop sequence if exists vpx_host_cnx_seq cascade"

If you are running into issues while through the the migration, one thing you can do is login to the Migration Appliance and go to another virtual console (ALT+F2) and view the Migration logs  under /var/log/migrate.log SSH is currently not installed by default. If you wish to pull out the logs for additional support, you can install which will require internet access and you can do so by running the following commands:

sudo apt-get -y update
sudo apt-get -y install openssh-server
sudo /etc/init.d/ssh start

The credentials to the Migration Appliance is vmware/vmware

Lastly, if there are other tidbits or workarounds that you would like to share, feel free to leave a comment and I will get it added to the list.

Tech Preview of Windows VC to VCSA Migration at VMworld

by //

A couple weeks back I had teased out the #migrate2vcsa hashtag on Twitter and said to stay tune for folks planning to attend VMworld US in person. If you attended VMworld last week, you may gotten more details during TAM Day, at the VMware Booth or in either of the VCSA breakout sessions INF5975 & INF4528. I just found out this week that some of the VMworld sessions have already been posted online for everyone and it looks like one of my sessions, INF4528 vCenter Server Appliance Best Practices & Tips / Tricks session was one of them.

Well, it looks like the cats out of the bag! To be perfectly honest, I am actually glad, since now I can share some more details with my readers on what the VC Engineering team has been working very hard on. As you probably can guess from the cryptic hashtag, the topic is related to migrating from a Windows vCenter Server to the vCenter Server Appliance. About 6 months ago, we had released the VCS to VCVA Converter Fling and the feedback from customers has just been phenomenal. Though the Fling only supported a limited set of configurations, it did allow us to quickly gather feedback from customers on whether such a tool should still be further developed and more importantly, if the current workflow met user expectations.

At VMworld, we showed off a video of an early but functional Tech Preview of migrating from a Windows vCenter Server to a the vCenter Server Appliance to help Engineering get feedback from customers on the overall workflow. From the customers that I have talked to, the feedback have been super positive and in fact, they were quite excited. I do have to stress that this is still a Tech Preview and you should review the disclaimer below, but it should give you an idea of our current thinking.

Disclaimer: This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

We would still love to hear from you in case you did not get a chance to talk to us during VMworld. We are interested in any feedback you may have in terms of the overlal workflow and whether the process is intuitive or not. If you have any feedback after watching the video, please either leave a comment on my blog here or tweet your feedback using the #migrate2vcsa hashtag on Twitter.

For comparison, you can watch the Fling's workflow video here and compare and contrast that with the Tech Preview video below. In case the video does not automatically start playing at the Migration portion of the presentation (46:35), you can click here for the direct link.

Considerations when migrating VMs between vCenter Servers

by // 19 Comments

Something that I really enjoy when I get a chance to, is to speak with our field folks and learn a bit more about our customer environments and some of the challenges they are facing. Last week I had quick call with one of our TAMs (Technical Account Managers) regarding the topic of Virtual Machine migration between vCenter Servers. The process of migrating Virtual Machines between two vCenter Servers is not particularly difficult, you simply disconnect the ESXi hosts from one vCenter Server and re-connect to the new vCenter Server. This is something I have performed on several occasions when I was a customer and with some planning it works effortlessly.

However, there are certain scenarios and configurations when migrating VMs between vCenter Servers that could potentially cause Virtual Machine MAC Address collisions. Before we jump into these scenarios, here is some background. By default, a Virtual Machine MAC Address is automatically generated by vCenter Server and the algorithm is based on vCenter Server's unique ID (0-63) among few other parameters which is documented here. If you have more than one vCenter Server, a best practice is to ensure that these VC IDs are different, especially if they are in the same broadcast domain.

As you can imagine, if you have two vCenter Servers that are configured with the same VC ID, there is a possibility that a duplicate MAC Address could be generated. You might think this is probably a rare event given the 65,000 possible MAC Address combinations. However, it actually happens more frequently than you think, especially in very large scale environments and/or Dev/Test for continuous build/integration environments which I have worked in the past and I have personally seen these issues before.

Going back to our vCenter Server migration topic, there are currently two main scenarios that I see occurring in customr environments and we can explore each in a bit more detail and their implications:

  • Migrate ALL Virtual Machines from old vCenter Server to new vCenter Server
  • Migrate portion of Virtual Machines from old vCenter Server to new vCenter Server

Migrate all Virtual Machines:

vm-migration-between-vcenter-0
In the diagram above, we have vCenter Server 1 and vCenter Server 2 providing a before/after view. To make things easy, lets say they have VC ID 1 and 2. If we migrate ALL Virtual Machines across, we can see their original MAC Addresses will be preserved as we expect. For any new Virtual Machines being created, the 4th octet of the MAC Address will differ as expected and the vCenter Server will guarantee it is unique. If you want to ensure that new Virtual Machines keep a similar algorithm, you could change the vCenter Server ID to 1. No issues here and the migration is very straight forward.

Migrate A portion of Virtual Machines:

vm-migration-between-vcenter-1
In the second diagram, we still have vCenter Server 1 and vCenter Server with unique VC IDs. However, in this scenario we are only migrating a portion of the Virtual Machines from vCenter Server 1 to vCenter Server 2. By migrating VM2 off of vCenter Server 1, the MAC Address of VM2 is no longer registered with that vCenter Server. What this means is that vCenter Server 1 can potentially re-use that MAC Address when it generates a new request. As you can see from the above diagram, this is a concern because VM2 is still using that MAC Address in vCenter Server 2, but vCenter Server 1 is no longer aware of its existence.

The scenario above is what the TAM was seeing at his customer's site and after understanding the challenge, there are a couple of potential solutions:

  1. Range-Based MAC Address allocation - Allows you to specify a range of MAC Addresses to allocate from which may or may not helpful if the migrated MAC Addresses are truly random
  2. Prefix-Based MAC Address allocation -  Allows you to modify the default VMware OUI (00:50:56) which would then ensure no conflicts would be created with previously assigned MAC Addresses. Though this could solve the problem, you potentially could run into collisions with other OUI's within your environment
  3. Leave VMs in a disconnected state - This was actually a solution provided by another TAM on an internal thread which ended up working for his customer. The idea was that instead of disconnecting and removing the ESXi host when migrating a set of Virtual Machines, you would just leave it disconnected in vCenter Server 1. You would still be able to connect the ESXi host and Virtual Machines to vCenter Server 2 but from vCenter Server 1 point of view, the MAC Addresses for those Virtual Machines are seen as in use and it would not be reallocated.

I thought option #3 was a pretty interesting and out of the box solution that the customer came up with. The use case that caused them to see this problem in the first place is due to the way they provision remote environments. The customer has a centralized build environment in which everything is built and then shipped off to the remote sites which is a fairly common practice. Since the centralized vCenter Server is not moving, you can see how previously used MAC Addresses could be re-allocated.

Although option #3 would be the easiest to implement, I am not a fan of seeing so many disconnected systems from an operational perspective as it is hard to tell if there is an issue with the ESXi host and Virtual Machines or because it has been migrated. I guess one way to help with that is to create a Folder called "Migrated" and move all disconnected ESXi hosts into that folder which would help mask that away and disable any alarms for those hosts.

Some additional per-requisite checks that you can perform prior to the partial Virtual Machine migration:

Ensure that the destination vCenter Server is not configured with the same VC ID else you can potentially run into duplicate MAC Address conflicts. You can do this either manually through the vSphere Web/C# Client or leveraging our CLI/API to do so.

Here is an example using PowerCLI to retrieve the vCenter Server ID:

Get-AdvancedSetting -Entity $server -Name instance.id

Ensure no duplicate MAC Addresses exists by comparing the MAC Addresses of the Virtual Machines to be migrated to the Virtual Machines in the new environment. Again, you can either do this by hand (which I would not recommend) or leveraging our CLI/API to extract this information.

Here is an example using PowerCLI to retrieve the MAC Addresses for a Virtual Machine:

Get-VM |  Select-Object -Property Name, PowerState, @{"Name"="MAC";"Expression"={($_ | Get-NetworkAdapter).MacAddress}}

If there are other scenarios or solutions that you have seen with Virtual Machine migrations between vCenter Serves, feel free to leave a comment. I am sure others can benefit from past experiences or any other lesson learns.