Search Results for: vsphere events

Automating Cross vCenter vMotion (xVC-vMotion) between the same & different SSO Domain

05.26.2016 by William Lam // 79 Comments

In the last couple of months, I have noticed an increase in customer interests in using the Cross vCenter vMotion (xVC-vMotion) capability that was introduced back in vSphere 6.0. In my opinion, I still think this is probably one of the coolest features of that release. There is no longer the limitation of restricting your Virtual Machine mobility from within a single vCenter Server, but you can now live migrate a running VM across different vCenter Servers.

The primary method to start a xVC-vMotion is by using the vSphere Web Client which requires your vCenter Servers Servers to be part of the same SSO Domain and will automatically enable the new Enhanced Linked Mode (ELM) feature. ELM allows you to easily manage and view all of your vCenter Servers from within the vSphere Web Client as shown in the example screenshot below.

However, the vSphere Web Client is not the only way to start a xVC-vMotion, you can also automate it through the use of the vSphere API. In fact, there is even an "Extended" capability of xVC-vMotion that is not very well known which I have written about here which allows to live migrate a running VM across two different vCenter Servers that are NOT part of the same SSO Domain. This Extended xVC-vMotion (unofficially I am calling it ExVC-vMotion) is only available when using the vSphere API as the vSphere Web Client is unable to display vCenter Servers that are part of another SSO Domain. Below is a quick diagram to help illustrate the point in which VM1 can be seamlessly migrated between different vCenter Servers from within the same SSO Domain as well as between different vCenter Servers that are not part of the same SSO Domain.

Note: For additional details and requirements for Cross vCenter vMotion, please have a look at this VMware KB 210695 and this blog post here for more information.

UPDATE (06/15/17) - I have added a few minor enhancements to the script to support migrating a VM given a vSphere Resource Pool which enables the ability to migrate to and from VMware's upcoming VMware Cloud on AWS (VMC). There is also an additional UppercaseUUID parameter which seems to be required for some xVC-vMotions where the vCenter Server's InstanceUUID must be provided as all upper case or the operation will fail. I have still not identified why this is needed for some migrations, but for now there is a nice flag that can be used to enable this if you are hitting this problem.

UPDATE (04/08/17) - In vSphere 6.0 Update 2, there is a known limitation which prevents a VM that has multiple VMDKs stored across different datastores to be xVC-vMotion (compute only) using the vSphere Web Client. This limitation no longer exists in vSphere 6.0 Update 3 but does require customers to upgrade. If you need to perform a compute-only xVC-vMotion where the VM has multiple VMDKs across different datastores, the vSphere APIs does not have this limitation and you do not necessary need to upgrade to be able to perform this operation. Huge thanks to Askar Kopayev who discovered this and also submitted an enhancement to my xMove-VM PowerCLI script to support this functionality.

Given the amount of interest recently and some of the feedback on my original ExVC-vMotion script which I had written about here, I figured it was time to refactor my code so that it could easily support both ExVC-vMotion as well as standard xVC-vMotion. In addition, I have also added support for migrating to and from a Distributed Virtual Switch (VDS), where as previously the example only supported Virtual Standard Switch (VSS). Lastly, the script now also supports migrating a VM that is configured with multiple vNICs.

The new script is now called xMove-VM.ps1 and is even more simpler than my original script. You will need to edit the script and update the following variables:

Variable	Description
vmname	Name of the VM to migrate
sourceVC	The hostname or IP Address of the source vCenter Server in which the VM currently resides in
sourceVCUsername	Username to the Source vCenter Server
sourceVCPassword	Password to the Source vCenter Server
destVC	The hostname or IP Address of the Destination vCenter Server in which to migrate the VM to
destVCUsername	Username to the Destination vCenter Server
destVCpassword	Password to the Destination vCenter server
datastore	Name of the vSphere Datastore to migrate the VM to
cluster	Name of the vSphere Cluster to migrate the VM to
resourcepool	Name of the vSphere Resource Pool to migrate the VM to
vmhost	Name of the ESXi host to migrate the VM to
vmnetworks	Name of the vSphere Network(s). in the order in of the vNIC interfaces to migrate the VM to
switch	Name of the vSphere Switch to migrate the VM to that is comma separated and ordered by vNIC
switchtype	The type of vSphere Switch (vss or vds)
xvctype	Whether this is a Compute-only Cross VC-vMotion (1=true or 0 = false)
UppercaseUUID	There cases where the vCenter Server InstanceUUID must be all caps ($true or $false)

Here is a screenshot of running the script:

Note: When changing the type of vSphere Switch, the following combinations will are supported by the script as well as using the vSphere Web Client: VDS to VDS, VSS to VSS and VSS to VDS. VDS to VSS is not supported using the UI or API and neither are 3rd party switches supported.

Here are some additional xVC-vMotion and vMotion articles that may also useful to be aware of:

Functional USB 3.0 Ethernet Adapter (NIC) driver for ESXi 5.5 & 6.0

03.28.2016 by William Lam // 81 Comments

Earlier this month I wrote an article demonstrating a functional USB ethernet adapter for ESXi 5.1. This was made possible by using a custom built driver for ESXi that was created over three years ago by a user named Trickstarter. After having re-discovered the thread several years later, I had tried reaching out to the user but concluded that he/she has probably moved on given the lack of forum activity in the recent years. Over the last few weeks I have been investigating to see if it was possible to compile a new version of the driver that would function with newer versions of ESXi such as our 5.5 and 6.0 release.

UPDATE (02/12/19) - A new VMware Native Driver for USB-based NICs has just been released for ESXi 6.5/6.7, please use this driver going forward. If you are still on ESXi 5.5/6.0, you can continue using the existing driver but please note there will be no additional development in the existing vmklinux-based driver.

UPDATE (01/22/17) - For details on using a USB-C / Thunderbolt 3 Ethernet Adapter, please see this post here.

UPDATE (11/17/16) - New driver has been updated for ESXi 6.5, please find the details here.

After reaching out to a few folks internally, I was introduced to Songtao Zheng, a VMware Engineer who works on some of our USB code base. Songtao was kind enough to provide some of assistance in his spare time to help with this non-sanction effort that I was embarking on. Today, I am please to announce that we now have a functional USB ethernet adapter driver based on the ASIX AX88179 that works for both ESXi 5.5 and 6.0. This effort could not have been possible without Songtao and I just want to say thank you very much for all of your help and contributions. I think it is safe to say that the overall VMware community also thanks you for your efforts. This new capability will definitely enable new use cases for vSphere home labs that were never possible before when using platforms such as the Intel NUC or Apple Mac Mini for example. Thank you Songtao! I would also like to extend an additional thank you to Jose Gomes, one of my readers, who has also been extremely helpful with his feedback as well as assistance on testing the new drivers.

Now, Before jumping into the goods, I do want to mention there are a few caveats to be aware of and that I think it is important to understand them before making any purchasing decisions.

First and foremost, this is NOT officially supported by VMware, use at your own risk.
Secondly, we have observed there is a substantial difference in transfer speeds between Transmit (Egress) and Receive (Ingress) traffic which may or may not be acceptable depending on your workload. On Receive, the USB network adapter is performing close to a native gigabit interface. However, on Transmit, the bandwidth mysteriously drops by ~50% which includes very inconsistent transfer speeds. We are not exactly sure why this is the case, but given ESXi does not officially support USB based ethernet adapters, it is possible that the underlying infrastructure was never optimized for such devices. YMMV
Lastly, for the USB ethernet adapter to properly function, you will need a system that supports USB 3.0 which kind of makes sense for this type of a solution to be beneficial in the home lab. If you have a system with USB 2.0, the device will probably not work at least from testing that we have done.

Note: For those interested in the required source code changes to build the AX88179 driver, I have published all of the details on my Github repo here.

Disclaimer: In case you some how missed it, this is not officially supported by VMware. Use at your own risk.

Without further ado, here are the USB 3.0 gigabit ethernet adapters that are supported with the two drivers:

StarTech USB 3.0 to Gigabit Ethernet NIC Adapter
StarTech USB 3.0 to Dual Port Gigabit Ethernet Adapter NIC with USB Port
j5create USB 3.0 to Gigabit Ethernet NIC Adapter (verified by reader Sean Hatfield 03/29/16)
Vantec CB-U300GNA USB 3.0 Ethernet Adapter (verified by VMware employee 05/19/16)
DUB-1312 USB 3.0 Gigabit Ethernet Adapter (verified by twitter user George Markou 07/29/16)

Note: There may be other USB ethernet adapters that uses the same chipset which could also leverage this driver but these are the only two that have been verified.

Here are the ESXi driver VIB downloads:

Note: Although the drivers were compiled against a specific version of ESXi, they should also work on the same major version of ESXi, but I have not done that level of testing and YMMV.

Verify USB 3.0 Support

As mentioned earlier, you will need a system that is USB 3.0 capable to be able to use the USB ethernet adapter. If you are unsure, you can plug in a USB 3.0 device and run the following command to check:

lsusb

What you will be looking for is an entry stating "Linux Foundation 3.0 root hub" which shows that ESXi was able to detect a USB 3.0 port on your system. Secondly, look for the USB device you just plugged in and ensure the "Bus" ID matches that of the USB 3.0 bus. This will tell you if your device is being claimed as a USB 3.0 device. If not, you may need to update your BIOS as some systems may have USB 2.0 enabled by default like earlier versions of Intel NUC as desribed here. You may also be running pre-ESXi 5.5 which did not support USB 3.0 as mentioned here, so you may need to upgrade your ESXi host to at least 5.5 or greater.

Install Driver

You can either install the VIB directly onto your ESXi host or by creating a custom ESXi ISO that includes the driver using a popular tool like ESXi Customizer by Andreas Peetz.

To install the VIB, upload the VIB to your ESXi host and then run the following ESXCLI command specifying the full path to the VIB:

esxcli software vib install -v /vghetto-ax88179-esxi60u2.vib -f

Lastly, you will need to disable the USB native driver to be able to use this driver. To do so, run the following command:

esxcli system module set -m=vmkusb -e=FALSE

You will need to reboot for the change to go into effect.

To verify that the USB network adapter has been successfully claimed, run either of the following commands to list your physical NICs:

esxcli network nic list
esxcfg-nics -l

To add the USB uplink, you will need to either use the vSphere Web Client or ESXCLI to add the uplink to either a Virtual or Distributed Virtual Switch.

To do so using ESXCLI, run the following command and specify the name of your vSwitch:

esxcli network vswitch standard uplink add -u vusb0 -v vSwitch0

Uninstall Driver

To uninstall the VIB, first make sure to completely unplug the USB network adapter from the ESXi first. Next, run the following ESXCLI command which will automatically unload the driver and remove the VIB from your ESXi host:

esxcli software vib remove -n vghetto-ax88179-esxi60u2

Note: If you try to remove the VIB while the USB network adapter is still plugged in, you may hang the system or cause a PSOD. Simply reboot the system if you accidentally get into this situation.

Troubleshooting

If you are not receiving link on the USB ethernet adapter, it is most likely that your system does not support USB 3.0. If you find the a similar message like the one below in /var/log/vmkernel.log then you are probably running USB 1.0 or 2.0.

2016-03-21T23:30:49.195Z cpu6:33307)WARNING: LinDMA: Linux_DMACheckConstraints:138: Cannot map machine address = 0x10f5b6b44, length = 2 for device 0000:00:1d.7; reason = address exceeds dma_mask (0xffffffff))

Persisting USB NIC Configurations after reboot

ESXi does not natively support USB NIC and upon a reboot, the USB NICs are not picked up until much later in the boot process which prevents them from being associated with VSS/VDS and their respective portgroups. To ensure things are connected properly after a reboot, you will need to add something like the following in /etc/rc.local.d/local.sh which re-links the USB NIC along with the individual portgroups as shown in the example below.

esxcfg-vswitch -L vusb0 vSwitch0
esxcfg-vswitch -M vusb0 -p "Management Network" vSwitch0
esxcfg-vswitch -M vusb0 -p "VM Network" vSwitch0

You will also need to run /sbin/auto-backup.sh to ensure the configuration changes are saved and then you can issue a reboot to verify that everything is working as expected.

Summary

For platforms that have limited built-in networking capabilities such as the Intel NUC and Apple Mac Mini, customers now have the ability to add additional network interfaces to these systems. This will now open up a whole new class of use cases for vSphere based home labs that were never possible before, especially with solutions such as VSAN and NSX. I look forward to seeing what our customers can now do with these new networking capabilities.

Additional Info

Here are some additional screenshots testing the dual USB 3.0 ethernet adapter as well as a basic iPerf benchmark for the single USB ethernet adapter. I was not really impressed with the speeds for the dual ethernet adapter which I had shared some more info here. Unless you are limited on number of USB 3.0 ports, I would probably recommend just sticking with the single port ethernet adapter.

iPerf benchmark for Ingress traffic (single port USB ethernet adapter):

iPerf benchmark for Egress traffic (single port USB ethernet adapter):

The future of the ESXi Embedded Host Client

03.04.2016 by William Lam // 14 Comments

As many of you know, the ESXi Embedded Host Client project is something that is very near and dear to my heart. I have always felt that we needed a simple web interface that customers can just point their web browser to an ESXi host after a new installation and be able to quickly get started. One of the biggest benefit in addition to simplicity is that it is also very intuitive from a user experience standpoint which I believe is very important in a world where things can quickly get complex. In addition, it can also provide an interface for basic troubleshooting and support greenfield deployments where vCenter Server has not been deployed yet.

It has truly been amazing to follow the Embedded Host Client development from the initial idea to the first prototype built by VMware Engineers Kevin Christopher and Jehad Affoneh to its current implementation lead by Etienne Le Sueur and the ESXi team. I have really been fortunate to have had the opportunity to be so involved in this project. It is hard to imagine that in just little over 6 months, we have had had 5 releases of the Embedded Host Client Fling, all of which, produced with high quality development and rich feature sets.

You can click on the links below to get more details about each release.

08/11/15 - EHC Fling v1 released
08/26/15 - EHC Fling v2 released
10/23/15 - EHC Fling v3 released
12/21/15 - EHC Fling v4 released
02/07/16 - EHC Fling v5 released

I think its an understatement to say that customers are genuinely excited about this project as well, just look at some of the comments left on the Flings page here. Interestingly, this excitement has also been felt internally at VMware as well and I think this goes to show that the team has built something really special that affects anyone who works with VMware's ESXi Hypervisor.

So where to do we go from here? Are we done? Far from it ...

For those of you who follow me on Twitter know that I had recently refreshed my personal vSphere home lab from a Apple Mac Mini to latest Intel NUC running the yet to be release VSAN 6.2 (vSphere 6.0 Update 2). I was pleasantly surprised to see that ESXi Embedded Host Client (EHC) is now included out of the box with ESXi! Although this has been said by a few folks including myself, it is another thing to actually see it in person 🙂

Although the VMware Flings program is a great way to share and engage with our customers to get early feedback, it may not always be a viable option. As some of you may know, Flings are not officially supported and this sometimes prevents some of our customers from engaging with us and really putting the Flings through its paces. By making EHC out of the box, not only are we officially supporting it but it will also make it easier for customers to try out this new interface.

UPDATE (03/04/16) - It looks like I made a mistake and that the ESXi Embedded Host Client will NOT be released as a "Tech Preview" as previously mentioned but rather it will be officially GA'ed with vSphere 6.0 Update 2. EHC is a fully supported feature of ESXi.

Although EHC is very close to parity with the vSphere C# Client, it is still not 100% there. We will continue to improve its capabilities and if you have any feedback when trying out the EHC, do not hesitate and leave feedback or file a Feature Request through GSS. For those looking to live on the "edge" a bit more, we will still continue to release updates to the EHC Fling but if you want something that is stable, you can stick with the stock EHC included in ESXi 6.0 Update 2. We will still ship the legacy Windows vSphere C# Client, so you will not be forced to use this interface. However, it is no secret that VMware wants to get rid of the vSphere C# Client and that EHC is the future interface to standalone ESXi hosts.

One feature that I know that many of you have been asking about is Free ESXi. Well, I am please to say that support for Free ESXi has been added in the latest version of EHC included with the upcoming ESXi 6.0 Update 2 release and below is a screenshot demonstrating that it is fully functional.

Lastly, I just want to say that EHC has really morphed beyond just a "simple UI" for managing standalone ESXi hosts and has also enabled other teams at VMware to do some really amazing things and create new experiences with this interface. As I said earlier, this is just the beginning 😀 Happy Friday!

Here are some additional cool capabilities provided by EHC