WilliamLam.com

Running Nested Xen Hypervisor with VMware Tools on ESXi

by // 1 Comment

I recently saw an interesting tweet from former VMware colleague Steve Muir who use to run our VMware Academic Program (VMAP). From the looks of things, Steve has been exploring the virtual rabbit hole by performing some cool multi-level vInception by running Xen in a Nested ESXi Virtual Machine running on top of VMware Fusion 🙂 I wonder if he has tried to run a VM in the Xen instance yet?

xen-vmware-tools-nested-4
Anyway, the thing that caught my eye was not the fact that you could run a Xen Hypervisor within ESXi, this has actually been done before in the past. What was interesting to me was the fact that he could not get VMware Tools to show up when Xen is installed and running. I was actually curious to see why this was the case and while waiting for some of my physical hosts to finish rebooting, I figure I give this a go myself. I was leveraging the Wiki page that Steve had initially provided but that gave me some issues as Xen would kernel panic upon a reboot.

In talking to Steve about this, he provided me with another link to a QuickStart guide for Xen which simplifies the setup and I was able to finally get it booted up (definitely not as easy as an ESXi install) . Afterwards, I just installed VMware Tools as I normally would and you can even use this one-liner script here that will automate the installation:

Funny enough, just by using this latest guide for the Xen installation, I was able to get VMware Tools to show up in the vSphere Web/C# Client without any additional tweaks. Here is a screenshot to prove this works:

xen-vmware-tools-nested-1
If you are interested in setting this up, here are the high level steps:

  1. Install latest CentOS which is 6.5 using either the full ISO installer or netinstall which is what I prefer to use
  2. Install Xen per the Wiki
  3. Install VMware Toosl (this step can be swapped with Step 2 if you, it works either way)

Thanks Steve for sharing your notes on getting Xen running on ESXi and I guess you ended up getting an answer to your question 😉

Quick Tip - How to snapshot & revert a physical ESXi host

by // 5 Comments

Nested environment, which is something I did quite a bit of as a customer and still continue to do so today. I could easily snapshot my Nested ESXi environment, perform my tests and then quickly rollback to my original starting state. However, when it comes to testing a physical ESXi host, it is a bit more challenging as there is no "quick" snapshot functionality as far as I was aware of. It was only until recently did I have a use case for this and picked up a nice tidbit from one of our engineers on the team. It turns out you could "snapshot" a physical or even virtual ESXi host by just backing up the state.tgz file and then restoring it. As the name suggest, the state.tgz file contains all the configurations of your ESXi host. The process is pretty straight forward:

  1. SCP /bootbank/state.tgz and back that up to your local system or shared storage
  2. Perform your tests or make changes to the system
  3. When you are ready to restore, copy the state.tgz back into /bootbank folder
  4. Login to ESXi Shell and run reboot -f which will ensure no changes are saved to our state.tgz

Once the ESXi host reboots, it will use the restored state.tgz file and your system will be back at its original state. This process is actually not new, ESXi already provides a way to backup/restore

Quick Tip - ESXCLI CSV --format-param options

by // Leave a Comment

When using ESXCLI, the output is formatted using a "default" formatter based on the type of data being displayed. However, you can easily modify the output by using one of the three supported formatters: xml, csv and keyvalue or even leverage some internal ones mentioned by Steve Jin here. When working with some of the ESXCLI 'storage' namespaces, such as listing all the devices on an ESXi host, the output can be quite verbose as seen in the example below:

~ # esxcli storage core device list
t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__
   Display Name: Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__)
   Has Settable Display Name: true
   Size: 572325
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__
   Vendor: ATA
   Model: INTEL SSDSC2BB60
   Revision: D201
   SCSI Level: 5
   Is Pseudo: false
   Status: on
   Is RDM Capable: false
   Is Local: true
   Is Removable: false
   Is SSD: true
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: unknown
   Other UIDs: vml.01000000004254574c3333363330345a3636303054474e2020494e54454c20
   Is Local SAS Device: false
   Is Boot USB Device: false
   No of outstanding IOs with competing worlds: 32

t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__
   Display Name: Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__)
   Has Settable Display Name: true
   Size: 572325
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__
   Vendor: ATA
   Model: INTEL SSDSC2BB60
   Revision: D201
   SCSI Level: 5
   Is Pseudo: false
   Status: on
   Is RDM Capable: false
   Is Local: true
   Is Removable: false
   Is SSD: true
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: unknown
   Other UIDs: vml.01000000004254574c3331383330314a4c36303054474e2020494e54454c20
   Is Local SAS Device: false
   Is Boot USB Device: false
   No of outstanding IOs with competing worlds: 32

Usually for such a command, you are interested in a couple of specific properties and I bet you are probably spend a good amount of time scrolling up and down, I know I do. One useful option that is not very well documented (will be filing a bug for this) is the --format-param options which goes in-conjunction with the csv formatter. I always forget the syntax and can never find it when I Google for it so I am documenting this for myself but I think this would also be useful for others to know about.

The --format-param option allows you to specify specific property fields you care about. If we use the our ESXCLI example above, what I really care about are the following for each device:

  • Display Name
  • Is Local
  • Is SSD

Using the following command, we can then extract only those fields we care about:

~ # esxcli --formatter=csv --format-param=fields="Display Name,Model, Is Local,Is SSD" storage core device list
DisplayName,Model,IsLocal,IsSSD,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__),INTEL SSDSC2BB60,true,true,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__),INTEL SSDSC2BB60,true,true,
Local ATA Disk (t10.ATA_____WDC_WD4000FYYZ2D01UL1B0_______________________WD2DWMC130199689),WDC WD4000FYYZ-0,true,false,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL3183002H600TGN__),INTEL SSDSC2BB60,true,true,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304XL600TGN__),INTEL SSDSC2BB60,true,true,

If we now look at our output, we can easily see that we have 5 devices on our ESXi host and I can quickly see the Display Name of our device, whether it is a local device seen by ESXi and if it is an SSD. I find this filtering mechanism especially handy during troubleshooting or when you need to quickly identify a device for configuration.