ESXi 6.0 on Apple Xserve 2,1

I really enjoy hearing from my readers, especially when they share some of the unique challenges they have come across or boundaries they have pushed with our VMware software. Several weeks back I received an interesting email from a reader named John Clendenen who was able to get ESXi 6.0 running on both his Apple Xserve 3,1 as well as Xserve 2,1! To be honest, I was pretty surprised to hear that this worked and not only that, there was not a whole lot that John had to do to get it working. I thought this was a pretty cool setup and asked if John was interested in sharing more details with the VMware/Mac OS X community in case others were interested.

Disclaimer: This is not officially supported by VMware, please use at your own risk.

*** This is a guest blog post from John Clendenen ***

For the past 5 years, I have lived in New York where I work at various print and post-production studios. The IT situations in many of these locations is often home-grown and sub-adequate, if not barely functional, so I took it upon myself to learn OS X Server administration. I have a background in computer science and networking, so it wasn’t a huge leap to step into administration. I was able to gradually build one studio’s IT infrastructure form a single AFP share with incessant permissions problems, to an Open Directory driven, single sign-on infrastructure with mobile homes, messaging etc.. However, with each OS X Server update, things broke, and I was spending a lot of time putting out fires.

This led me to pursue virtualization in hopes of isolating services on separate machines, without telling the studio to go buy 8 Apple Mac Minis. However, I needed to learn how to do this, before pitching it to them, so I looked on Craigslist for some cheap hardware. I found an Xserve 2,1, which I was able to talk down to $100. I also found a brief post in a thread that said that the Xserve 2,1 ran ESXi 5.5 without issue. I figured I’d take the plunge and just resell it on eBay if it didn’t work out.

Since then, my home lab has grown considerably, and I have learned that the best way to provide Mac services, is simply to use Linux (I’ve had a great experience with Netatalk). That said, Open Directory, Software Update, Caching and a few other services still need to be run on a Mac, so it’s still necessary to have the hardware. For a while, I had 3 Xserves, all purchased very cheaply, running VMs. I just sold one, and will sell another in the next month or two in favor of some Supermicro hardware (I really am mostly running Linux VM’s at this point). I’ll keep the one to run a few Mac OS X VMs. I’m still working on getting Samba 4 to work as the PDC, but once that is running smoothly, I’ll have a functional environment that I can take to the studios where I work (with approved hardware of course). Then I’ll have an improved work experience, while also pulling some extra income on the installation/maintenance.

Anyway, you’ve come here to read about running ESXi 6.0 on an Xserve 2,1. Well, it works. It’s 7 years old, and not officially supported, but you already know that. So, if you were considering it, I doubt anything here will dissuade you. On top of that, there’s not much for me to say other than, it works. No tricks to it, just install and you’re off.

That said, I do have some recommendations and tips to help you if you want to get the most out of this hardware. 2 months ago, since I swapped out the RAID card for the standard backplane/interconnect and upgraded the RAM. It hasn’t skipped a beat since.

My System

This is my home custom 13u rack with sound insulation and active air flow. Networking is in the back (Ubiquiti). It sits in a lofted storage area with an A/C and vents into the bathroom.

Here you see an Xserve 3,1 on top of an Xserve 2,1. There’s a blank unit because I just sold an Xserve 2,1 on eBay, and the other 2,1 will be for sale soon as well to make room for a 4 node 2U Supermicro Server. The NAS is comprised of a 4u head unit and a 4U JBOD running Centos. Last of course, is a 2U CyberPower UPS which is really just to condition the power and keep brownouts from taking down the system.

I have about a dozen VM’s running between the 2 Xserves. I have each Mac OS X service separated on it’s own installation. This way I can run updates per service. It’s especially nice to have Open Directory separate from less important services. I also have Debian, Centos and OpenBSD VMs running various services. Getting the Xserve 3,1 running ESXi 6 is possible, but more problematic. Now that I have it working, I’m dropping the 2,1’s simply because the processors aren’t multithreaded. I am currently working on a companion article to this one, detailing my experience with the Xserve 3,1, so that information will be available soon.

Storage

1. Don’t use the RAID backplane/interconnect or count on using it. ESXi 6 does not recognize it, and RDM appears to work at first and then will crash your VM and never show up again. You can have it installed in the Xserve without any issue, but you’ll get a lot more mileage out of the hardware if you have the standard backplane/interconnect.

The backplane you want appears periodically on eBay. The part number is: 661-4648

2. If/once you have the non-RAID backplane/interconnect, keep in mind that it is SATAII and will only support 3Gb/s. I am using 3 old 500Gb WD RE3’s, but I’d recommend using some older SSDs that will max out the SATAII interface without paying for speed you can’t use. Be sure you consult the Apple Drive Module compatibility page to make sure you have the right drive caddies. They all fit, but they don’t all work.

Apple Drive Module Compatibility Chart: https://support.apple.com/en-us/HT1219

3. PCIE Flash is a good idea, whether you use it for cache, VM’s or as the ESXi boot disk, it is by far the fastest storage option. I have not invested in it, but the good people at Mushkin have told me their Scorpion PCIE flash will work in ESXi 6. Please contact them yourself though before investing in one of their cards. I have not tested them. While this will give you the best performance out of your Xserve 2,1, it seems like overkill to me, but hey, if you want to push this thing, you might find some meaningful performance gains here.

Mushkin Scorpion PCIE Flash: http://www.poweredbymushkin.com/index.php/products/solid-state-drives.html

4. It might occur to you that replacing the optical drive with an ssd might be a good idea. While MCE Technologies makes “Optibay” for Xserve 2,1, the connection is IDE, so this is not recommended. I also don’t know if ESXi would recognize it. My gut says probably, but again, it’s too slow to be useful. It isn’t that cheap either.

MCE Technologies Optibay: http://store.mcetech.com/mm/merchant.mvc?Store_code=MTOS&Screen=PROD&Product_Code=OBSXGB-XS

Network

There are a lot of options here. You can plug any network card that you want really as long as you can at least find an anecdotal account of it working on ESXi 6.

I have one such anecdote for you, and it is also a strong suggestion. The company Silicom makes/made several models of gigabit NICs which are all incredibly inexpensive, are all over eBay and work in ESXi 6. Buy the 6 port model. It’s cheap and you’ll get great scaling with ESXi load balancing across 8 gigabit ports.

The Xserve 2,1 has the added advantage here of an optional PCIX riser. If your model has one, or you find one for cheap, you can save even more on the Silicom NIC. The PCIE models go for $60-$80 on eBay, while the PCIX models go for $40.

Memory

ECC DDR2 is pretty cheap and easy to find used. I recommend memory4less though. I had to return some RAM that was mislabeled from a random eBay distributor. Memory4less will get it right. http://www.memory4less.com/

Processors

One great perk of the Xserve 2,1, is that you can upgrade the single processor hardware to dual processor. You can pick up an additional processor on eBay for $40 or so, but you’ll need to get a heat sink as well. The single processor units come with a fake aluminum heat sink, but do not use it. You want a copper one. I believe the heat sinks in the Xserve 1,1 are the same. Don’t forget the thermal paste.

Minor Issues

1. The Performance tab throws an error.

2. Not sure about the hardware sensors, but it looks like not everything is working even if it’s showing up. I did not do any testing here.

Stay tune for Part II of John's guest blog post on running ESXi 6.0 on Xserve 3,1.