fusion – Page 8

Community stories of VMware & Apple OS X in Production: Part 8

09.25.2014 by William Lam // 3 Comments

Company: Mid-Pacific Institute (Private School in Hawaii)
Software: VMware vSphere and Fusion
Hardware: Apple Mac Pro

[William] - Hi Derick, I appreciate you taking some time out of your busy schedule to talk with us all the way from Hawaii 🙂 Before we get started, can you quickly introduce yourself?

[Derick] - Sure William. My name is Derick Okihara, and I work for Mid-Pacific Institute. We are a private K-12 institution with about 1600 students. My role here is general IT and server administration. I've been working with computers since I was in high school. I have been a long time Apple user (since //gs), but really started working with them professionally about 10 years ago. We currently have a 1:1 iPad program for the students, and 2:1 iPad+laptops for faculty, so there's a lot of technology to support.

[William] - That’s awesome Derick. Look forward to hearing more about your environment. Speaking of which, I hear you are currently managing some Apple hardware running on VMware? Could you tell us a little bit about the hardware configuration and the VMware software you are currently using?

[Derick] - We are currently using vCenter with ESXi 5.5, we have 2 Mac Pro (5,1s) in a cluster with a Synology 1813+ shared storage. The network storage is connected via the iSCSI software initiator using round-robin. We also use VMware Fusion for the Mac. The Mac Pros have 24GB of RAM and 4 port Intel Gigabit Ethernet cards for a total of 6 ports each.

[William] - What made you decide on using a Synology for shared storage and what configuration/capacity did you go with? Were there any other options you were looking at?

[Derick] - A lot of the decisions for this setup was made on price. How this all started, was that I was asked to create an ESXi server to host a VM Appliance to run our campus wide Informacast speaker system. I had already been planning an ESXI deployment on the Mac, testing on Mac Mini. Instead of building a PC server just for this appliance, I was given the OK to build on an existing Mac Pro, so it could serve multiple purposes.

Being forward thinking, I knew we needed redundancy, so I opted to go for network storage. With a tight budget, and being able to use CPUs we already had, I decided on the Synology 1813+ for it's 4 gigabit ports. This allowed me to later add the 2nd host in our vcenter cluster when we expanded.

[William] - Can you talk a little bit about the type of workloads and applications you are currently virtualizing on the Mac Pro’s and are these all OSX VMs?

[Derick] - Since this is still version 1.0, we aren't heavily taxing our cluster. Right now it hosts 5 VMs (2 Virtual Appliance, 2 OS X Server, 1 Windows Server). I'd want to add more RAM as well, OS X VMs are very RAM hungry.

The OS X servers are a student file server (AFP/SMB) and an Apple Caching server / Munki repository. The Windows server is mostly a test bed, the Virtual Appliances are the aforementioned Informacast manager and VCenter Appliance.

Here is a picture Derick's two Apple Mac Pros:

[William] - The Mac Pro’s have a maximum amount of memory that it supports, do you plan on expanding the infrastructure to accommodate additional workloads or would you be looking at upgrading to the latest generation of Mac Pro (black)?

[Derick] - Honestly, with our current needs and budget, I think I would be looking at the next generation Mac Mini combined with some sort of PCI-E enclosure. Like the Sonnet XMac server. I know the Mini will likely not be fully supported, but I like what i've seen on virtuallyGhetto with the current generation 🙂 That is, if the next gen Mac Mini supports 32GB of RAM!

[William] - Very cool! So from your point of view, you would rather have more Mac Mini’s than a couple of Mac Pro’s? It sounds like cost plays a huge factor, but what other constraints or requirements that is making you lean more towards Mac Mini’s instead of going to a new Mac Pro which can get up to 64GB of memory and 6-Core CPU?

[Derick] - Footprint - the Mac Pros we currently have take up a large amount of rack space. Even the new mac pros would not be rack mountable without an additional enclosure. For us, having 3 x Mac Mini with 32GB of RAM would be ideal price/performance ratio. (We have a 3 CPU license). Eventually our Mac Pro 5,1's will die, so I'm already thinking about what's next. Having 3 x Mac Mini servers in a cluster, that takes up only 3U would be pretty sweet!

[William] - Speaking of support, did you purchase any type of extended contracts with Apple on the hardware or you going to deal with them on a case by case basis? Have you had any issues with failing hardware on the Mac Pro’s?

[Derick] - We only had the initial Apple Care (now since expired). We have 1 spare Mac Pro currently running other loads but that could be migrated in the event that we have a hardware failure. We have not had many issues on the Mac Pro 5,1s other than internal hard drive failure. They've been rock solid.

[William] - Has there been any interesting issues or challenges you had faced while setting up this infrastructure? Either the hardware, software or managing the VMs?

[Derick] - This whole process was a learning experience for me. At a previous job, I had inherited an ESXI server running multiple CentOS and Ubuntu VMs, but I had never set it up myself, let alone on a Mac. Thanks to the multiple resources on the web (Rich Trouton's blog, VirtuallyGhetto, and a P2V script from Alan Gordon at MacSysAdmin) the process and gotchas were mostly done before me

The biggest challenge for me was configuring the Synology for iSCSI-round robin. In my research I found that one could utilize multiple gigabit connections with Multi-path IO for higher bandwidth. After lots of configuring and back and forth with Synology / VMware support, I finally found the proper settings that allowed me to utilize more than 1 gigabit link.
However, after I updated to ESXi 5.5, it broke.

I was stuck, because I needed to upgrade to 5.5 in order to run an OS X caching server (12-character serial number fix in 5.5). But Synology said the 1813+ was not compatible with 5.5 and would not help me. Long story short, one of my hosts is running 5.5 (with OS X Caching server) and the other host is running 5.1 (file services) because it needs the greater throughput.

[William] - Derick, I want thank you for taking the time and sharing with us your experiences with managing VMware and Apple hardware. Before I let you go, do you have any tips for our readers that may be in a similar environment (academic) and needs to build out an infrastructure to support their end users? Any gotchas or things you would recommend if you had to do this all over again?

[Derick] - Anyone looking to reduce their machine footprint should definitely look into virtualization. VMware has very attractive pricing for the EDU market if you're looking to build a cluster with high availability, or you can run a single host for free. The best piece of advice I can give is just to test thoroughly. Virtualization is very complicated, and combines a multitude of areas of expertise (Storage, Networks, Workloads, and ESXI platform itself). It can be daunting but it's very rewarding. If you get stuck, just ask William on Twitter @lamw, jk

If you are interested in sharing your story with the community (can be completely anonymous) on how you use VMware and Mac OS X in Production, you can reach out to me here.

Community stories of VMware & Apple OS X in Production: Part 3

08.11.2014 by William Lam // 2 Comments

In this third post of the community stories, I had the pleasure to interview with Blake Garner who shares with us how he leverages VMware and Apple Technologies in his production environment.

Company: Adobe
Product: VMware Fusion
Hardware: Apple Mac Mini & Mac Pro

[William] - Hi Blake, thanks for your comment on Twitter today, it sounds like this blog series might be useful for your organization. I understand that you have some experiences working with VMware and Apple OS X that you can share with the community? Before we get started, can you quickly introduce yourself?

[Blake] - Sure, I have been working at Adobe for 11 years in the Seattle location. Started off as a Dev/Test lab administrator focused on Macs, Printers & color management. Been working with VMware seriously starting with VMware Fusion 1.0 and Lab Manager 2.x.

[William] - Wow, a VMware old timer 🙂 So, I take it from your Tweet that you must be doing some really cool stuff with VMware and Apple OS X? Can you describe the environment and how you leverage these technologies together?

[Blake] - We provide "bare metal" Mac Mini's and Mac Pro's to our users. They are mostly developers and testers of Adobe desktop software. The process involves approvals and users requesting a Mac from our web portal.

Once users have the Mac they load VMware Fusion which is the majority of the time. We do not have a centralized command and control of VMware Fusion. A lot of the teams then hook up VMware Fusion to our testing harnesses. Things like Jenkins and a lot of custom code to manage the VM's. We looked into using ESXi/vSphere/vCloud on the Mac but due to the Apple EULA restrictions it just wasn't a good fit.

[William] - Very cool! This sounds like you’re offering Mac as a Service for your internal customers? How do you go about managing the requests of which Mac Mini or Mac Pro are leased out? Is all this custom software Adobe has built?

[Blake] - Yes we do both Mac and x86 systems (Windows & Linux) as part of our bare metal offering on our internal IT Cloud. The bare metal part is custom code. We have rolled our own deployment systems to go along with it as well. Users access one portal that currently spans vCloud, AWS & bare metal along with a few other services.

[William] - You mention that the majority of users install VMware Fusion on the “bare metal” once they get their assigned Mac Mini or Mac Pro. What is VMware Fusion being used for?

[Blake] - Lot's of automation. vmrun gets a good workout here. One team has a "Test as a Service" that can control VMware Fusion and rollback snapshots to provide a clean testing state. The consumers of the Mac's enlist them into their own existing automation systems. Often if you look at the VM's you will see Creative Cloud applications running through test cases super fast. Builds of products for Mac OS and iOS also happen.

[William] - Ah, so they are leveraging VMware Fusion as a platform to be able to run sort of a “Continuous Integration” build environment for your internal Mac OS X and iOS builds? This sounds like it could be quite challenging to manage, have the end users had any issues or have they automated everything all already?

[Blake] - In a larger software company you end up with a number of approaches. I did not get involved at the build level enough to speak on that. Adobe IT tends to focus on the common services that all the teams can use. In general there is a lot of automation herding going on..

[William] - Gotcha. So, going back to physical Mac Mini and Mac Pro, roughly how many are you managing and what type of configurations did you spec out? Any particular reasoning for choosing these configurations?

[Blake] - Our initial launch of the bare metal mac service has 50 Mac Pro's and 50 Mac Minis split equally between two sites. This service is just a couple months old and we expect it to grow fast as the engineers figure out they can get Mac's if their manager approves a monthly fee to their cost center.

We have one config for Mac Pro and Mac Mini. The Mac Mini 6,2 has 250GB SSD and i7 2.6GHz with 16GB RAM. The Mac Pro 6,1 6-Core E5 3.5 GHz with 500GB SSD, 16GB RAM and lower end GPU's as those are not used that much. We really just looked for a sweet spot that matches what Creative Cloud needs

Here is a picture of one of the Mac Pro racks courtesy of Blake:

[William] - How do you monitor for hardware issues and what is the most common issues have you seen for both the Mac Mini and Mac Pro?

[Blake] - That is a real challenge with Apple hardware. With no out of band management we rely on our staff to troubleshoot via KVM or in person if needed. Pre-release software can really crash a system and that often needs a finger on the power button. I'm looking into using our remote controlled PDU to power cycle systems via the portal. If a system is truly FUBAR we just give people a new one.

In a lot of cases we can simply re-image the system via netboot and add it back to the available pool. VMware Fusion comes really in handy for developing our custom netboot environments as well. Netbooting an OS X VM's is one of my favorite features of VMware Fusion.

[William] - I can definitely see it being easier to re-image than to troubleshoot unless you are seeing a consistent issue. Curious, what version of Mac OS X and VMware Fusion are you using today? In addition to snapshot & Netbooting, any other useful features VMware Fusion provides?

[Blake] - VMware Fusion 6.0.3 is what we support today. On the hardware we encourage people to use the latest release version of OS X. In the VM's it depends on what kind of testing. Validation of pre-release OS's is a big chunk of the testing work. For OS X that happens in VMware Fusion and other x86 it's done in vCloud and some VMware Workstation. The automation capabilities of VMware Fusion are a key component. That's what gives us the upper hand in managing these virtual systems.

[William] - Hey Blake, I really appreciate you taking the time and sharing with us on how Adobe leverages VMware and Mac OS X. Before I let you go, do you have any words of advice for other customers looking to provide a similar type of environment? Any gotchas or things you would change if you could start fresh?

[Blake] - Go talk to those people who have offices crammed full of systems and find out what they are doing. You can often find the common requirements and start building against that. Don't try and dictate the whole solution to engineers. Once they are happy customers it's much easier to get them onboard with centralized services. Provide a set of functional services and engineers will pick it up quickly. Keep adding services and they will grow along with you.

Things that I would change all end up on our features roadmap. I have my eyes on providing an API to access to re-image or reserve bare metal systems and providing vagrant along with VMware Fusion for automation.

If you are interested in sharing your story with the community (can be completely anonymous) on how you use VMware and Mac OS X in Production, you can reach out to me here.

How to run Nested Mac OS X guest on ESXi VM on top VMware Fusion?

08.08.2014 by William Lam // 1 Comment

You might be asking, why would anyone want to do this? Well, luckily this is not a "because you can" type of answer but was it was an interesting solution that one of our VMware Engineers (Darius) had shared with me after helping out on this VMTN Community forum thread.

The user was running VMware Fusion on his physical Mac OS X system and wanted to be able to test OS X Mavericks under ESXi. Not having a physical ESXi host to test with, the next best thing was to run a ESXi VM under VMware Fusion and then run the Mavericks guest on top of that.

Here is a quick diagram of the user setup:

The issue with just simply doing this is that for a Mac OS X guest to properly run on ESXi, the underlying hardware must be Apple Hardware. The reason for this is not a technical challenges, but rather a legal one per Apple's EULA. The way in which ESXi detects that the underlying hardware is Apple is by checking whether Apple's SMC (System Management Controller) is available.

In the scenario above, the Nested ESXi VM is not automatically passing through the SMC from the physical Mac OS X system and hence the Mac OS X VM at the very top of the stack will not properly function. The solution that Darius found was to add the following two Advanced VM Settings (VMX) entries to the ESXi VM:

smc.present = "TRUE"
smbios.reflectHost = "TRUE"

This will allow the passing of the underlying SMC up into the Nested ESXi VM which will then allow Mac OS X guest VMs to properly function. We can also confirm this by check the Nested ESXi MOB by pointing a browser to the following URL: https://[ESXI-IP]/mob/?moid=ha-host&doPath=hardware

If you did not add the two entries above, then the smcPresent property would show up as false. In our case, we did add the following two entries and we now run our Mac OS X Guest. Here are a couple of screenshots of performing this on my iMac at home running the same exact configuration:

Thanks Darius for sharing this with me and the community! I am sure this will come in handy for anyone wanting to test Mac OS X guests under ESXi but do not have a physical ESXi host and can easily substitute using VMware Fusion.