EMC Project OnRack now RackHD

Back in May, EMC announced a new initiative at EMC World called Project OnRack which had an ambitious goal of providing a new software abstraction layer that would sit on top of existing "industry standards" for server out-of-band management. Standards such as IPMI, CIM, SMI-S and CIM-SMASH to just name a few were supposed to help IT administrators manage and operate the life-cycle of their physical servers. Instead, we ended up with even more complexity and inconsistency due to the different implementations of these "industry standards" across vendors and sometimes even within the same vendor. Trying to keep firmware, BIOS, hardware drivers, etc. up to date across different hardware platforms from the same vendor in a consistent and automated fashion was already painful enough. As If this was not already challenging enough, try doing this for a mix of hardware platforms across different vendors and you have just given your operational and datacenter team a never ending nightmare.

Frankly, I am pretty surprised that it has taken us this long to finally tackle this problem. This is something we have needed for quite some time now and I still remember the early days as an admin trying to script around the inconsistencies of IPMI to configure things like asset tags and serial numbers across different hardware platforms.

OnRack http://t.co/I6dpMSPgSB Interesting initiative from EMC. Something we've needed for a LONG time! Reminds me of few startups doing same

— William Lam (@lamw) May 7, 2015

In my opinion, having a consistent and programmable interface to this low level of hardware is a critical component to a Software-Defined Datacenter and has often been overlooked. Kudos to EMC for taking on this initiative and more importantly driving this change through open-source and the community in mind.

Since the announcement back in May, things have been been pretty quiet about OnRack, until recently that is. I was listening to a recent episode of The Hot Aisle Podcast with guest Brad Maltz of EMC talking about Hyper-Converged Infrastructure. Among the different topics discussed, OnRack was brought up along with dis-aggregated hardware/infrastructure where individual compute resources can scale up independently of each other. There were a couple of nice tidbits mentioned on the podcast. First, it looks like OnRack which was the internal EMC project name has now been renamed to RackHD as the external project name. Second, it looks like the RackHD repo is already on Github with some initial content including some pretty detailed documentation on the architecture and components which can be found here.

The OnRack project looks to be made up of the following sub-projects per the documentation:

• on-tftp - NodeJS application provided TFTP service integrated with the workflow engine
• on-http - internal HTTP REST API interfaces integrated with the workflow engine
• on-syslog - syslog endpoint integrated to feed data into workflow engine
• on-taskgraph - NodeJS application providing the workflow engine
• on-dhcp-proxy - NodeJS application providing DHCP proxy support integrated into the workflow engine
• onserve - OnServe Engine
• core library - Core libraries in use across NodeJS applications
• task library - NodeJS task library for the workflow engine
• tools - Useful dev tools for running locally
• webui - Initial web interfaces to some of the APIs - multiple interfaces embedded into a single project
• integration tests - Integration tests with code for deploying and running those tests, as well as the tests themselves
• statsd - A local statsD implementation that makes it easy to deploy on a local machine for capturing application metrics

Brand mentioned that many of the Github repos are still marked private as they are still working through the process of releasing RackHD to the public. It looks like RackHD and all relevant repos are now all open source as of Monday Nov 2nd, for more details please visit the Github repo here. I am definitely excited to see how this project will evolve with the larger community and some of the new innovations which will be unlocked due to this barrier being removed. Hopefully we will see positive collaboration from other hardware vendors which will help us move forward and finally solve this problem once and for all! I can already see huge benefits for software only vendors like VMware who can integrate RackHD directly into provisioning tools like Auto Deploy or configuration management tools like Puppet, Chef and Ansible for example. It will also be interesting to see how other startups in this area like NodePrime and another stealth company, who is also working on solving a similar problem and whether they would leverage RackHD or not.