WilliamLam.com

EMC Project OnRack now RackHD

by // 1 Comment

Back in May, EMC announced a new initiative at EMC World called Project OnRack which had an ambitious goal of providing a new software abstraction layer that would sit on top of existing "industry standards" for server out-of-band management. Standards such as IPMI, CIM, SMI-S and CIM-SMASH to just name a few were supposed to help IT administrators manage and operate the life-cycle of their physical servers. Instead, we ended up with even more complexity and inconsistency due to the different implementations of these "industry standards" across vendors and sometimes even within the same vendor. Trying to keep firmware, BIOS, hardware drivers, etc. up to date across different hardware platforms from the same vendor in a consistent and automated fashion was already painful enough. As If this was not already challenging enough, try doing this for a mix of hardware platforms across different vendors and you have just given your operational and datacenter team a never ending nightmare.

Frankly, I am pretty surprised that it has taken us this long to finally tackle this problem. This is something we have needed for quite some time now and I still remember the early days as an admin trying to script around the inconsistencies of IPMI to configure things like asset tags and serial numbers across different hardware platforms.

In my opinion, having a consistent and programmable interface to this low level of hardware is a critical component to a Software-Defined Datacenter and has often been overlooked. Kudos to EMC for taking on this initiative and more importantly driving this change through open-source and the community in mind.

Since the announcement back in May, things have been been pretty quiet about OnRack, until recently that is. I was listening to a recent episode of The Hot Aisle Podcast with guest Brad Maltz of EMC talking about Hyper-Converged Infrastructure. Among the different topics discussed, OnRack was brought up along with dis-aggregated hardware/infrastructure where individual compute resources can scale up independently of each other. There were a couple of nice tidbits mentioned on the podcast. First, it looks like OnRack which was the internal EMC project name has now been renamed to RackHD as the external project name. Second, it looks like the RackHD repo is already on Github with some initial content including some pretty detailed documentation on the architecture and components which can be found here.

The OnRack project looks to be made up of the following sub-projects per the documentation:

  • on-tftp - NodeJS application provided TFTP service integrated with the workflow engine
  • on-http - internal HTTP REST API interfaces integrated with the workflow engine
  • on-syslog - syslog endpoint integrated to feed data into workflow engine
  • on-taskgraph - NodeJS application providing the workflow engine
  • on-dhcp-proxy - NodeJS application providing DHCP proxy support integrated into the workflow engine
  • onserve - OnServe Engine
  • core library - Core libraries in use across NodeJS applications
  • task library - NodeJS task library for the workflow engine
  • tools - Useful dev tools for running locally
  • webui - Initial web interfaces to some of the APIs - multiple interfaces embedded into a single project
  • integration tests - Integration tests with code for deploying and running those tests, as well as the tests themselves
  • statsd - A local statsD implementation that makes it easy to deploy on a local machine for capturing application metrics

Brand mentioned that many of the Github repos are still marked private as they are still working through the process of releasing RackHD to the public. It looks like RackHD and all relevant repos are now all open source as of Monday Nov 2nd, for more details please visit the Github repo here. I am definitely excited to see how this project will evolve with the larger community and some of the new innovations which will be unlocked due to this barrier being removed. Hopefully we will see positive collaboration from other hardware vendors which will help us move forward and finally solve this problem once and for all! I can already see huge benefits for software only vendors like VMware who can integrate RackHD directly into provisioning tools like Auto Deploy or configuration management tools like Puppet, Chef and Ansible for example. It will also be interesting to see how other startups in this area like NodePrime and another stealth company, who is also working on solving a similar problem and whether they would leverage RackHD or not.

Hope to see you at the Nordic VMUG UserCon in Dec!

by // Leave a Comment

In case you missed it, a couple of weeks back, Liselotte Foverskov one of the leaders for the Nordic VMUG had announced some of the speakers for the upcoming Nordic VMUG UserCon taking place on Dec 1st at the Scandic Hotel in Copenhagen.

I was very fortunate to have been invited out to speak at the event. As you can see from the tweet above, I will also be joined by several of my colleagues from VMware like Paul Strong, Duncan Epping, Cormac Hogan, Paudie O'Riordan and Joerg Lew who will be covering a variety of topics, so there should be something for everyone. This will be my first time visiting Denmark and I am looking forward to meeting everyone and I hope that you can make it out. I also hear Liselotte has an amazing keynote speaker lined up that you will not want to miss! For more details about the event, be sure to visit Nordic VMUG site here.

Below is the topic that I will be presenting on but feel free to find me at the event if you are interested in talking about other topics that you may have on your mind.

Session Title: 

Harnessing the power of the vSphere API for the everyday Admin 

Session Abstract: 

In this session, you will get a brief introduction to the vSphere API and learn how to use and navigate it without needing to become a developer. Building on this foundation, we will then explore some of the different methods and tools that are available to a variety of audiences (vSphere Administrators, Developers, etc) when Automating against the vSphere API. Finally, we will take a look at some practical examples that demonstrates the vSphere API in action which you can then take back with you when you leave the session. 

New method of enabling Multiwriter VMDK flag in vSphere 6.0 Update 1 (UI + API)

by // 22 Comments

Prior to vSphere 6.0, in order for multiple Virtual Machines to share a VMFS-backed VMDK, the Multiwriter VMDK flag must be enabled, which is accomplished by adding a specific VM Advanced Setting as shown in this VMware KB 1034165. For customers who were accustomed to this old method, you may have found that this option no longer works. This was true regardless if you had used the vSphere Web/C# Client or the vSphere API to apply the configuration.

To provide for a better user experience, this behavior was changed in vSphere 6.0 and a new API was introduced for enabling and disabling the Multiwriter VMDK flag. In vSphere 6.0, there is now a new sharing attribute on the Virtual Disk backing property which accepts one of two values: sharingMultiWriter or sharingNone for specifying the Multiwriter flag. In my opinion, this is a positive change as we too often rely on the VM Advanced Setting as a generic "catch all" for enabling or configuring various settings versus adding proper APIs to a VM.

Although there is now a proper API which will can help enable new Automation use cases, one thing that was still lacking was an easy way to enable the Multiwriter VMDK flag using the UI. In vSphere 6.0 Update 1, we have now introduced a new UI dropdown option called "sharing" in the vSphere Web Client for configuring the Multiwriter VMDK flag which can be found in the Virtual Disk section when editing a VM as shown in the screenshot below.

Screen Shot 2015-10-16 at 10.19.05 AM
Note: The new Sharing property is only available in the vSphere Web Client UI and is not available in the vSphere C# Client. If you need to configure the Multiwriter VMDK flag and do not have access to the vSphere Web Client, you can use the vSphere API to help automate this configuration change.

UPDATE (06/27/16) - Created two scripts which now cover scenarios where VM is online and/or offline.

For those interested in Automating the Multiwriter VMDK flag, I have created two PowerCLI scripts called: configureMultiwriterVMDK.ps1 (offline VM configuration) and addMultiwriterVMDK.ps1 (online VM configuration) which demonstrates this new vSphere API.

The first script configureMultiwriterVMDK.ps1 allows you enable the Multiwriter Flag for an existing VMDK that has already been added to a VM. This operation must be done while the VM is powered off and to use the script you will need to specify the name of the VM as well as the label of the VMDK in which you wish to enable the Multiwriter VMDK flag (e.g. Hard disk 2). Below is an example of running the script.

Screen Shot 2015-10-16 at 8.24.46 PM
The second script addMultiwriterVMDK.ps1 allows you to hot-add a new VMDK and enables the Multiwriter Flag to a VM. This operation is done while the VM is powered on which is a common workflow for customers needing to hot-add storage to an existing Cluster solution such as Oracle RAC for example all while the system is running. To use the script, there are a few variables you will need to edit:

  • vmName - The name of the VM you wish to perform th operation on
  • vmdkFileNamePath - This is the full datastore path to the name of the underlying VMDK. See the script for more information but the syntax will look like "[datastore-name] vm-home-dir/vmdk-name.vmdk"
  • diskSizeGB - The capacity of the VMDK to add (GB)
  • diskControllerNumber - The SCSI controller number (0-3)
  • diskUnitNumber - The Unit number (0-16)