Home Lab – Page 41

Subscribe to vGhetto Nested ESXi Template Content Library in vSphere 6.0

04.08.2015 by William Lam // 23 Comments

During the early development of vSphere 6.0, one of the features that I got an early sneak preview of was the Content Library which originated from vCloud Director's Content Catalog capability and has now been pushed down into the core vSphere platform as part of VCD's "Convergence" plan.

Although there are some initial limitations with this first release of Content Library such as not being able to mount an ISO directly from the Content Library as example, which I do agree it should have just worked and not requiring a manual datastore browse for this operation to work. I know the Engineering team is aware of this as it was something our team had also provided feedback among other things, so hopefully this will be fixed very shortly.

Having said that, I do see a huge potential with the Content Library and all the interesting use cases it can enable not only for vSphere but also for other products such as vRealize Automation as well as vCloud Air. One area that caught my attention when I first heard about Content Library is the the fact that the publishing and subscription capability works over simple HTTP(s). I immediately had a light-bulb moment and thought would it not be cool if you could have a custom Content Library that would be hosted on some external cloud storage such as Amazon S3 as en example and be able to publish that so others could subscribe to it in their vSphere environment?

Fortunately, because Content Library works over standard HTTP(s) and with the help of one of the Content Library Engineers I was able to create my very own vGhetto Nested ESXi Template Content Library for both vSphere 6.0 and vCloud Directory based environments which is currently being hosted on Amazon S3.

vSphere: https://s3-us-west-1.amazonaws.com/vghetto-content-library/lib.json
vCD: https://s3-us-west-1.amazonaws.com/vghetto-content-library-vcd/lib.json

The library contains all of my Nested ESXi / VSAN OVF Templates that I have created over the years and by publishing them in in my public Content Library, anyone can now easily subscribe and pull down the latest OVF templates to deploy directly in their vSphere/vCD environment. You no longer have to manually download the OVFs and as I add new content, the Content Library will automatically synchronize the changes to your local environment.

The vGhetto Nested ESXi Template Content Library currently contains the following 7 OVF's which total to 1.43MB:

Nested-ESXi-3-Node-VSAN-6.0-All-Flash-Template
Nested-ESXi-3-Node-VSAN-6.0-Template
Nested-ESXi-3-Node-VSAN-Template
Nested-ESXi-32-Node-VSAN-Template (not available when subscribing from VCD)
Nested-ESXi-6-Node-VSAN-6.0-FD-Template
Nested-ESXi-64-Node-VSAN-6.0-Template (not available when subscribing from VCD)
Nested-ESXi-VM-Template

Here are the instructions for subscribing to my vGhetto Content Library using the vSphere 6.0 Web Client:

Step 1 - In the main Home page, click on the Content Library icon

Step 2 - Create a new Content Library which will be used to subscribe to my vGhetto Content Library.

Step 3 - Select "Subscribed content library" and enter the following URL: https://s3-us-west-1.amazonaws.com/vghetto-content-library/lib.json and make sure to select "Download all library content immediately". There is currently a limitation in the way the metadata is read from OVFs that prevents the on-demand setting to be used when subscribing to 3rd party Content Libraries.

Step 4 - Select the storage which will be backing your Content Library. This can be either a local filesystem path on your vCenter Server or by using a vSphere Datastore.

Step 5 - If the new Content Library had been successfully created, the content will start to synchronize to your local system and once the OVF/VM icon appears, then you know the Nested ESXi / VSAN OVFs templates are ready to be deployed in your environment as seen in the screenshot below.

Note: You will notice that even though we are pulling down all seven OVF templates, they are only consume a measly 1.43MB as these are empty VMs shells and I am also using the OVF Dynamic Disks feature which allows you to specify the size of the VMDK upon deployment.

UPDATE (4/24) - It turns out that 3rd Party Content Libraries can also be subscribed from within vCloud Director but because it does not support Dynamic Disks, the synchronized will fail. I have created a separate library specifically optimized for vCD which you can find the URL at the top and you can then subscribe to my vGhetto Content Library as seen in the screenshot below. You will notice that the 32 and 64 Node VSAN Template is not available and the reason for this is that apparently there is a limitation in the number of elements it can parse.

This is a pretty powerful feature in my opinion and I can already see custom content libraries not only from VMware but also from our partner eco-systems providing their latest solutions (Virtual Appliances) as well as other file content as Content Library can store pretty much anything. In a future blog post, I will go through the details on how you can create your own custom Content Library, so stay tune. In the mean time, if you are using vSphere 6.0 and and would like to be able to quickly pull down the latest Nested ESXi / VSAN OVF templates, be sure to subscribe to my vGhetto Content Library!

Home Labs made easier with VSAN 6.0 + USB Disks

03.04.2015 by William Lam // 23 Comments

VSAN 6.0 includes a large number of new enhancements and capabilities that I am sure many of you are excited to try out in your lab. One of the challenges with running VSAN in a home lab environment (non-Nested ESXi) is trying to find a platform that is both functional and cost effective. Some of the most popular platforms that I have seen customers use for running VSAN in their home labs are the Intel NUC and the Apple Mac Mini. Putting aside the memory constraints in these platforms, the number of internal disk slots for a disk drive is usually limited to two. This would give you just enough to meet the minimal requirement for VSAN by having at least a single SSD and MD.

If you wanted to scale up and add additional drives for either capacity purposes or testing out a new configurations, you are pretty much out of luck, right? Well, not necessary. During the development of VSAN 6.0, I came across a cool little nugget from one of the VSAN Engineers where USB-based disks could be claimed by VSAN which could be quite helpful for testing in a lab environment, especially using the hardware platforms that I mentioned earlier.

For a VSAN home lab, using cheap consumer USB-based disks which you can purchase several TB's for less than a hundred dollars or so and along with USB 3.0 connectivity is a pretty cost effective way to enhance hardware platforms like the Apple Mac Mini and Intel NUCs.

Disclaimer: This is not officially supported by VMware and should not be used in Production or evaluation of VSAN, especially when it comes to performance or expected behavior as this is now how the product works. Please use supported hardware found on the VMware VSAN HCL for official testing or evaluations.

Below are the instructions on how to enable USB-based disks to be claimable by VSAN.

Step 1 - Disable the USB Arbitrator service so that USB devices can been seen by the ESXi host by running the following two commands in the ESXi Shell:

/etc/init.d/usbarbitrator stop
chkconfig usbarbitrator off

Step 2 - Enable the following ESXi Advanced Setting (/VSAN/AllowUsbDisks) to allow USB disks to be claimed by VSAN by running the following command in the ESXi Shell:

esxcli system settings advanced set -o /VSAN/AllowUsbDisks -i 1

Step 3 - Connect your USB-based disks to your ESXi host (this can actually be done prior) and you can verify that they are seen by running the following command in the ESXi Shell:

vdq -q

Step 4 - If you are bootstrapping vCenter Server onto the VSAN Datastore, then you can create a VSAN Cluster by running "esxcli vsan cluster new" and then contribute the storage by adding the SSD device and the respective USB-based disks using the information from the previous step in the ESXi Shell:

esxcli vsan storage add -s t10.ATA_____Corsair_Force_GT________________________12136500000013420576 -d mpx.vmhba32:C0:T0:L0 -d mpx.vmhba33:C0:T0:L0 -d mpx.vmhba34:C0:T0:L0 -d mpx.vmhba40:C0:T0:L0

If we take a look a the VSAN configurations in the vSphere Web Client, we can see that we now have 4 USB-based disks contributing storage to the VSAN Disk Group. In this particular configuration, I was using my Mac Mini which has 4 x USB 3.0 devices that are connected and providing the "MD" disks and one of the internal drives that has an SSD. Ideally, you would probably want to boot ESXi from a USB device and then claim one of the internal drives along with 3 other USB devices for the most optimal configuration.

As a bonus, there is one other nugget that I discovered while testing out the USB-based disks for VSAN 6.0 which is another hidden option to support iSCSI based disks with VSAN. You will need to enable the option called /VSAN/AllowISCSIDisks using the same method as enabling USB-based disk option. This is not something I have personally tested, so YMMV but I suspect it will allow VSAN to claim an iSCSI device that has been connected to an ESXi host and allow it to contribute to a VSAN Disk Group as another way of providing additional capacity to VSAN with platforms that have restricted number of disk slots. Remember, neither of these solutions should be used beyond home labs and they are not officially supported by VMware, so do not bother trying to do anything fancy or running performance tests, you are just going to let your self down and not see the full potential of VSAN 🙂

Thunderbolt Storage for ESXi

01.21.2015 by William Lam // 47 Comments

A question that I frequently receive is whether ESXi supports Thunderbolt-based storage devices. This is especially interesting for folks running ESXi on an Apple Mac Mini due to the limited number of IO connections the Mac Minis' have. If you look on VMware's HCL, you will not find any supported Thunderbolt Storage devices nor are there any that are being actively tested with ESXi, at least as far as I know.

Having said that, generally speaking from an ESXi host point of view, the Thunderbolt interface is just seen as an extended PCIe bus. This means that whatever storage device is connected on the other end can work with ESXi as long as there is a driver in ESXi that can communicate with that devices. This is analogous to having a RAID card and having the proper device driver on ESXi to see its storage.

Even though VMware is not actively testing Thunderbolt-based storage devices, there are a few folks out in the community who have and have been successful. I wanted to share these stories with the community for those that might be interested in this topic and hopefully others who have had similar success can also share more details about their setup.

UPDATE (09/12/16) - ESXi Thunderbolt Driver to Fibre Channel Storage from ATTO

Disclaimer: All solutions listed below are from the community and decisions to purchase based on these solutions will be at your own risk. I hold no responsibility if the listed solutions do not work for whatever reason.

Solution #1 - Pegaus R6 Thunderbolt Storage Enclosure

This was the first Thunderbolt storage device that I had ever seen confirmed publicly to work with ESXi after installing a STEX driver VIB. You can find more details here.

Solution #2 - Sonnet Echo Express III-R Rackmount Thunderbolt 2 Expansion Chassis & RacMac Mini Enclosure

This next solution was recently shared with me from Marc Huppert who has recently expanded his home lab. Marc combined a Thunderbolt expansion chassis with a Mac Mini chassis to exposed Fibre Channel storage to his Mac Minis. You can find more details here.

Solution #3 - xMac Mini Server Enclosure

I came across this solution while searching online which also uses another Mac Mini Thunderbolt expansion chassis connected to Fibre Channel based storage. You can find more details here.

Solution #4 - Sonnet xMac Pro Server Enclosure

Thanks to Joshua for sharing his solution. You can find more details in the comments here.

Solution #5 - LaCie Rugged Thunderbolt drives

Thanks to Philip for sharing his solution. You can find more details in the comments here.

Solution #6 - ARC-8050T2 Thunderbolt 2 RAID

Thanks to Jason for sharing his solution. You can find more details in the comments here.

Solution #7 - Another Sonnet xMac Pro Server Enclosure + EMC VNX

Thanks to Johann for sharing his solution. You can find more details here.

Solution #8 - LaCie Little Big Disk Thunderbolt 2 with 2013 Mac Pro w/ESXi 6.0

Thanks to Thomas for sharing his solution. You can find more details here.

Solution #9 - Sonnet Echo Express III with Mac Pro 6,1 and ATTO ExpressSAS H680 w/ESXi 6.0

Thanks to Grasshopper for sharing details here and here.

Solution #10 - OWC ThunderBay 4 RAID 4-Bay External Drive w/Dual Thunderbolt 2

Thanks to Gregg Green for sharing his use of the Thunderbay with Mac Mini 2012

If there are other Thunderbolt-based storage devices that you or others have had success with ESXi, feel free to leave a comment with details and I will add it to the post. If there are any Thunderbolt storage device vendors that would like to send me a demo unit, I would be more than happy to give the system a test to see if it works with ESXi 🙂