WilliamLam.com

You are here: Home / ESXi / Supermicro E300-9D (SYS-E300-9D-8CN8TP) is a nice ESXi & vSAN kit

Supermicro E300-9D (SYS-E300-9D-8CN8TP) is a nice ESXi & vSAN kit

by // 33 Comments

Supermicro kits such as the E200-8D is a very popular platform amongst the VMware community and with powerful Xeon-based CPUs and support for up to 128GB of memory, it is perfect for running a killer vSphere/vSAN setup!

Earlier this Fall, Supermicro released a "big daddy" version to the E200-8D, dubbed E300-9D and specifically, I want to focus on the 8-Core model (SYS-E300-9D-8CNTP) as this system actually listed on the VMware HCL for ESXi! The E300-9D can support up to half a terabyte of memory and with the 8-Core model, you have access to 16 threads. The E200-8D is also a supported platform by VMware, you can find the VMware HCL listing here.


I was very fortunate to get my hands on a loaner E300-9D (8-Core) unit, thanks to Eric and his team at MITXPC, a local bay area shop specializing in embedded solutions. In fact, they even provided a nice vGhetto promo discount code for my readers awhile back, so definitely check it out if you are in the market for a new lab. As an aside, when doing a quick search online, they also seem to be the only ones actually selling the E300-9D (8-Core) system which you can find here and in general, they seem to be priced fairly competitively. This is not an endorsement for MITXPC, but recommend folks to compare all prices when shopping online, especially as today is Black Friday in the US and Cyber Monday is just a few days away.

Storage

When I first heard about the E300-9D platform, I was excited for the platform as it listed support for up to two M.2 devices, which would match what the Intel NUC Hades Canyon supported. The other reason is that there are actually a number of supported M.2 NVMe devices listed on VMware's HCL which you can find here. This is actually greats news for those in the market for a small form factor systems that is officially supported by VMware, especially useful for ROBO or Edge-based deployment where space and cost is usually a constraint.

However, upon closer inspection, it looks like the system only supports a single M.2 (M-Key) which is the traditional M.2 SSD devices you would purchase for a platform like the Intel NUC (lower left hand corner of the picture). Support for the other M.2 device is actually using the B-Key form factor (immediately to the right of the M-Key in the picture) and this would be SATA based rather than PCIe.


Having said that, thanks to David Chung who was the first to share the news, you can easily add an M.2 to PCIe add-on card enabling a secondary M.2 NVMe (M-Key) device for just $13 bucks. Here is the Amazon listing, in case folks are interested. This add-on card fits nicely in both the E200-8D and E300-9D (pictured above) and this will give you another M.2 NVMe using PCIe!

Several folks have also shared that Supermicro makes a PCIe add-on card that supports dual M.2 that can also be used for these 1U low profile systems, which is nice from a support standpoint with just having a single vendor for the system itself.

Once ESXi is installed, both of my NVMe M.2 devices were automatically detected as shown in the output below. I also used a SATADOM for installing ESXi (pictured upper right hand corner) which not only gives me a higher endurance device, but it also provides a place for me to store vSAN tracefiles and coredumps.


You can run standard VMFS, but with two of these devices, it would be a waste not to run vSAN 🙂 which of course I had setup as you can see from the screenshot below.

Networking

The networking on the E300-9D has also been significantly upgraded compared to the E200-8D which comes with 1 x 1GbE for the BMC, 2 x 1GbE and 2 x 10GbE (RJ45 or SFP+). The E300-9D includes the standard 1 x GbE for BMC, but now includes 4 x 1GbE, 2 x 10GbE (RJ45) and 2 x 10GbE (SFP+), this should be plenty of ports for anyone wanting to do some serious NSX networking!


For those interested:

  • 4 x 1GbE is based on Intel I350 and uses the igbn driver
  • Both 2 x 10GbE (RJ45) and 2 x 10GbE (SFP+) is based on Intel X722 and uses the i40en driver

All network adapters were also automatically detected by ESXi, no additional drivers or tricks are required.

Noise

Compared to the E200-8D, it is definitely a bit more louder when the system boots up but then idles out with a normal hum. Paul over at TinkerTry did a review of the 4-Core system which includes noise analysis which you can check out here. Zack Widing who also works for VMware has been spending quite a bit of time working with the E200-8D, he recently shared on Twitter that you can purchase Noctura silent fans that work nicely with both E200/E300 systems.

Size

Compared to its little brother E200-8D, the E300-9D is definitely bigger but trying to get a sense of how large the unit is not always an easy thing if you do not have it in front of you. I thought it would be nice to compare the new E300-9D to a number of other systems the community is already familiar with.

Top to bottom:

  1. Intel NUC 6th Gen
  2. Intel NUC Hades Canyon
  3. Apple Mac Mini 5,3
  4. Supermicro E200-8D
  5. Supermicro E300-9D

BOM

  • 1 x Supermicro E300-9D (SYS-E300-9D-8CN8TP) which MITXPC sells here. (vGhetto Discount Code)
  • 1 x M.2 to PCIe Add-On Card
  • Any supported LRDIMM memory (I had 128GB from a previous setup using Samsung 32GB DIMM)
  • Any supported M.2 NVMe storage (I had 2 x WD which I had pulled from my Hades Canyon setup)

Summary

The E300-9D is definitely a powerful platform that packs a ton of resources in a relatively small amount of space. Depending on your goals, it may be a nice home lab setup (but I suspect it might be out of most folks budget) but where a system like this really shines would be for the Enterprise, especially for ROBO/Edge-based deployments. The other huge benefit is that this system is on VMware's HCL and is fully supported including networking and storage! If you are in the market for a small form factor system that packs a punch, definitely take a look at this system and who knows, this might even make a nice vSAN Ready Node if you ask me 😀

Additional Notes

UPDATE (01/14/19) - Using the Supermicro AOC, you can get two additional M.2 devices, including Intel's latest M.2 4801x Optane devices. For more details, please take a look at this blog post here.

The E300-9D also comes with two U.2 connectors (located at the very back of the board pictured above) which in theory can then be connected to supported SSD devices for use with vSAN. However, looking at the space within the chassis, it might be difficult to get two standard devices inside, at least with no proper mounting which maybe a concern for Production usage. I know some folks were discussing the potential of mounting the devices outside via an external enclosure but I think that is probably more for home labs usage than having something that can just be shipped out to remote locations and just simply work. In any case, we have reached out to a partner to see if we can get access to supported SSD devices and using the U.2 slots. I will update this blog post if I make any progress in this area.

Thanks to our friends over at Intel, I was able to get my hands on a couple of the Intel Optane P4800X Series drives (2.5"). You will need to install the Intel NVMe driver, which you can find the latest version for ESXi 6.7 Update 1 here.

Although I was able to get vSAN up and running using the Optane drive in the E300-9D, as you can see from picture below, there is simply no room to fit both the drive + cable (which also requires an additional power cable), yet alone trying to close up the case.

For both E200-8D and E300-9D, the recommendation is definitely to stick with the PCIe AOC riser card and using two standard M.2 (M-Key) which you can verify using the VMware HCL.

More from my site

Comments

  1. Why not use the AOC-SLG3-2M2 which can also support 22110 sized cards (which you pointed out are even on the HCL).
    That would also mean you could use the Optane M.2 version of the 905P too!
    I'm building a version of the same thing David Chung built, using the 900P 58GB M.2 sticks in my E200-8D's and using a 24 port 10G switch - you can follow along as I build it out at https://sddc.info

    Reply

      • Hi William,

        The one you bought from MITXPC is called "SYS-E300-9D-8CN8TP". It's not E300-9D.

        In order to let people to buy the correct one, please help to fine tune the name from your web-page.

        Thanks a lot!

        Jerry Huang (*protected email*)

        Reply

      • FYI for anyone thinking... it looks like the Supermicro AOC-SLG3-2M2 doesn't fit the E200-8D. There doesn't seem to be a riser that would fit the card with the onboard x16 slot in the position it is 🙁

        Reply

    • Hi Brad. Could you provide information on that case? Where did you buy it and what are the components?

      Reply

    • For Home Labs, I would mostly agree ... but for Enterprise/Production usage, anyone who needs to develop for iOS will have a need to Virtualize MacOS and the only way to easily do so at scale and legally is using Mac HW, so Mini is still a very viable option and with 2018 models, the performance even outpaces the higher end Mac Pro systems.

      Reply

      • You can virtualize Mac os x on esxi? When esxi runs on Apple hardware
        .. I didn't know this

        Reply

        • Yup! It’s been possible since vSphere 5.0, I still remember attending on-site beta when I was a customer when this was announced and how happy I was since we had number of iOS/MacOS development apps and it was not manageable. Check out “Apple” section in my blog to see various customer Production stories on how they consume this feature

          Reply

  3. The "top to bottom" list says "Supermicro E900-9D" as the bottom item, think that should be E300-9D (photo has correct description)

    Reply

  4. hello William,

    I am looking to use this host for a customer as a esxi to host some witness VM for a strech cluster vsan.
    does this host have a controler that can match the vsan HCL with all flash capacity and all flasch caching ssd device.

    can we find some M2 drive that can match the vsan hcl "all flash capacity tier" and the "all flash caching tier" that can go in this host

    thanks a lot for your help. and have a gread day

    Reply

    • If you are using m.2 no need for HBA controller as they are attached directly and supported.

      Reply

  5. Hey there William,

    I'm currently considering either the newer SYS-E300-9D-4CN8TP or to go with SYS-E300-8D. My main concern currently is noise. Although, if the performance boost is really huge, i'd go for the E300-9D-4CN8TP.

    Thanks for your advise.
    Kind regards,
    Boris

    Reply

  6. Hello William!

    I cannot seem to get my NVMe devices to be recognized by ESXi (version 6.7 U1) in this identical E300-9D system. I have a Crucial P1 1TB 3D NAND NVMe PCIe M.2 SSD (CT1000P1SSD8). When I run the 'esxcli nvme device list' command it has a Status of 'offline' and I cannot figure out what I need to do to get the SSD to appear to the ESXi host. I also have a second NVMe drive, Patriot SCORCH M.2 2280 PCIe 3.0 x2 (PS256GPM280SSDR) that I purchased simply because it was cheap and perfect for a lab server. If you can shed some light on what I can do to fix this I would greatly appreciate it.

    THANK YOU!!!

    Reply

  7. Hi William,

    I have replaced 3 Noctura silent fans on my e200-8d, but CPU temperature is much high(99~100) than before(55~65). Do you experience the same issue? It seems it's trade off that less noise higher temperature.
    I consider to change one of previous fan back....Too sad!!

    Reply

      • Hi William, I just got another high speed fan(6500~7000 rpm )but less noise than previous one,right now it keeps under 70 degrees which I am comfortable to run any testing I want. Thank you.

        Reply

  8. Have you been having a look at Supermicro X11SDV-8C-TP8F

    ?

    Do you know how this compares to the E300-9D for a homelab setup? I think this will match my needs better as it is possible to put more "stuff" in it. Would like to have internal disks.

    Regards, Lars.

    Reply

    • Hi Lars, the E300-9D comes in 4 (SYS-E300-9D-4CN8TP) core and 8 (E300-9D-8CN8TP) core versions. The Supermicro X11SDV-8C-TP8F is the motherboard in the E300-9D-8CN8TP. The E300-9D-8CN8TP has loads of more I/O!!!

      Reply

  10. Great article. Couple of comments. Supermicro also over a few different chassis. For example 5019D (or 1019D dial power) is a rack mount version with space for 4 2.5" form factor drives. I didn't provide the full part number but one uses the same motherboard that is discussed here. Same is true of the 5019A (Atom C3000).

    Regarding noise and CPU temp, I've done quite a bit of testing on this. I started by ordering an air shroud (ref 5019D accessories). I then modified for the E300 chassis. The increased air focus helped lower fan RPM.

    However, with hot air dumping onto the NIC components it also had the effect of increasing temp in that area.

    I also tested with Noctua 40x40x20 fans. They simply can't push enough CFM in my opinion and a system under heavy load would create a problem.

    So the Frankenstein mod to truly solve is a top mounted larger fan which removed the heat from the chassis at the point of origin.

    I selected a high quality 90mm fan 4 pin from an old HPE desktop. I also borrowed the RPM reduction adapters that ship with Noctua fans.

    The result is as follows:

    40x40x28 fans running at 1300 RPM
    90mm fan running at 900 RPM
    CPU temp at 60 degrees C
    Whisper quiet!

    The next mod I plan is to update the air shroud to seal the heat sink with the top of the case. This will have the effect of forcing the top mounted fan to pull air only from the heat sink instead of also pulling air from the gap between the heat sink and top of the case.

    Back to the air shroud, which is the OEM optional accessory for the 5019D chassis, that was a fun party of the experiment. The two 40x40x28 fans behind the heat sink, they are not perfectly centered on the heat sink So, I cut and refabricated to create the shape needed. Thanks to some Scotch tape the result was effective and relatively clean looking.

    Reply

  11. Do you have any plans to test AS-E301-9D-8CN4? It uses AMD EPYC, and it looks like promising.

    Reply

  12. I too would like to see a comparison of the AMD AS-E301-9D-8CN4 vs the Intel SYS-E300-9D-8CN8TP. I'm currently trying to make a decision on which on to buy.

    Reply

    • I decided to take a risk and asked my contractor to acquire one. No regrets, it works smoothly. It's almost the same size as E200 but if the same processing capabilities of E300.

      Reply

  14. Im trying to decide with the following two servers here, Supermicro E200-8D vs SUPERMICRO SUPERSERVER 5018D-FN8T XEON D.

    ESXi 7/vSphere, vCenter, vSAN, vCloud, etc for home lab/SOHO setup.

    Reply

  15. Thanks for your articles. The SYS-E300-9D-8CN8TP looks like it is still a great system for ESXi 7. In a home lab with 120 v AC power sockets, how do you handle the DC power cable?

    Reply

  16. In late 2019 I bought a SYS-E300-9D-8CN8TP from MITXPC and took it with me overseas. The system stopped posting shortly after it went out of warranty, so I sent it off to SuperMicro TW. CPU is bad - they're charging me 645 USD to swap it. :/ Think I'll stick with swappable systems or better warranties in the future.

    Reply

Thanks for the comment!

This site uses Akismet to reduce spam. Learn how your comment data is processed.