ESXi with Intel Arc 750 / 770 GPU

I have to say, the @IntelGraphics team does a very nice job on their packaging!

Look forward to kicking the tires on their new Arc GPUs, luckily this fits nicely in the Intel 12 Extreme NUC (Dragon Canyon) 😁 pic.twitter.com/DHTC93pzmc

— William Lam (@lamw.bsky.social | @*protected email*) (@lamw) January 21, 2023

I recently had the opportunity to play with one of Intel's Arc 7 series discrete GPU (A750), which is part of the new Intel Arc desktop graphics product line, that was just released last Fall. While the primary audience for the Intel Arc graphics is for creators and desktop gaming, I was really interested in its applicability for a vSphere-based environment which also has a number of interesting use cases for graphics from Virtual Desktop Infrastructure (VDI), video transcoding/rendering, AI/ML, Kubernetes, general purpose graphics acceleration and even basic gaming to name a few.

While there are many GPU options in the market for vSphere, depending on your needs and budget, the new Intel's new Arc 7 series could be a nice modern option as it is touted to be both price and performance competitive with other offerings within the market.

The Intel Arc 7 series has two models that are both compatible with the latest releases of ESXi. For the detailed technical specifications, please see the A750 page and A770 page, with the latter being the more performant model.

I have not built my own computer in a LONG time, which also means that I did not have a spare motherboard that could use to try out the A750, which is a full height dual-slot GPU (detailed sizing can be found on this page). I do however, have access to an Intel 12 Extreme NUC (Dragon Canyon), which is one of the expandable Intel NUCs that can support both the A750 and A770 GPU. To be clear, you do NOT need an Intel NUC 12 Extreme to be able to leverage the A750/A770 GPU, this was just what I had on hand and any system that can support a PCIe 4.0 x16 dual-slot can be used.

For my avid readers, you may have recalled my experimentation with the A770M (mobile version of the Intel Arc GPU) that is included with the latest Intel 12 Enthusiast NUC (Serpent Canyon), which can only be passthrough'ed and accessed by a Linux VM. The great news about the A750/A770 GPU is that it can be successfully passthrough and used by both a Linux and Windows operating systems!

When toggling PCI passthrough for the Intel Arc GPU, I noticed that the description for the A750 is just "<class> VGA compatible controller" and you will need to look at the DeviceID to confirm that you using the A750/A770. A quick search online for <vendorId>:<deviceId> (e.g. 8086:56A1), we can confirm that this is our A750 and can then be added to either a Linux or Windows VM for use.

Linux

Step 1 - Create and install Ubuntu Server 22.04 VM (recommend using 60GB storage or more, as additional packages will need to be installed). Once the OS has been installed, go ahead and shutdown the VM.

Step 2 - Enable passthrough of the GPU under the ESXi Configure->Hardware->PCI Devices settings and then add a new PCI Device to the VM and select the GPU. You can use either DirectPath IO or Dynamic DirectPath IO, it does not make a difference.

Step 3 - Edit the VM and under VM Options->Advanced->Configuration Parameters and add the following:

pciPassthru.use64bitMMIO = "TRUE"

Step 4 - Optionally, if you wish to disable the default virtual graphics driver (svga), edit the VM and under VM Options->Advanced->Configuration Parameters change the following setting from true to false:

svga.present

Step 5 - Power on the VM and then follow these instructions for installing the Intel Graphic Drivers for Ubuntu 22.04 and once completed, you will now be able to successfully use the GPU from within the Ubuntu VM.

Windows

Step 1 - Create and install Windows 10 or 11 VM. Once the OS has been installed, go ahead and shutdown the VM.

Step 2 - Enable passthrough of the GPU under the ESXi Configure->Hardware->PCI Devices settings and then add a new PCI Device to the VM and select the GPU. You can use either DirectPath IO or Dynamic DirectPath IO, it does not make a difference.

Step 3 - Edit the VM and under VM Options->Advanced->Configuration Parameters and add the following:

pciPassthru.use64bitMMIO = "TRUE"

Step 4 - Optionally, if you wish to disable the default virtual graphics driver (svga), edit the VM and under VM Options->Advanced->Configuration Parameters change the following setting from true to false:

svga.present

Step 5 - Power on the VM and then install the latest Intel Arc graphic Windows drivers and once completed, you will now be able to successfully use the GPU from within the Windows VM.

Here is a screenshot of both a Windows 11 and Windows 10 VM using the A750 running on the latest ESXi 8.0a release:

In addition to being able to use the GPU directly from within the VM, I can also confirm that if you have an external monitor connected to the GPU, the display of the VM can be outputted onto the external connected monitor, which can be useful for a desktop or gaming setup. In fact, you can passthrough additional devices like a USB keyboard and mouse, as shown in the screenshot below, to fully interact with the VM by implementing the instructions in this blog post.

While the Intel NUC 12 Extreme is just one of the many options for housing the A750/A770, I definitely wish there was a bit more room while installing the GPU. The space was quite tight once the GPU was installed and there was very little room left for the power cables which must be tuck behind the card and along the edge. The light up Intel Arc logo was pretty neat but it is a shame that the rest of the lights on the GPU were hidden on the other side and you really can not see them but the NUC 12 Extreme lights makes up for that both on the bottom and the front of the chassis 🙂

With the success of passing through A750/A770 to both a Linux and Windows VM, this definitely opens up the possibilities for many of the use cases that I had described earlier. Now, I am curious if our vSphere with Tanzu solution can also make use of the Intel Arc GPU via the Kubernetes interface provided by our VM Service Operator? 🤔 Time to explore a bit more further and report back 😀