Nvidia Quadro K5000 does not support GPU pass-through

Posted Nov 27

BREAKING NEWS

After a burst of e-mails with engineers from Nvidia, I’ve been informed that the flagship GPU of their new Kepler based cards, the Quadro K5000, is NOT supported for GPU pass-through. Without the GPU pass-through, the specific recipe of software I’m using for my cloud test isn’t going to be possible. Luckily after we were all brought up to speed, my contacts are providing a Fermi based Quadro 6000 to finish the test.

My current project is an attempt at generating data that would shed light on the number of users it takes to fatigue a XenApp graphic intensive server. Some of the hardware dedicated to this project were two Nvidia Quadro GPUs: the Fermi based 5000 and the Kepler based K5000.

Recently, hardware issues with the VM using the K5000 GPU tied to it led me to reach out to various support engineers at Citrix and Nvidia. The Windows Server 2008 R2 (x64) VM would not enable the K5000 GPU. The hardware manager would simply list “Error 43” and force the generic graphics to take over. This is not unusual behavior and at times simply powering down the host/physical server and then bringing it all back up enables the GPU properly in the VM. Unfortunately, when I attempted this the entire server would crash and reboot the host.

One of my contacts pointed me toward the XenServer Hardware Compatibility List (HCL) to show me that the K5000 was not listed. I knew that it wasn’t on the HCL but assumed that it was too new for the list or the site hosting the list was yet to be updated. As the flagship GPU in the new Kepler line of Nvidia Quadro GPUs one would expect the same if not better performance than the Fermi based cards.

I then received official word from Nvidia: “We (Nvidia) do not support GPU pass-through on the Quadro K5000 product.” I did some digging and couldn’t find any documentation that mentioned this. (If I missed it please let me know and it will be posted as an attachment.)

They went on to say that “We have a product line that is focused on virtualization, and that is the VGX product.”

The VGX system is a bag of Pros and Cons at this point:

Pro

Dedicated to virtualization.
Essentially 2 Quadro K5000s on a single card (VGX K2 edition)

Excellent! A product that focuses on the issue and packs quite a punch

Cons

Passively cooled. You can’t (shouldn’t) put this in a workstation, but in a server with internal fans.
Unpublished release date
Unpublished price point

So right now, if you’re upgrading or you’re looking to stand up a new Citrix farm you either wait for the VGX or stick with the Fermi based GPUs.

What does the tech community out there think of this separation of workstation and virtualization? Is this a good thing because we now have dedicated hardware etc. or is it a loss of ability to move to the cloud with pre-purchased hardware?

How many of you out there bought this card hoping to do virtualization just like this?

Let me know your thoughts.

Leave a Reply