Category: Citrix

By Fredrik Brattstig @virtualbrat

May 2018 – My “Racing on thin clients” project had its peak at Citrix Synergy in Anaheim. This was the main attraction at the IGEL booth, it looked something like this:

Here we connected a Logitech racing wheel to a IGEL UD3-LX51 device, connected it all to Citrix Windows 10 VDA and did a Citrix session capped to 30 FPS. The experience was amazing and the attraction to the booth was insane.

During Synergy, i met up with @Thomas Poppelgaard, and we started talking about doing some advanced graphics project together.
During summertime and vacation in Sweden, you know, in Sweden you close down business during whole summer as everybody anyways are on vacation, i had some good time to think about what to do and came up with the crazy idea of extending my demo to Virtual Reality (VR), I have always had a passion for VR. But can it be done on Remote sessions? Can it be done using thin client operating system? CAN WE DO IT ON IGEL OS????
I also found a YouTube clip where Citrix was demonstrating a prototype – aha, it can be done!!

RemoteVR powered by IGEL OS project was born!

During the last few years i also met @Jared Cowart, Nvidia at different events and got him interested in the projects i was planning for. Me, Thomas and Jared started talking which ended up in Nvidia sponsoring me on a few graphics accelerators. The goal was to do RemoteVR.
The main focus was to use Nvidia vgpu (Nvidia Tesla P40) for back-end, to really show the power of vgpu and the possibility to have multiple users sharing the same GPU.

September 2018 – me, IGEL Development and Citrix Development –@Muhammad Dawood and Thomas Poppelgaard had a meeting where IGEL OS was confirmed to be suitable for what Citrix calls Virtual Virtual Reality, the only piece missing from the IGEL OS was that we were using a different routine for h264 hardware decoder, so Muhammad handed over the VDPAU binaries, which the IGEL Development team turned into a IGEL OS Custom Partition.

During the meeting the discussion was about that the requirements for a smooth VR experience was to reach 90 FPS on the endpoint, as the Head Mounted Display (HMD) has a refresh rate of 89Hz (yes, you are correct 89 FPS) when the session is running the frame rate varies between 89 and 90 FPS.
If you are a sane and up to date engineer in Citrix Virtual Apps and Desktops, you will here say, “it can’t be done as the VDA can maximum deliver 60 FPS“. There are a undocumented registry hack that can get the VDA to deliver up to blazing 120 FPS, that’s the current max (Citrix Virtual Apps and Desktop 1808). (Thanx to @Roberto Moreno to allow me to share)
The registry value to enable more than 60 FPS (90 represent a template value, you might have to test some different between 89 up to 120) is:

HKLM\Software\Citrix\Graphics
MaxFPS = 90 (DWORD)

vgpu and 90 FPS = No can do. The vgpu has a microcode limiter to maximum deliver up to 67 FPS, so the recommendation from Muhammad was to use a Nvida Quadro card instead. Lucky me that Jared arranged me to get one of those too, a Nvidia Quadro P6000.

Muhammad also gave me 2 more registry settings to prioritize the USB traffic. Click here to get those registry values. There is a lot of that traffic going on as the trackers receives the position data and sends it over for compute.

The decision was made to get this going using the HTC Vive HMD. This one has a good quality on the screen, 2160×1200. It is USB 2.0 compatible, which shows that it limits the data to be transferred. And it’s available everywhere.

So, let’s get going ad build the infrastructure! As this particular setup is designed to be able to bring easily to events, I had to keep the size down. With full length graphics cards, that can be a challenge. I started by trying to build my own case, and in as light material as possible. I went for plexi and did numerous try’s without great success. Giving that up, I found this small mini itx case. I had to cut the front metal plate open to fit the power connectors of the P6000. I love my Dremel 😂.

I built two machines, one for the RemoteVR worker, and one for the Citrix infrastructure. The later also contain a Nvidia Tesla P4 vgpu, to add some extension at events to show vgpu powered desktops on IGEL OS, why not the race driving?

I did have loads of lousy RemoteVR experiences feeling dizzy and nausea, I remember one when i was working on testing at about 70-80 FPS, a numerous try’s during the morning, in the afternoon i took my kids to a amusement park, Liseberg in Gothenburg, to do roller-coasters. Roller-coasters isn’t my favorite activity, and i can tell you that there were no rides that was fun when having the brain messed up by the morning exercise, the nausea stays for quite some time.
Continuous testing until i finally hit the gold vein, stable 90 FPS!! Here are the setup described:

So, the RemoteVR worker has a Intel i7 CPU@3.2 GHZ, 16GB RAM, 1 TB .M2 storage and Nvidia Quadro P6000 (24GB Frame Buffer). I installed Windows 10 Enterprise (currently build 1803) bare metal. Added on top of that here are the current software and drivers version: 
Nvidia Quadro 416.78
SteamVR  1.2.10 (this one auto updates, be aware of that)

The endpoint: here I’m using a Intel NUC Vega booting this one on IGEL OS UD Pocket with firmware 10.05.500. The particular NUC was chosen as the weapon of choice, due to that it is way overpowered and i wanted to have that extra space for fault tolerance. I haven’t started testing other hardware yet. In regular manner, you create the Citrix session, using the default Workspace app 1808 plus add those below specific settings
Enabling h264 Hardware acceleration:

Native USB Redirection of the HTC VIVE HMD:
The HTC Vive consist of a numerous USB Devices, showing up at the Linux Endpoint. I took all of them (which you can get quickly by issuing the lsusb command in a terminal on the IGEL OS).

And add them to the Native USB redirection part of the Citrix session configuration:

Adding the VDPAU Custom Partition:
This is done in regular Custom Partition (CP) manner, you request the CP by opening a support ticket @ IGEL support asking for it. It will be delivered with a instruction on how to add it to your IGEL UMS for distribution to your target endpoints. That’s one of the beauties of the IGEL ecosystem! SIMPLE!!

The Result:

Nothing is better than video proof right? Here you have some:

Whats next:
vgpu: I’ll try to figure out if it is possible to remove the frame limiter on the Tesla cards, and if so use the Tesla P40 virtualized. Or maybe a V100?! Or why not the new Turing T4!!

Endpoints: As the Intel NUC Vega is way overpowered, ill start up tests with lower spec’d devices, Initially the IGEL UD7 and some cheaper barebone models from the market.

Thanx for reading, hope you enjoyed and that it might give you hints on getting your setup running! Best /Fredrik