Tag: user experience

By Fredrik Brattstig @virtualbrat

14th February 2024
As the title say, yes, latency does have an impact on the user experience, but let me try to explain what happens if you are running remote sessions, with high framerates, and what impact it will have on the user experience. Let’s start with some basic maths, that’s crucial to understand before we continue:
Latency is measured in Milliseconds (ms), a millisecond is 1/1000 of a second. Thousand ms in one second (man I like divide by 10, Americans, learn!! 🙂 ). Quite short time frame. When your systems operate on a Local Area Network (LAN), usually latency between your endpoint and your backend servers eg. your remote desktop workers are less than 1ms. When you move your workloads to the cloud, you will not have your endpoints on the same LAN, session traffic will need to traverse the Internet. Traversing Internet includes adding router hops, firewalls and other traffic obstacles. Do you know the impact it can have on your user’s experience? Did you validate what happens to your remote sessions when latency comes in to play? Could it be affecting your Frames Per Second (fps) in a negative way? Could too much fps actually have a bad impact? Let’s look at fps now.

Frames Per Second, usually you will try to match the frames per second sent from the backend system, over the network to be displayed in the same frame rate on your endpoint. Makes sense! The human eye can distinguish up to 60 frames per second, but as low as 25 fps will be seen as a fluid video experience, suitable for viewing. 25 fps is what the fat-TV era relied on, or as we say 25Hz (hertz). 25 Hz was used in the PAL regions (Europe), while NTSC regions (N America and Japan) was using a mindblowing 29.97Hz. This is not the complete story; I’m not trying to give a lesson on old TV’s. Take a moment on wiki read to learn more.
Todays computer monitors generally operates at 60Hz or higher, that means that they can display 60 FPS. Most remoting protocols defaults to delivering 30 FPS, while they can be modified to deliver more, or less, frames depending on your preferences AND limitations. This is where the math’s will be needed – limitations! Why?
Let’s look at an example of 30 FPS remote desktop session. each frame will then be displayed for 1000 divided by 30 ms. 1000/30 = 33.33333 ms. That means each frame will be visible for 33.3 ms. For reference here is a table for a few common framerates and its display time:

FPS	ms
25	40
30	33,333333333
60	16,666666666
100	10

We see that each frame will be displayed for 40ms if running at 25FPS. 33ms for 30FPS etc. You should have this in mind when setting up your configurations for your users’ session! Let me tell you why, But first a side note. The more FPS you push, the more bandwidth will be consumed. I will not dig to deep in this, but here is an example. Let’s say that you have a FullHD session, equals to 1920×1080=2.073.600 pixels. You run this session at 30FPS in full quality with 16.7M colors per pixel. 24bits per pixel x 2.073.600 x 30 = 1.492.992.000 bits/second would mean that in an uncompressed state each second would require 186,6 MegaBytes of raw data to be transferred. Luckily the display protocol vendors have some nifty compressions algorithms being applied 🙂 . We will leave that there..

Back to framerates and latency. As @30fps each frame is displayed for 33.333ms this means that if there is a latency that’s less than 33ms, the user will usually have a theoretical possibility to interact with the current frame shown. The more we reduce the latency, the more likely it is that the user interacts with the current frame. Though, if latency is higher than 33ms, the backend remote host will already have rendered the next frame, before the actual frame is displayed on the user’s screen. Imagine having a 60 FPS session, where latency is 70ms. That would mean that when the user sees the frame, the host has already produced 4 new frames, that are now queued to be delivered to the endpoint for display.
When you are typing in a word processing app, or similar, what I call “stationary target session” the latency is forgiven to some extent. It can happen that the delay from when you type a letter, it might take half a second before it is displayed on the screen, but when looking at something needing more real-time feedback, like drawing app, CAD, 3D modeling etc., what I call “moving target session” an effect described “rubber band effect” will get in play, it can look something like this:

In the video, you see my IGEL OS 12 endpoint connected to a Citrix session, where I’m using Microsoft Paint. In a remoted environment, a crosshair representing where your mouse cursor is located, and a dot representing where the pen tip is located. On a low latency session the dot follows the crosshair closely. To emulate high latency, I’ve installed the Tata WANem (https://wanem.sourceforge.net/) as a virtual machine. I created a static route from my IGEL OS endpoint to go through the WANem box to reach the Citrix host, and I set a static route on the Citrix host to go through the WANem to reach the IGEL OS endpoint. The effect will be that when I set the WANem to introduce 50ms latency, it will result in 100ms RTT (Round Trip Time). In the video I configure 50ms latency and it gives a very obvious effect when zooming in on the video where i move the mouse pointer in circles. The session is reported by Remote Display Analyzer (https://rdanalyzer.com/) to deliver around 23 FPS while moving the mouse pointer in circles.
Disclaimer, this is not a super scientific measurement, as it depends on the speed of the mouse movement of course, but i see the result as obvious.

In the above example and video, 23 FPS means that every frame is shown 43.5ms and with a latency of 50ms, it creates the rubber-band effect, eg. the mouse pointer is out of sync from the pen and the pen kind of drags along over the screen trying to catch up with the crosshair.

Conclusion

Higher latency introduces challenges when it comes to the end user experience if “moving targets” are in use. The effect is there for “stationary target sessions” too, but it is not as obvious for the user . If you have the same frontend system and the same backend system, but you introduce latency, let’s say that you move your workloads to the cloud, or you start connecting over Wide Area Network (WAN) or internet to your workloads in some shape or form (read move to home office and remote connect to the data center with the powerful Desktop workers, like the covid years), it’s easy to make the mistake and think that the user experience is choppy due to low framerate. Increasing the framerate will probably not help, but lowering and limiting the framerate can be the right answer.
I have seen demos of GPU powered systems in the cloud where the presenter zooms in on a benchmark apps framerate and it shows 100+ FPS, the mistake here is that the GPU in the remote machine is producing way more than enough frames, what would be more tied to reality is to analyze the actual session framerate, and the latency as these are the values that matter for a smooth user experience.

Know your workloads, and where to place the workers based on the scenario you are trying to fulfill! When you choose IGEL OS as your endpoints, you can be certain that the features in the operating system will allow your sessions to run in an optimal state, as long as the endpoint hardware, and the location and sizing of your Desktop workers is made correct. Sometimes, on-prem is the right way to go, sometimes lower framerate is better than higher framerates no matter how strange it may sound!

Final note

Display protocol vendors have intelligent ways to bring order to the chaos described here, by example discarding missed frames, by only sending incomplete frames that just contain the pixels that have changed, and merge it together in the endpoint etc, What I’m trying to say is, some workloads are suitable for moving to cloud, or access over WAN connections, while some still will give the best user experience by being executed as close to the user as possible.