Home
This page gives a brief overview of some of the technical details on how Steam Link streams VR to enable users to better understand and optimise their streaming enviornment.
Steam Link takes square images that are submitted by the application and then encodes, transmits, decodes and presents them to be viewed in the headset. It does this with a technique called foveated encoding. This is where the image that is encoded for transmitting is modified so that quality is focused on a specific region of the image where the user is directing their attention. This high quality region is called the foveated region. The rest of the image is sent a lower resolution to save on the amount of data being sent to the headset, which helps improve latency and increase the number of frames that can be sent per second to the headset.
Steam Link's foveated encoding is based on the principle that the foveated region should be modified as little as possible. Changing this area to a different resolution is known as resampling and can produce artifacts such as aliasing, degrading image quality. As such, Steam Link encodes this region at the same resolution the game submitted the image at, which means it does not have to incur a resample. How this looks, and the consequences of this, are explained in the settings described below.
Encoded video width and render resolution are two of the key settings that can impact the quality of the image Steam Link transmits. This is
Render resolution refers to the dimensions of the image that is rendered, per eye, by the application. A higher render resolution means a higher quality image, at the cost of additional processing that is needed by your PC.
Encoded video width relates to the dimensions of the foveated region. The configured video encode width relates to the number of pixels of the application's image that will be encoded 1:1 to the pixels that will be presented on the display (not accounting for any video encoding artifacts). As such, the higher the encoded video width, the large the number of pixels on the headset that will display the high quality region.
It's important to remember that Steam Link will not resample this image from the application, and as such, the overall "size" of how much of the original image the foveated region accounts for is also influenced by the render resolution. For example, assume the application has a render resolution of 2048x2048, and the encoded video width is 1152. This might look like:
However, if the render resolution of the application is then increased (for example to 2512), then the foveated region might now look like this:
The encoded video width still covers the same number of pixels (as the setting has not changed), but now those pixels account for a smaller portion of the image, so the "size" of the foveated region has decreased.
While we've spoken about the foveated region, it's also important to mention the relation the encoded video width has on the periphery regions (the image outside of the foveated region). This region is obviously expected to degrade in quality and so is not sampled 1:1 from the original game. Instead, the entire image is scaled from the original render resolution to the dimensions of the video encode width. Once this image has been decoded, it is then rescaled back to the headset's dimensions, necersarrily degrading its quality.
This means that there are 4 images (2 per eye consisting of one foveated and one periphery image) being sent that are all the same size, which helps the video encoder. What this looks like is pictured below for one eye, where the top image is the foveated region:
The Steam Link Debug Graph (SteamVR Settings > Steam Link > Advanced Detail Debug) displays useful information relating to the current state of the streaming envionrnment.
The main area to focus on is the bottom graph, which shows the current latency between the application submitting a frame and the headset showing the image on the display. There is a reference on the left hand side which shows the time taken in milliseconds. The top of the graph is the overall total latency between when the application submits the frame and the headset showing it on the display. In this case, around ~46ms.
The colors listed below give an indication of the time taken for the steps (along with the color code right at the bottom):
DEnc (red): Time taken to encode a frame.
DRx (purple): Time taken to transmit the video over wireless.
DDec (green): Time taken to decode the image and submit it to the headset.
ILU (grey): Time taken for the runtime to display the image.
Note that the ILU is necersarilly high due to certain steps the runtime must do to display the image.
Steam Link reduces it's overall latency by trying to start the pipeline between the application starting to render a frame, and the headset displaying that frame as late as possible to reduce time waiting in the pipeline. The absolute latest time that Steam Link can submit a frame and it be displayed "on-time" (without incurring a dropped frame) is drawn by the green line in the grey area. As wireless networks are unpredicatable, we leave a couple of milliseconds as a buffer in case of data arriving late, for example due to congestion on the network.