Nvidia DLSS 2.0 and the Nvidia RTX2060-Max Q

With all the focus that has been given to Raytracing on the RTX-cards there is actually an technology that I found much more interesting: Nvidia DLSS (Deep learning super sampling).  The idea is that by using machine learning games that support DLSS are able to give you better or similar image quality at higher frame rates. I will touch on the technology briefly in the next section but one of the things I have been curious about is how it would help a mid-range card like the Nvidia RTX 2060 Max-Q that you now find on many mid/high-end gaming laptops.  The laptop I am using for this article is the ASUS ROG Zephyrus G15 that I reviewed earlier.

Nvidia DLSS 2.0 – How does it work?

Now before we start looking at how it works it is important to note that this is for RTX-cards only. The reason is quite simple: these cards have the special AI-accelerating Tensor cores that are used for this function. I’m guessing that there could be some way to emulate his on GTX-cards but I’m also guessing it would be slow so I would not hold my breath if  I owned a GTX-card, even a GTX1080. If you want to use DLSS you need to upgrade.

Nvidia released the first version of DLSS in august 2018 together with the RTX-cards and a new features called NGX (Neural Graphics Framework).

““NGX utilizes deep neural networks (DNNs) and set of ‘Neural Services’ to perform AI-based functions that accelerate and enhance graphics, rendering, and other client-side applications.” 

Basically Nvidia is using a super-computer and AI to go through a game and optimise it for the different NGX-features. One of these features are DLSS. This is how they describe it in a white-paper:

The key to this result is the training process for DLSS, where it gets the opportunity to learn how to produce the desired output based on large numbers of super-high-quality examples. To train the network, we collect thousands of “ground truth” reference images rendered with the gold standard method for perfect image quality, 64x supersampling (64xSS). 64x supersampling means that instead of shading each pixel once, we shade at 64 different offsets within the pixel, and then combine the outputs, producing a resulting image with ideal detail and anti-aliasing quality. We also capture matching raw input images rendered normally. Next, we start training the DLSS network to match the 64xSS output frames, by going through each input, asking DLSS to produce an output, measuring the difference between its output and the 64xSS target, and adjusting the weights in the network based on the differences, through a process called back propagation. After many iterations, DLSS learns on its own to produce results that closely approximate the quality of 64xSS, while also learning to avoid the problems with blurring, disocclusion, and transparency that affect classical approaches like TAA.

In addition to the DLSS capability described above, which is the standard DLSS mode, we provide a second mode, called DLSS 2X. In this case, DLSS input is rendered at the final target resolution and then combined by a larger DLSS network to produce an output image that approaches the level of the 64x super sample rendering – a result that would be impossible to achieve in real time by any traditional means.”

With the first version of DLSS Nvidia had to use AI on each game separately which also meant that it took time to get support out. This meant that not many games supports DLSS 1.0. The quality also was not super-good in the beginning which made many, including me, disregard the technology as not really useful. Nvidia however continued to improve the training for each game and rolled out updates via new drivers and the GeForce Experience. In the end though I think it is fair to sat that Nvidia DLSS 1.0 failed to live up to its promises.

With Nvidia DLSS 2.0 the company has made some big changes to make it much more useful.

• Superior Image Quality – DLSS 2.0 offers image quality comparable to native resolution while rendering only one quarter to one half of the pixels. It employs new temporal feedback techniques for sharper image details and improved stability from frame to frame.
• Great Scaling Across All GeForce RTX GPUs and Resolutions – A new AI network more efficiently uses Tensor Cores to execute 2X faster than the original. This improves frame rates and eliminates previous limitations on which GPUs, settings, and resolutions could be enabled.
• One Network For All Games – The original DLSS required training the AI network for each new game. DLSS 2.0 trains using non-game-specific content, delivering a generalized network that works across games. This means faster game integrations, and ultimately more DLSS games.
• Customizable Options – DLSS 2.0 offers users 3 image quality modes – Quality, Balanced, Performance – that control the game’s internal rendering resolution, with Performance mode enabling up to 4X super resolution (i.e. 1080p → 4K). This means more user choice, and even bigger performance boosts.

I think the main feature that has made DLSS 2.0 much more interesting is that each game does not have to be optimized individually and that DLSS 2.0 now train on content that work across games. I don’t know if there still can be some optimizations added afterwards but it is a big plus that more games now more easily can get access to DLSS-support. Games still have to implement it though so it is not automatic (yet).