NVIDIA NGX: AI to Enhance Graphics with RTX

The simplest question, if we don’t know much about any of these topics, might be how does GPU / GPU work with artificial intelligence? Everything is much simpler, but at the same time, it is somewhat difficult to explain if we take into account the RTX architecture of NVIDIA video cards, especially a specific module for this purpose – Tensor cores.

Deep Learning Shapes NVIDIA NGX AI

As we probably already know, Tensor Core Blocks are used by NVIDIA in video games for supersampling deep learning, because supersampling pixels requires AI output.

But this understanding of tensor kernels is incomplete, since we only see what happens from the player’s side, not from the developer’s side, who creates those images, which are then scaled. This is where NVIDIA NGX comes in, and is at the heart of its use to create more lifelike images as it is nothing more than a stack of technologies based on Deep learning which provides developers with artificial intelligence features to speed up or improve the processing of graphics, images, and even real-time video.

As we mentioned, it needs a specific block to work that runs from the bottom and this is where the RTX GPU Tensor Cores come into play. The NVIDIA NGX SDK provides developers with a range of artificial intelligence functions at their disposal through networks trained on the NVIDIA Saturn V supercomputer.

What is this technology capable of and how does it do it?

It can be said that in general there are 3 scenarios that cover everything else: images, videos, and games. For each of them, there are different features that developers can use for different purposes as we are talking about a complete and very powerful SDK.

To include a few examples, in the image, we can remove the content from it and create it with AI, which will ideally replace the content since the neural network (InPainting) has been trained.

We can insert interpolated frames into video, animate any liquid in a series of images that did not contain it, and at two different speeds, since the AI ​​detects camera movement and allows us to add those FPS to the screen that do not exist.

With low resolutions or defective images, you can scale up to 8x the original quality by stretching existing pixels and filtering them in between thanks to AI.

It creates new pixels from your workouts and places them where they are needed, improving both image quality and sharpness. In gaming, this is even more surprising since the feature is included in the NVIDIA driver and, as we guessed, is used for DLSS in a previous tutorial on Saturn V.

Its operation is simple because when the driver detects a function call, it gives NGX full hardware access and real-time frame scaling using DLSS.