Nvidia presentation overview: AI Jarvis and the future of graphics in games

May 14, Nvidia held an introductory presentation for the GTC. GTC, or GPU Technology Conference, is an event for specialists, there are no consumer products there. At GTC 2020, they talked about smart cars, servers, data centers and artificial intelligence. Let's see where the graphics industry is heading. I want to highlight a number of interesting points.

Content

Data center as a computer unit

But let's start with the coronavirus. This topic could not be ignored. Solutions from Nvidia help in all areas, from deciphering genomes to using them in medical bots.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

By the way, in their presentation, Nvidia showed a model of the University of Austin, but I will assume that the Russian company Visual Science also used graphics from Nvidia to create their detailed model of the SARS-CoV-2 virus.

Nvidia CEO Jensen Huang shared his vision for the development of the industry. According to him, there are two main forces pushing the development of computer computing in recent years. In the first place is machine learning, that is, building a new one based on already known data.

The second driver is the growth in the complexity of tasks and, accordingly, the growth in the size of applications, when the computer alone cannot cope. This has led to an increase in the number of data centers. According to Mr. Huang, we are moving towards the fact that not a computer or a server, but a whole data center will be taken as one computing unit. Accordingly, the industry is faced with a new problem of how to create, transfer, store and optimize data so that it can be used as productively as possible at the level of data centers.

Perhaps as a consumer inference from this prediction, the game streaming model, where the powerful hardware to play the game is somewhere in the cloud, will become the main type of gaming for the majority of the population.

To speed up data delivery (i.e. networking), Nvidia bought Mellanox. This appears to have been the biggest deal in Nvidia's history. Mellanox manufactures equipment for high-speed networks with a focus on supercomputers and big data data centers.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Artificial intelligence in graphics creation

Mr. Huang started from afar. From a story about how 40 years ago one of the company's employees described a model of the behavior of light on an object.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

The ball rolls, the surface is reflected in it, the light plays

It was only 38 years later, in 2018, that Nvidia was able to bring the model to the masses in the form of a new generation of GeForce RTX cards with two innovative features: real-time ray tracing and artificial intelligence aimed at completing and improving the picture.< /p>

According to Mr. Huang, even after so much time since the creation of the model, the power of video cards for ray tracing was not enough, so machine learning just came to the rescue. In recent years, Nvidia has been actively training its AI. He is given a 540p picture, and his goal is to synthesize a Full HD picture based on what he has learned.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

This is probably the most exciting part of the entire presentation. For training, Nvidia made a 16K model and ran the AI ​​"trillions of times" through it. And then the trained AI is delivered to user computers in the form of drivers. This technology is called DLSS, or Deep Learning Super Sampling, that is, deep learning on super samples (still not super when the original model is 16K).

Our great site is desperately squeezing all the pictures, so under this paragraph, I'll leave a link to a video with the timing of this place. Better watch in 4K!

So, first the picture was shown in the original 16K.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Then they showed how the video card itself can display it in 720p.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Next, they showed how the first version of DLSS 1.0 improved the picture by taking a 720p source and trying to "stretch" it. Here it is immediately worth noting that DLSS 1.0 did not take off, since in order to train AI for each task (game), it was necessary to create a separate training ground. And this is not an easy and expensive task, so the developers simply sabotaged DLSS 1.0.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

But Nvidia believed in success. About a month ago, the company introduced DLSS 2.0. This technology is likely to succeed, as it is more viable. You no longer need to create a separate training ground for each game. But the best thing is that DLSS 2.0 allows you to get better image quality than the Full HD picture that the card can display on its own.

Nvidia keynote review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

The thing is that the video card simply renders the image based on its capabilities. And DLSS 2.0 knows (or thinks it does) what the source image looks like at super-large resolution, so the AI ​​builds on even things that don't exist. And, perhaps, this is the main advantage, why you should buy RTX cards and not GTX cards. There are no so-called Tensor Cores in GTX - these are separate processors for artificial intelligence.

Video from mark: 2.35:

For example, here is a performance boost provided by DLSS 2.0 technology in Minecraft.

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games Nvidia presentation overview: Jarvis artificial intelligence and the future of graphics in games

And along the way, Nvidia boasted that all the cool guys work with its graphics. Take a bite, AMD! It's certainly worth pointing out here that Nvidia is long overdue for burying the hatchet and making friends with Apple again, as many in the movie industry use Mac Pros to render graphics. Although everything is relative, for example, the picture shows the Pixar studio, which, as it were, historically uses Apple products, but, apparently, does not forget about Nvidia either.

Nvidia keynote review: Jarvis AI and the future of gaming graphics

But in general, the story was served under the sauce that Nvidia tools allow several users to work on 3D models at the same time. And all this is possible with the help of a new server, studded with a bunch of RTX cards. Buy, stick cards, and go! In fact, a super hit, especially in conditions of self-isolation, and indeed. The designer is sitting somewhere in Hawaii and making a new cartoon from Pixar with other dudes around the world. Only high-speed Internet is needed.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

For example, a roller with a ball. It is notable for the lighting, shadows, natural physics, and the fact that it was made together by designers and engineers sitting all over the country.

About high-performance computers

In the high-end computing business, Mr. Huang told boring things to the average consumer. In short, Spark, or Apache Spark, is an analytics engine for working with big data. So Nvidia says: what if our cool work is used to speed up data processing?

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

For example, let Spark use video card processors and video card memory to process data! And let Spark plan and share the work so that everything is considered everywhere and at once at the same time - both in the processor and in the video card! And let's make a special library that Spark can access. And let's call it all Spark 3.0. So they did there.

As a result, everything began to work much faster. As proof, they showed a certain superserver from Dell, which, in addition to a bunch of speed characteristics, costs a whole million dollars and consumes 16 kW! And gives a data processing speed of 17 Gb / s. And if you make a Spark 3.0 superserver with parts from Nvidia, then it will cost $ 2 million, but the data processing speed will be 163 Gb / s. And all scientists and big data processors should be thrilled, because it's only twice as expensive, but 10 times more productive. Well, a data center needs at least a dozen of these cabinets.

And at the end, Jensen Huang threw up his hands like that and said: “The more you buy, the more you save!”.

Nvidia keynote review: Jarvis AI and the future of gaming graphics

On this, I think, the discussion of this topic can be completed. For the thing, of course, is good, but I'm not sure what I can afford right now. Although, of course, one or two lockers for the future will have to be bought. In the meantime, let Google and Microsoft test the new technology.

Nvidia keynote review: Jarvis AI and the future of gaming graphics

Nvidia recommendation system

Another technology that the average user won't use, but will definitely come across. Nvidia introduced the Merlin recommendation system, which is suitable for everything from movies and music to clothing.

Although the topic of recommendations seems boring, it is super important. Just remember how Yandex.Music unexpectedly shot after Yandex introduced a new recommendation system. The service turned from an ugly duckling into a candy!

Each recommendation service is a complex system where each user is passed through filters, and the system tries to choose the most appropriate one from billions of options. All this requires both computing power and artificial intelligence. Nvidia, they say, will make this technology available to the masses. The company introduced the Nvidia Merlin. This is a framework, that is, a software infrastructure for creating recommender systems. Merlin allows you to significantly speed up the learning process of predictive AI. As an example, they cited the fact that a typical system for 1 TB of some data takes 1.5 days to train, and Merlin can do it in 16 minutes. Well, in order for all this to work, of course, you need RTX servers with Tensor processors. Imagine how Yandex Music will get even prettier if Yandex has enough money to buy it. For although it was said about democratization, they did not name the price.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Jarvis is an artificial intelligence that can communicate

Probably, Nvidia thought: since we are so cool and make products that allow us to dramatically speed up the analysis, selection and prediction of the behavior of objects, then why not offer a solution for conversational artificial intelligence. In fact, this type of AI is faced with an abundance of complex tasks that require significant computing power - you need to recognize colloquial speech, guess what exactly the user means, find it, then try to answer him naturally and clearly. Despite the fact that there are already quite a lot of conversational assistants, they know little and are significantly limited. Nvidia invented Jarvis. This is not a specific voice assistant, but only a framework that, for example, Yandex can use to make Alice smarter. Jarvis will be sold already with a set of behaviors, which should simplify implementation and further training.

Nvidia presentation review: Jarvis artificial intelligence and the future of gaming graphics

And the slide below is just an example Nvidia did with Omniverse. The demo bot not only responds, but it even has facial expressions. The bot was trained to answer questions about the weather. And in the context of the coldest city in the world, Yakutsk was mentioned. This is how Russia celebrated at the presentation of the GTC 2020. We are proud!

Nvidia presentation review: Jarvis AI and the future of gaming graphics

And you can watch the chatbot in action on the video here:

Two news we'll miss: Nvidia EGX A100 and EGX Jetson Xavier NX

A100 is a GPU for data centers. If we come across these solutions, we are unlikely to know, since the A100 can, for example, control hundreds of cameras in airports, while the EGX Jetson Xavier NX is suitable for controlling several cameras in small shops.

As Mr. Huang described it, “The NVIDIA EGX Edge AI platform transforms a standard server into a small, secure, AI-enabled cloud data center. With NVIDIA AI frameworks, companies can build AI services ranging from smart retail and robotic factories to automated call centers.”

Nvidia keynote review: Jarvis AI and the future of gaming graphics

All of this is based on the new NVIDIA Ampere architecture. To quote the official press release:

NVIDIA Ampere Architecture - NVIDIA's 8th GPU architecture - provides the largest performance boost for a wide range of demanding workloads, including AI inference and 5G edge applications. This allows the EGX A100 to process large amounts of real-time data from cameras and other IoT sensors to quickly gain insights and improve business efficiency.

With the NVIDIA Mellanox ConnectX-6 Dx network card, the EGX A100 can receive up to 200 Gbps of data and send it directly to GPU memory for AI or 5G signal processing. With NVIDIA Mellanox time-driven telecom technology (5T for 5G), the cloud-based, software-defined accelerator EGX A100 is capable of handling most latency-sensitive 5G tasks. This creates a full-fledged AI/5G platform for real-time decision-making directly at the scene - stores, hospitals and factories.

"Together with NVIDIA, we are building a high-performance 5G virtualized radio access network and an accelerated 5G packet network," said Pardeep Kohli, President and CEO of Mavenir. “This will enable us to provide a wide range of GPU-accelerated 5G services, from AI and machine learning to augmented and virtual reality.”

Nvidia Ampere in smart cars

Nvidia Ampere can be used in smart cars. The trick is that, relatively speaking, you only need this board, which is in the picture. All calculations will take place in it, there is also a smart trained AI.

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

Nvidia's smart car platform is open, so it's being enjoyed all over the world. Mostly Chinese and Indians. Also Mercedes and Toyota. Where is Tesla?

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Conclusion

Nvidia is briskly striding forward. I guess you, like me, were primarily interested in consumer stuff like DLSS 2.0. But in general, everything presented by Nvidia is designed for business users. Even DLSS 2.0 should reduce the load on streaming data centers. It seems that while Microsoft has opened up Azure and the cloud as a new source of growth, Nvidia is betting on rethinking the computing market and artificial intelligence. The only vulnerable point may look like the fact that the company still prefers to create hardware and infrastructure that other companies will already configure, build data centers, and then sell computing capacities.

Nvidia presentation overview: AI Jarvis and the future of graphics in games

May 14, Nvidia held an introductory presentation for the GTC. GTC, or GPU Technology Conference, is an event for specialists, there are no consumer products there. At GTC 2020, they talked about smart cars, servers, data centers and artificial intelligence. Let's see where the graphics industry is heading. I want to highlight a number of interesting points.

Content

Data center as a computer unit

But let's start with the coronavirus. This topic could not be ignored. Solutions from Nvidia help in all areas, from deciphering genomes to using them in medical bots.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

By the way, in their presentation, Nvidia showed a model of the University of Austin, but I will assume that the Russian company Visual Science also used graphics from Nvidia to create their detailed model of the SARS-CoV-2 virus.

Nvidia CEO Jensen Huang shared his vision for the development of the industry. According to him, there are two main forces pushing the development of computer computing in recent years. In the first place is machine learning, that is, building a new one based on already known data.

The second driver is the growth in the complexity of tasks and, accordingly, the growth in the size of applications, when the computer alone cannot cope. This has led to an increase in the number of data centers. According to Mr. Huang, we are moving towards the fact that not a computer or a server, but a whole data center will be taken as one computing unit. Accordingly, the industry is faced with a new problem of how to create, transfer, store and optimize data so that it can be used as productively as possible at the level of data centers.

Perhaps as a consumer inference from this prediction, the game streaming model, where the powerful hardware to play the game is somewhere in the cloud, will become the main type of gaming for the majority of the population.

To speed up data delivery (i.e. networking), Nvidia bought Mellanox. This appears to have been the biggest deal in Nvidia's history. Mellanox manufactures equipment for high-speed networks with a focus on supercomputers and big data data centers.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Artificial intelligence in graphics creation

Mr. Huang started from afar. From a story about how 40 years ago one of the company's employees described a model of the behavior of light on an object.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

The ball rolls, the surface is reflected in it, the light plays

It was only 38 years later, in 2018, that Nvidia was able to bring the model to the masses in the form of a new generation of GeForce RTX cards with two innovative features: real-time ray tracing and artificial intelligence aimed at completing and improving the picture.< /p>

According to Mr. Huang, even after so much time since the creation of the model, the power of video cards for ray tracing was not enough, so machine learning just came to the rescue. In recent years, Nvidia has been actively training its AI. He is given a 540p picture, and his goal is to synthesize a Full HD picture based on what he has learned.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

This is probably the most exciting part of the entire presentation. For training, Nvidia made a 16K model and ran the AI ​​"trillions of times" through it. And then the trained AI is delivered to user computers in the form of drivers. This technology is called DLSS, or Deep Learning Super Sampling, that is, deep learning on super samples (still not super when the original model is 16K).

Our great site is desperately squeezing all the pictures, so under this paragraph, I'll leave a link to a video with the timing of this place. Better watch in 4K!

So, first the picture was shown in the original 16K.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Then they showed how the video card itself can display it in 720p.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Next, they showed how the first version of DLSS 1.0 improved the picture by taking a 720p source and trying to "stretch" it. Here it is immediately worth noting that DLSS 1.0 did not take off, since in order to train AI for each task (game), it was necessary to create a separate training ground. And this is not an easy and expensive task, so the developers simply sabotaged DLSS 1.0.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

But Nvidia believed in success. About a month ago, the company introduced DLSS 2.0. This technology is likely to succeed, as it is more viable. You no longer need to create a separate training ground for each game. But the best thing is that DLSS 2.0 allows you to get better image quality than the Full HD picture that the card can display on its own.

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

The thing is that the video card simply renders the image based on its capabilities. And DLSS 2.0 knows (or thinks it does) what the source image looks like at super-large resolution, so the AI ​​builds on even things that don't exist. And, perhaps, this is the main advantage why you should buy RTX cards, and not GTX cards. There are no so-called Tensor Cores in GTX - these are separate processors for artificial intelligence.

Video from mark: 2.35:

For example, here is a performance boost provided by DLSS 2.0 technology in Minecraft.

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games Nvidia presentation overview: Jarvis artificial intelligence and the future of graphics in games

And along the way, Nvidia boasted that all the cool guys work with its graphics. Take a bite, AMD! It's certainly worth pointing out here that Nvidia is long overdue for burying the hatchet and making friends with Apple again, as many in the movie industry use Mac Pros to render graphics. Although everything is relative, for example, the picture shows the Pixar studio, which, as it were, historically uses Apple products, but, apparently, does not forget about Nvidia either.

Nvidia keynote review: Jarvis AI and the future of gaming graphics

But in general, the story was served under the sauce that Nvidia tools allow several users to work on 3D models at the same time. And all this is possible with the help of a new server, studded with a bunch of RTX cards. Buy, stick cards, and go! In fact, a super hit, especially in conditions of self-isolation, and indeed. The designer is sitting somewhere in Hawaii and making a new cartoon from Pixar with other dudes around the world. Only high-speed Internet is needed.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

For example, a roller with a ball. It is notable for the lighting, shadows, natural physics, and the fact that it was made together by designers and engineers sitting all over the country.

About high-performance computers

In the high-end computing business, Mr. Huang told boring things to the average consumer. In short, Spark, or Apache Spark, is an analytics engine for working with big data. So Nvidia says: what if our cool work is used to speed up data processing?

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

For example, let Spark use video card processors and video card memory to process data! And let Spark plan and share the work so that everything is considered everywhere and at once at the same time - both in the processor and in the video card! And let's make a special library that Spark can access. And let's call it all Spark 3.0. So they did there.

As a result, everything began to work much faster. As proof, they showed a certain superserver from Dell, which, in addition to a bunch of speed characteristics, costs a whole million dollars and consumes 16 kW! And gives a data processing speed of 17 Gb / s. And if you make a Spark 3.0 superserver with parts from Nvidia, then it will cost $ 2 million, but the data processing speed will be 163 Gb / s. And all scientists and big data processors should be thrilled, because it's only twice as expensive, but 10 times more productive. Well, a data center needs at least a dozen of these cabinets.

And at the end, Jensen Huang threw up his hands like that and said: “The more you buy, the more you save!”.

Nvidia keynote review: Jarvis AI and the future of gaming graphics

On this, I think, the discussion of this topic can be completed. For the thing, of course, is good, but I'm not sure what I can afford right now. Although, of course, one or two lockers for the future will have to be bought. In the meantime, let Google and Microsoft test the new technology.

Nvidia keynote review: Jarvis AI and the future of gaming graphics

Nvidia recommendation system

Another technology that the average user won't use, but will definitely come across. Nvidia introduced the Merlin recommendation system, which is suitable for everything from movies and music to clothing.

Although the topic of recommendations seems boring, it is super important. Just remember how Yandex.Music unexpectedly shot after Yandex introduced a new recommendation system. The service turned from an ugly duckling into a candy!

Each recommendation service is a complex system where each user is passed through filters, and the system tries to choose the most appropriate one from billions of options. All this requires both computing power and artificial intelligence. Nvidia, they say, will make this technology available to the masses. The company introduced the Nvidia Merlin. This is a framework, that is, a software infrastructure for creating recommender systems. Merlin allows you to significantly speed up the learning process of predictive AI. As an example, they cited the fact that a typical system for 1 TB of some data takes 1.5 days to train, and Merlin can do it in 16 minutes. Well, in order for all this to work, of course, you need RTX servers with Tensor processors. Imagine how Yandex Music will get even prettier if Yandex has enough money to buy it. For although it was said about democratization, they did not name the price.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Jarvis is an artificial intelligence that can communicate

Probably, Nvidia thought: since we are so cool and make products that allow us to dramatically speed up the analysis, selection and prediction of the behavior of objects, then why not offer a solution for conversational artificial intelligence. In fact, this type of AI is faced with an abundance of complex tasks that require significant computing power - you need to recognize colloquial speech, guess what exactly the user means, find it, then try to answer him naturally and clearly. Despite the fact that there are already quite a lot of conversational assistants, they know little and are significantly limited. Nvidia invented Jarvis. This is not a specific voice assistant, but only a framework that, for example, Yandex can use to make Alice smarter. Jarvis will be sold already with a set of behaviors, which should simplify implementation and further training.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

And the slide below is just an example Nvidia did with Omniverse. The demo bot not only responds, but it even has facial expressions. The bot was trained to answer questions about the weather. And in the context of the coldest city in the world, Yakutsk was mentioned. This is how Russia celebrated at the presentation of the GTC 2020. We are proud!

Nvidia presentation review: Jarvis AI and the future of gaming graphics

And you can watch the chatbot in action on the video here:

Two news we'll miss: Nvidia EGX A100 and EGX Jetson Xavier NX

A100 is a GPU for data centers. If we come across these solutions, we are unlikely to know, since the A100 can, for example, control hundreds of cameras in airports, while the EGX Jetson Xavier NX is suitable for controlling several cameras in small shops.

As Mr. Huang described it, “The NVIDIA EGX Edge AI platform transforms a standard server into a small, secure, AI-enabled cloud data center. With NVIDIA AI frameworks, companies can build AI services ranging from smart retail and robotic factories to automated call centers.”

Nvidia keynote review: Jarvis AI and the future of gaming graphics

All of this is based on the new NVIDIA Ampere architecture. To quote the official press release:

NVIDIA Ampere Architecture - NVIDIA's 8th GPU architecture - provides the largest performance boost for a wide range of demanding workloads, including AI inference and 5G edge applications. This allows the EGX A100 to process large amounts of real-time data from cameras and other IoT sensors to quickly gain insights and improve business efficiency.

With the NVIDIA Mellanox ConnectX-6 Dx network card, the EGX A100 can receive up to 200 Gbps of data and send it directly to GPU memory for AI or 5G signal processing. With NVIDIA Mellanox time-driven telecom technology (5T for 5G), the cloud-based, software-defined accelerator EGX A100 is capable of handling most latency-sensitive 5G tasks. This creates a full-fledged AI/5G platform for real-time decision-making directly at the scene - stores, hospitals and factories.

"Together with NVIDIA, we are building a high-performance 5G virtualized radio access network and an accelerated 5G packet network," said Pardeep Kohli, President and CEO of Mavenir. “This will enable us to provide a wide range of GPU-accelerated 5G services, from AI and machine learning to augmented and virtual reality.”

Nvidia Ampere in smart cars

Nvidia Ampere can be used in smart cars. The trick is that, relatively speaking, you only need this board, which is in the picture. All calculations will take place in it, there is also a smart trained AI.

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

Nvidia's smart car platform is open, so it's being enjoyed all over the world. Mostly Chinese and Indians. Also Mercedes and Toyota. Where is Tesla?

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Conclusion

Nvidia is briskly striding forward. I guess you, like me, were primarily interested in consumer stuff like DLSS 2.0. But in general, everything presented by Nvidia is designed for business users. Even DLSS 2.0 should reduce the load on streaming data centers. It seems that while Microsoft has opened up Azure and the cloud as a new source of growth, Nvidia is betting on rethinking the computing market and artificial intelligence. The only vulnerable point may look like the fact that the company still prefers to create hardware and infrastructure that other companies will already configure, build data centers, and then sell computing capacities.

Nvidia presentation overview: AI Jarvis and the future of graphics in games

May 14, Nvidia held an introductory presentation for the GTC. GTC, or GPU Technology Conference, is an event for specialists, there are no consumer products there. At GTC 2020, they talked about smart cars, servers, data centers and artificial intelligence. Let's see where the graphics industry is heading. I want to highlight a number of interesting points.

Content

Data center as a computer unit

But let's start with the coronavirus. This topic could not be ignored. Solutions from Nvidia help in all areas, from deciphering genomes to using them in medical bots.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

By the way, in their presentation, Nvidia showed a model of the University of Austin, but I will assume that the Russian company Visual Science also used graphics from Nvidia to create their detailed model of the SARS-CoV-2 virus.

Nvidia CEO Jensen Huang shared his vision for the development of the industry. According to him, there are two main forces pushing the development of computer computing in recent years. In the first place is machine learning, that is, building a new one based on already known data.

The second driver is the growth in the complexity of tasks and, accordingly, the growth in the size of applications, when the computer alone cannot cope. This has led to an increase in the number of data centers. According to Mr. Huang, we are moving towards the fact that not a computer or a server, but a whole data center will be taken as one computing unit. Accordingly, the industry is faced with a new problem of how to create, transfer, store and optimize data so that it can be used as productively as possible at the level of data centers.

Perhaps as a consumer inference from this prediction, the game streaming model, where the powerful hardware to play the game is somewhere in the cloud, will become the main type of gaming for the majority of the population.

To speed up data delivery (i.e. networking), Nvidia bought Mellanox. This appears to have been the biggest deal in Nvidia's history. Mellanox manufactures equipment for high-speed networks with a focus on supercomputers and big data data centers.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Artificial intelligence in graphics creation

Mr. Huang started from afar. From a story about how 40 years ago one of the company's employees described a model of the behavior of light on an object.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

The ball rolls, the surface is reflected in it, the light plays

It was only 38 years later, in 2018, that Nvidia was able to bring the model to the masses in the form of a new generation of GeForce RTX cards with two innovative features: real-time ray tracing and artificial intelligence aimed at completing and improving the picture.< /p>

According to Mr. Huang, even after so much time since the creation of the model, the power of video cards for ray tracing was not enough, so machine learning just came to the rescue. In recent years, Nvidia has been actively training its AI. He is given a 540p picture, and his goal is to synthesize a Full HD picture based on what he has learned.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

This is probably the most exciting part of the entire presentation. For training, Nvidia made a 16K model and ran the AI ​​"trillions of times" through it. And then the trained AI is delivered to user computers in the form of drivers. This technology is called DLSS, or Deep Learning Super Sampling, that is, deep learning on super samples (still not super when the original model is 16K).

Our great site is desperately squeezing all the pictures, so under this paragraph, I'll leave a link to a video with the timing of this place. Better watch in 4K!

So, first the picture was shown in the original 16K.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Then they showed how the video card itself can display it in 720p.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Next, they showed how the first version of DLSS 1.0 improved the picture by taking a 720p source and trying to "stretch" it. Here it is immediately worth noting that DLSS 1.0 did not take off, since in order to train AI for each task (game), it was necessary to create a separate training ground. And this is not an easy and expensive task, so the developers simply sabotaged DLSS 1.0.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

But Nvidia believed in success. About a month ago, the company introduced DLSS 2.0. This technology is likely to succeed, as it is more viable. You no longer need to create a separate training ground for each game. But the best thing is that DLSS 2.0 allows you to get better image quality than the Full HD picture that the card can display on its own.

Nvidia keynote review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

The thing is that the video card simply renders the image based on its capabilities. And DLSS 2.0 knows (or thinks it does) what the source image looks like at super-large resolution, so the AI ​​builds on even things that don't exist. And, perhaps, this is the main advantage, why you should buy RTX cards and not GTX cards. There are no so-called Tensor Cores in GTX - these are separate processors for artificial intelligence.

Video from mark: 2.35:

For example, here is a performance boost provided by DLSS 2.0 technology in Minecraft.

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games Nvidia presentation overview: Jarvis artificial intelligence and the future of graphics in games

And along the way, Nvidia boasted that all the cool guys work with its graphics. Take a bite, AMD! It's certainly worth pointing out here that Nvidia is long overdue for burying the hatchet and making friends with Apple again, as many in the movie industry use Mac Pros to render graphics. Although everything is relative, for example, the picture shows the Pixar studio, which, as it were, historically uses Apple products, but, apparently, does not forget about Nvidia either.

Nvidia keynote review: Jarvis AI and the future of gaming graphics

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

For example, let Spark use video card processors and video card memory to process data! And let Spark plan and share the work so that everything is considered everywhere and at once at the same time - both in the processor and in the video card! And let's make a special library that Spark can access. And let's call it all Spark 3.0. So they did there.

As a result, everything began to work much faster. As proof, they showed a certain superserver from Dell, which, in addition to a bunch of speed characteristics, costs a whole million dollars and consumes 16 kW! And gives a data processing speed of 17 Gb / s. And if you make a Spark 3.0 superserver with parts from Nvidia, then it will cost $ 2 million, but the data processing speed will be 163 Gb / s. And all scientists and big data processors should be thrilled, because it's only twice as expensive, but 10 times more productive. Well, a data center needs at least a dozen of these cabinets.

And at the end, Jensen Huang threw up his hands like that and said: “The more you buy, the more you save!”.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

On this, I think, the discussion of this topic can be completed. For the thing, of course, is good, but I'm not sure what I can afford right now. Although, of course, one or two lockers for the future will have to be bought. In the meantime, let Google and Microsoft test the new technology.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Nvidia recommendation system

Another technology that the average user won't use, but will definitely come across. Nvidia introduced the Merlin recommendation system, which is suitable for everything from movies and music to clothing.

Although the topic of recommendations seems boring, it is super important. Just remember how Yandex.Music unexpectedly shot after Yandex introduced a new recommendation system. The service turned from an ugly duckling into a candy!

Each recommendation service is a complex system where each user is passed through filters, and the system tries to choose the most appropriate one from billions of options. All this requires both computing power and artificial intelligence. Nvidia, they say, will make this technology available to the masses. The company introduced the Nvidia Merlin. This is a framework, that is, a software infrastructure for creating recommender systems. Merlin allows you to significantly speed up the learning process of predictive AI. As an example, they cited the fact that a typical system for 1 TB of some data takes 1.5 days to train, and Merlin can do it in 16 minutes. Well, in order for all this to work, of course, you need RTX servers with Tensor processors. Imagine how Yandex Music will get even prettier if Yandex has enough money to buy it. For although it was said about democratization, they did not name the price.

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Jarvis is an artificial intelligence that can communicate

Probably, Nvidia thought: since we are so cool and make products that allow us to dramatically speed up the analysis, selection and prediction of the behavior of objects, then why not offer a solution for conversational artificial intelligence. In fact, this type of AI is faced with an abundance of complex tasks that require significant computing power - you need to recognize colloquial speech, guess what exactly the user means, find it, then try to answer him naturally and clearly. Despite the fact that there are already quite a lot of conversational assistants, they know little and are significantly limited. Nvidia invented Jarvis. This is not a specific voice assistant, but only a framework that, for example, Yandex can use to make Alice smarter. Jarvis will be sold already with a set of behaviors, which should simplify implementation and further training.

Nvidia presentation review: Jarvis artificial intelligence and the future of gaming graphics

And the slide below is just an example Nvidia did with Omniverse. The demo bot not only responds, but it even has facial expressions. The bot was trained to answer questions about the weather. And in the context of the coldest city in the world, Yakutsk was mentioned. This is how Russia celebrated at the presentation of the GTC 2020. We are proud!

Nvidia presentation review: Jarvis AI and the future of gaming graphics

And you can watch the chatbot in action on the video here:

Two news we'll miss: Nvidia EGX A100 and EGX Jetson Xavier NX

A100 is a GPU for data centers. If we come across these solutions, we are unlikely to know, since the A100 can, for example, control hundreds of cameras in airports, while the EGX Jetson Xavier NX is suitable for controlling several cameras in small shops.

As Mr. Huang described it, “The NVIDIA EGX Edge AI platform transforms a standard server into a small, secure, AI-enabled cloud data center. With NVIDIA AI frameworks, companies can build AI services ranging from smart retail and robotic factories to automated call centers.”

Nvidia keynote review: Jarvis AI and the future of gaming graphics

All of this is based on the new NVIDIA Ampere architecture. To quote the official press release:

NVIDIA Ampere Architecture - NVIDIA's 8th GPU architecture - provides the largest performance boost for a wide range of demanding workloads, including AI inference and 5G edge applications. This allows the EGX A100 to process large amounts of real-time data from cameras and other IoT sensors to quickly gain insights and improve business efficiency.

With the NVIDIA Mellanox ConnectX-6 Dx network card, the EGX A100 can receive up to 200 Gbps of data and send it directly to GPU memory for AI or 5G signal processing. With NVIDIA Mellanox time-driven telecom technology (5T for 5G), the cloud-based, software-defined accelerator EGX A100 is capable of handling most latency-sensitive 5G tasks. This creates a full-fledged AI/5G platform for real-time decision-making directly at the scene - stores, hospitals and factories.

"Together with NVIDIA, we are building a high-performance 5G virtualized radio access network and an accelerated 5G packet network," said Pardeep Kohli, President and CEO of Mavenir. “This will enable us to provide a wide range of GPU-accelerated 5G services, from AI and machine learning to augmented and virtual reality.”

Nvidia Ampere in smart cars

Nvidia Ampere can be used in smart cars. The trick is that, relatively speaking, you only need this board, which is in the picture. All calculations will take place in it, there is also a smart trained AI.

Nvidia presentation review: Jarvis AI and the future of gaming graphics Nvidia Presentation Review: artificial intelligence Jarvis and the future of graphics in games

Nvidia's smart car platform is open, so it's being enjoyed all over the world. Mostly Chinese and Indians. Also Mercedes and Toyota. Where is Tesla?

Nvidia presentation review: Jarvis AI and the future of gaming graphics

Conclusion

Nvidia is briskly striding forward. I guess you, like me, were primarily interested in consumer stuff like DLSS 2.0. But in general, everything presented by Nvidia is designed for business users. Even DLSS 2.0 should reduce the load on streaming data centers. It seems that while Microsoft has opened up Azure and the cloud as a new source of growth, Nvidia is betting on rethinking the computing market and artificial intelligence. The only vulnerable point may look like the fact that the company still prefers to create hardware and infrastructure that other companies will already configure, build data centers, and then sell computing capacities.

Nvidia presentation overview: AI Jarvis and the future of graphics in games

May 14, Nvidia held an introductory presentation for the GTC. GTC, or GPU Technology Conference, is an event for specialists, there are no consumer products there. At GTC 2020, they talked about smart cars, servers, data centers and artificial intelligence. Let's see where the graphics industry is heading. I want to highlight a number of interesting points.

Content

Data center as a computer unit

But let's start with the coronavirus. This topic could not be ignored. Solutions from Nvidia help in all areas, from deciphering genomes to using them in medical bots.

By the way, in their presentation, Nvidia showed a model of the University of Austin, but I will assume that the Russian company Visual Science also used graphics from Nvidia to create their detailed model of the SARS-CoV-2 virus.

Nvidia CEO Jensen Huang shared his vision for the development of the industry. According to him, there are two main forces pushing the development of computer computing in recent years. In the first place is machine learning, that is, building a new one based on already known data.

The second driver is the growth in the complexity of tasks and, accordingly, the growth in the size of applications, when the computer alone cannot cope. This has led to an increase in the number of data centers. According to Mr. Huang, we are moving towards the fact that not a computer or a server, but a whole data center will be taken as one computing unit. Accordingly, the industry is faced with a new problem of how to create, transfer, store and optimize data so that it can be used as productively as possible at the level of data centers.

Perhaps as a consumer inference from this prediction, the game streaming model, where the powerful hardware to play the game is somewhere in the cloud, will become the main type of gaming for the majority of the population.

To speed up data delivery (i.e. networking), Nvidia bought Mellanox. This appears to have been the biggest deal in Nvidia's history. Mellanox manufactures equipment for high-speed networks with a focus on supercomputers and big data data centers.

Artificial intelligence in graphics creation

Mr. Huang started from afar. From a story about how 40 years ago one of the company's employees described a model of the behavior of light on an object.

The ball rolls, the surface is reflected in it, the light plays

It was only 38 years later, in 2018, that Nvidia was able to bring the model to the masses in the form of a new generation of GeForce RTX cards with two innovative features: real-time ray tracing and artificial intelligence aimed at completing and improving the picture.< /p>

According to Mr. Huang, even after so much time since the creation of the model, the power of video cards for ray tracing was not enough, so machine learning just came to the rescue. In recent years, Nvidia has been actively training its AI. He is given a 540p picture, and his goal is to synthesize a Full HD picture based on what he has learned.

This is probably the most exciting part of the entire presentation. For training, Nvidia made a 16K model and ran the AI ​​"trillions of times" through it. And then the trained AI is delivered to user computers in the form of drivers. This technology is called DLSS, or Deep Learning Super Sampling, that is, deep learning on super samples (still not super when the original model is 16K).

Our great site is desperately squeezing all the pictures, so under this paragraph, I'll leave a link to a video with the timing of this place. Better watch in 4K!

So, first the picture was shown in the original 16K.

Then they showed how the video card itself can display it in 720p.

Next, they showed how the first version of DLSS 1.0 improved the picture by taking a 720p source and trying to "stretch" it. Here it is immediately worth noting that DLSS 1.0 did not take off, since in order to train AI for each task (game), it was necessary to create a separate training ground. And this is not an easy and expensive task, so the developers simply sabotaged DLSS 1.0.

But Nvidia believed in success. About a month ago, the company introduced DLSS 2.0. This technology is likely to succeed, as it is more viable. You no longer need to create a separate training ground for each game. But the best thing is that DLSS 2.0 allows you to get better image quality than the Full HD picture that the card can display on its own.

The thing is that the video card simply renders the image based on its capabilities. And DLSS 2.0 knows (or thinks it does) what the source image looks like at super-large resolution, so the AI ​​builds on even things that don't exist. And, perhaps, this is the main advantage, why you should buy RTX cards and not GTX cards. There are no so-called Tensor Cores in GTX - these are separate processors for artificial intelligence.

Video from mark: 2.35:

For example, here is a performance boost provided by DLSS 2.0 technology in Minecraft.

And along the way, Nvidia boasted that all the cool guys work with its graphics. Take a bite, AMD! It's certainly worth pointing out here that Nvidia is long overdue for burying the hatchet and making friends with Apple again, as many in the movie industry use Mac Pros to render graphics. Although everything is relative, for example, the picture shows the Pixar studio, which, as it were, historically uses Apple products, but, apparently, does not forget about Nvidia either.

But in general, the story was served under the sauce that Nvidia tools allow several users to work on 3D models at the same time. And all this is possible with the help of a new server, studded with a bunch of RTX cards. Buy, stick cards, and go! In fact, a super hit, especially in conditions of self-isolation, and indeed. The designer is sitting somewhere in Hawaii and making a new cartoon from Pixar with other dudes around the world. Only high-speed Internet is needed.

For example, a roller with a ball. It is notable for the lighting, shadows, natural physics, and the fact that it was made together by designers and engineers sitting all over the country.

About high-performance computers

In the high-end computing business, Mr. Huang told boring things to the average consumer. In short, Spark, or Apache Spark, is an analytics engine for working with big data. So Nvidia says: what if our cool work is used to speed up data processing?

For example, let Spark use video card processors and video card memory to process data! And let Spark plan and share the work so that everything is considered everywhere and at once at the same time - both in the processor and in the video card! And let's make a special library that Spark can access. And let's call it all Spark 3.0. So they did there.

As a result, everything began to work much faster. As proof, they showed a certain superserver from Dell, which, in addition to a bunch of speed characteristics, costs a whole million dollars and consumes 16 kW! And gives a data processing speed of 17 Gb / s. And if you make a Spark 3.0 superserver with parts from Nvidia, then it will cost $ 2 million, but the data processing speed will be 163 Gb / s. And all scientists and big data processors should be thrilled, because it's only twice as expensive, but 10 times more productive. Well, a data center needs at least a dozen of these cabinets.

And at the end, Jensen Huang threw up his hands like that and said: “The more you buy, the more you save!”.

On this, I think, the discussion of this topic can be completed. For the thing, of course, is good, but I'm not sure what I can afford right now. Although, of course, one or two lockers for the future will have to be bought. In the meantime, let Google and Microsoft test the new technology.

Nvidia recommendation system

Another technology that the average user won't use, but will definitely come across. Nvidia introduced the Merlin recommendation system, which is suitable for everything from movies and music to clothing.

Although the topic of recommendations seems boring, it is super important. Just remember how Yandex.Music unexpectedly shot after Yandex introduced a new recommendation system. The service turned from an ugly duckling into a candy!

Each recommendation service is a complex system where each user is passed through filters, and the system tries to choose the most appropriate one from billions of options. All this requires both computing power and artificial intelligence. Nvidia, they say, will make this technology available to the masses. The company introduced the Nvidia Merlin. This is a framework, that is, a software infrastructure for creating recommender systems. Merlin allows you to significantly speed up the learning process of predictive AI. As an example, they cited the fact that a typical system for 1 TB of some data takes 1.5 days to train, and Merlin can do it in 16 minutes. Well, in order for all this to work, of course, you need RTX servers with Tensor processors. Imagine how Yandex Music will get even prettier if Yandex has enough money to buy it. For although it was said about democratization, they did not name the price.

Jarvis is an artificial intelligence that can communicate

Probably, Nvidia thought: since we are so cool and make products that allow us to dramatically speed up the analysis, selection and prediction of the behavior of objects, then why not offer a solution for conversational artificial intelligence. In fact, this type of AI is faced with an abundance of complex tasks that require significant computing power - you need to recognize colloquial speech, guess what exactly the user means, find it, then try to answer him naturally and clearly. Despite the fact that there are already quite a lot of conversational assistants, they know little and are significantly limited. Nvidia invented Jarvis. This is not a specific voice assistant, but only a framework that, for example, Yandex can use to make Alice smarter. Jarvis will be sold already with a set of behaviors, which should simplify implementation and further training.

And the slide below is just an example Nvidia did with Omniverse. The demo bot not only responds, but it even has facial expressions. The bot was trained to answer questions about the weather. And in the context of the coldest city in the world, Yakutsk was mentioned. This is how Russia celebrated at the presentation of the GTC 2020. We are proud!

And you can watch the chatbot in action on the video here:

Two news we'll miss: Nvidia EGX A100 and EGX Jetson Xavier NX

A100 is a GPU for data centers. If we come across these solutions, we are unlikely to know, since the A100 can, for example, control hundreds of cameras in airports, while the EGX Jetson Xavier NX is suitable for controlling several cameras in small shops.

As Mr. Huang described it, “The NVIDIA EGX Edge AI platform transforms a standard server into a small, secure, AI-enabled cloud data center. With NVIDIA AI frameworks, companies can build AI services ranging from smart retail and robotic factories to automated call centers.”

All of this is based on the new NVIDIA Ampere architecture. To quote the official press release:

NVIDIA Ampere Architecture - NVIDIA's 8th GPU architecture - provides the largest performance boost for a wide range of demanding workloads, including AI inference and 5G edge applications. This allows the EGX A100 to process large amounts of real-time data from cameras and other IoT sensors to quickly gain insights and improve business efficiency.

With the NVIDIA Mellanox ConnectX-6 Dx network card, the EGX A100 can receive up to 200 Gbps of data and send it directly to GPU memory for AI or 5G signal processing. With NVIDIA Mellanox time-driven telecom technology (5T for 5G), the cloud-based, software-defined accelerator EGX A100 is capable of handling most latency-sensitive 5G tasks. This creates a full-fledged AI/5G platform for real-time decision-making directly at the scene - stores, hospitals and factories.

"Together with NVIDIA, we are building a high-performance 5G virtualized radio access network and an accelerated 5G packet network," said Pardeep Kohli, President and CEO of Mavenir. “This will enable us to provide a wide range of GPU-accelerated 5G services, from AI and machine learning to augmented and virtual reality.”

Nvidia Ampere in smart cars

Nvidia Ampere can be used in smart cars. The trick is that, relatively speaking, you only need this board, which is in the picture. All calculations will take place in it, there is also a smart trained AI.

Nvidia's smart car platform is open, so it's being enjoyed all over the world. Mostly Chinese and Indians. Also Mercedes and Toyota. Where is Tesla?

Conclusion

Nvidia is briskly striding forward. I guess you, like me, were primarily interested in consumer stuff like DLSS 2.0. But in general, everything presented by Nvidia is designed for business users. Even DLSS 2.0 should reduce the load on streaming data centers. It seems that if Microsoft has opened Azure and clouds as a new source of growth, then Nvidia is betting on rethinking the market for computing and artificial intelligence. The only vulnerable point may look like the fact that the company still prefers to create hardware and infrastructure that other companies will already configure, build data centers, and then sell computing capacities.

Nvidia is briskly striding forward. I guess you, like me, were primarily interested in consumer stuff like DLSS 2.0. But in general, everything presented by Nvidia is designed for business users. Even DLSS 2.0 should reduce the load on streaming data centers. There is a feeling that if Microsoft has opened Azure and clouds as a new source of growth, Nvidia is betting on rethinking the market for computing and artificial intelligence. The only weak point may be that the company still prefers to create hardware and infrastructure that other companies will already configure, build data centers, and then sell computing capacities.