How Gaming Engines and AI are Shaping AV


In 2016 I wrote a blog called, “Why aren’t we talking about NVIDIA?” If you missed it, the general conclusion was that AI and large-scale video applications would drive NVIDIA to success well beyond gaming. When I wrote that blog in April of 2016, NVIDIA was trading at $8.50.

Today, NVIDIA is at $776 with a $1.92 trillion market cap.

Analysts are debating as to whether NVIDIA will hold onto its first mover advantage long term, but whether or not the company holds the top spot is beside the point today. The takeaway is that NVIDIA’s success is due to capitalizing on a set of larger trends going on in the marketplace that we should be capitalizing on as well.


The easiest mental bridge from home gaming PCs to commercial environments that need the same type of computing power is the eSports vertical. If you understand the importance of GPUs in gaming, then taking that potential and multiplying it for the scale of a eSports lab or arena is easy math. eSports though is only the tip of the iceberg.

Virtual Production and XR

Virtual production and XR have been hot topics as of late, and their impact will extend well beyond movie and broadcast studios and into enterprise applications, higher education, and live events. If you understand XR, you know how the engines can “extend” the digital environment beyond the LED volumes to create scale on camera that doesn’t exist in studio. Gaming engines like Unreal and Unity are at the heart of this technology and rendering real time pixels takes some processing power. Add in things like multi camera tracking, AR layers, and ghost frames, and all of a sudden you can triple the needed GPU resources.

Immersive environments

By definition, immersive environments surround those experiencing them with sound and video. Creating a large curved, spherical, or faceted multi-screen display requires some computing in the background, and bespoke media servers with high end GPUs are at the heart of them. Today, these systems often leverage Unreal or Unity gaming engines to replicate environments ranging from mock ups of buildings to digital twins of factories.  Of course, rendering pixels real time relies on state of the art GPU power, and these environments are gaining steam again as 3D gaming engines are used more and more for modeling physical environments and visualizing data.

See related  SPIRE Academy Adds Esports Training Program


MicroLED increases pixel density, which in turn increases pixel count given that images are the same dimensions. Increased pixel count, coupled with deeper bit depths and higher refresh rates equal the need for some amazing processing power that goes well beyond the capabilities of digital signage players. It is also interesting to note that a standard LED controller doesn’t do any scaling or processing, so if you have odd aspect ratio screens or content in resolutions that differ from the native resolutions of your microLED, then you’ll need to add some processing between those sources and the controller. There is a definite shift toward microLED happening, especially as costs continue to decrease on sub 1mm pixel pitch LED, and this is an area we should be continuing to focus on.


I know we’re all exhausted from the AR/VR/Metaverse talk, but I think theses areas will expand at a more rapid pace due to some of the other trends above. I believe one of the hurdles to AR/VR/Metaverse has been content creation, but with the increased use of 3D render engines in these other physical environments, it opens the door to leveraging those assets across virtual platforms as well to maximize ROI and enhance remote communication and collaboration.


The world has been talking a lot about AI, especially generative AI for creating video, imagery, ambient art, as well as for things like professional writing, etc. There are about 25-30 other types of AI beyond generative, and artificial intelligence requires parallel processing to make real time decisions based on sensors, data, and knowledge bases. This type of intelligence requires parallel processing that is ideal for GPUs to do, which is why NVIDIA has been so relevant in this area. Machine vision, camera tracking, predictive analytics, and generative art are just the beginning of AI.