Nvidia to Announce Innovative AI Research for Computer Graphics

Highlights:

• The papers, written in collaboration with researchers from dozens of US, European, and Israeli universities, cover topics like generative AI models that convert text into images, inverse rendering tools that create 3D versions of still images, physics models that use AI to simulate complex 3D elements, and more.

Nvidia Corp. researchers have introduced a variety of developments aimed at assisting developers and artists in combining AI and computer graphics software to bring their creative ideas to life.

The business claims it will 18 new research papers outlining its advances at SIGGRAPH 2023, an annual computer graphics conference from August 6-10. The papers, written in collaboration with researchers from dozens of universities in the Europe, United States, and Israel, cover topics such as generative AI models that convert text into images, physics models that utilize AI to replicate complex 3D elements, inverse rendering tools that can create 3D versions of still images, and more.

Nvidia explained in a blog post that creators already have access to various text-to-image generative AI models. Such tools are used to create conceptual art, movie storyboards, video games, and 3D virtual environments. However, they remain limited, particularly when the artist has a precise vision in mind. For instance, an advertising executive planning a campaign for a new teddy bear may want to construct various scenarios showcasing the toy in multiple settings, such as a teddy bear tea party.

Nvidia’s researchers have developed a technique that enables generative AI models to use a single example image to customize their output in specific ways. Existing tools cannot produce this level of specificity, so this technique enables generative AI models to use a single example image to customize their output in very specific ways. A second method outlines a highly condensed model called Perfusion, which enables users to combine multiple personalized elements with a small number of concept images and generate more specific AI-generated visuals.

Nvidia researchers have also worked to speed up 2D-to-3D picture rendering process. A third research paper focuses on a novel technology that, according to Nvidia, can operate on a standard laptop and generate a photorealistic 3D head-and-shoulders model from a 2D portrait. According to the company, this is a significant technological advancement that will drastically accelerate the creation of 3D avatars, with substantial implications for videoconferencing and virtual reality applications.

In a separate initiative, Nvidia collaborated with Stanford University researchers to imbue 3D characters with convincing movement. Users can, for instance, input the model with a video of tennis matches and then transmit the realistic motion to a 3D tennis-playing character. Nvidia stated that the simulated user can engage in protracted battles with other characters. Without expensive motion-capture video data, the model can address the problem of producing 3D characters with diverse skills and realistic movement.

Nvidia uses AI to generate light reflection in virtual scenes using neural rendering. Its research demonstrates how artificial intelligence models for textures, materials, and volumes can generate photorealistic, film-quality visuals of objects in real-time for video games and virtual environments.

The business highlighted how its latest neural rendering compression methods might significantly boost the realism of such situations, capturing far more significant information than earlier formats, where, for example, the text of a book remains indistinct.

Finally, Nvidia researchers demonstrated their most recent advances in neural materials research. The article describes an AI system that can detect how light reflects from photorealistic, multilayered materials before reducing the complexity of these assets to simpler, real-time neural networks. According to Nvidia’s experts, the outcome is up to ten times quicker shading.

Nvidia stated that their most recent research will be available at this year’s SIGGRAPH conference. It hopes that developers and businesses will use its innovative approaches for creating synthetic characters and objects to fill in the virtual environments for applications like robotics and autonomous car training. Furthermore, it hopes that artists, filmmakers, architects, and video game designers will use its techniques to create higher-quality visuals than previously possible.