Nvidia Introduces Omniverse Updates with Generative AI OpenUSD

Highlights:

• Generative AI and OpenUSD enable novel app and 3D environment creation.
• With ChatUSD, developers access a generative AI that generates Python-USD code from text and answers USD-related questions.

Nvidia Corp. recently announced a major update to Omniverse, its hyper-realistic real-time 3D graphics collaboration and simulation platform, which will enable enterprises to create better 3D models and scenes by leveraging OpenUSD and artificial intelligence.

Nvidia Omniverse is a platform that provides a suite of “metaverse” tools for artists, engineers, creators, and others to collaborate remotely on large-scale virtual worlds to recreate and simulate anything they can imagine. They can use this powerful tool to simulate virtual worlds such as buildings, cars, airplanes, and factories for video games or industrial solutions such as “digital twins” that can react with actual physics, just like the real world.

Nvidia Omniverse updates announced recently at SIGGRAPH 2023 included improvements to its use of Universal Scene Description, also known as OpenUSD. This Nvidia-supported open infrastructure enables collaboration between artists and engineers on large-scale 3D projects. Among these were a new portfolio of cloud and generative AI application programming interfaces.

The use of generative AI and OpenUSD will enable developers to create entirely new applications and 3D environments. They may just input what they want, and AI will provide USD specs or renderings, saving them time and energy.

ChatUSD provides developers with access to a large language model that can generate Python-USD code scripts from text and respond to USD queries. It was developed using Nvidia’s NeMo artificial intelligence framework, which enables anyone to become an OpenUSD expert by rapidly bringing them up to speed and enabling them to write code. It works like OpenAI’s ChatGPT to generate computer code, but it’s trained on OpenUSD Python scripts and can work with whatever the artist or developer already has.

DeepSearch is a large language model agent capable of performing fast semantic searches through 3D models and untagged assets using text prompts. Many enterprise developers and artists have vast stores of 3D models at their disposal and constantly search for them, but they are not always in well-categorized or searchable stores. Having a search agent to whom they can simply ask questions is intended to make quick work of finding exactly what they are looking for.

Jensen Huang, Co-founder and Chief Executive of Nvidia, said, “Just as HTML ignited a major computing revolution of the 2D internet, OpenUSD will spark the era of collaborative 3D and industrial digitalization.”

Developers, artists, and engineers will be able to build advanced scene compositions on a large scale at the same time with access to the new Omniverse USD Composer and easily share them between colleagues and different applications, according to Nvidia.

Nvidia recently formed the Alliance for OpenUSD, an industry body that will build and develop OpenUSD, with Apple Inc., Adobe Inc., Autodesk Inc., and Walt Disney Co.’s Pixar Animation Studios earlier this month. Under the auspices of the alliance, the organization will work toward the adoption of OpenUSD and provide a written specification of the standard while advancing its development. Nvidia has also provided a roadmap of framework features that will improve its suitability for autonomous industrial robots and industrial AI.

Adobe and Nvidia also announced plans to bring Adobe Firefly, Adobe’s family of image-generating foundational AI models, into the Omniverse platform. Adobe introduced Firefly as a suite of artificial intelligence (AI) tools for image generation that operate similarly to prominent image generation models such as Stable Diffusion, Midjourney, and OpenAI LP’s DALL-E 2. The company added it to Photoshop in May, allowing users to use “generative fill” to modify images with text prompts.

Rev Lebaredian, Vice President of Omniverse and simulation technology at Nvidia, Rev Lebaredian, said, “The latest Omniverse update lets developers tap generative AI through OpenUSD to enhance their tools, and it allows enterprises to build larger, more complex world-scale simulations as digital testing grounds for their industrial applications.”

A new Kit Extension Registry will provide developers with a centralized repository for accessing and sharing Omniverse extensions, which are plugins within the application. This will provide them with an easy resource for discovering what they’re using and how it affects the application, as well as the ability to quickly turn on and off functions for building custom apps from Nvidia’s more than 600 core Omniverse extensions.

Nvidia additionally improved the performance of Omniverse by implementing new rendering optimizations that make use of the new Nvidia RTX graphics processing units with DLSS 3.0 technology present in Ada Lovelace architecture. The platform also includes a native RTX-powered extended-reality developer tool suite for creating “spatial computing” experiences for virtual and augmented reality applications.