OpenAI Introduces POINT-E, a Text-to-image AI System

Highlights:

• Point-E’s text-to-image model generates a synthetic rendered object that’s fed to the image-to-3D model, which then generates a point cloud.
• The thing to note is that the model can sometimes miss certain parts of objects, resulting in distorted or blocky images.

OpenAI LLC this week detailed POINT – E. It is a new Artificial Intelligence (AI) system that helps in generating 3D models based on text prompts. The research group has made the code for POINT-E available on GitHub.

There are numerous AI applications that can generate 2D images based on a user-supplied text description. When running on a single data center graphics card, such applications, according to OpenAI, render images in a matter of seconds or minutes. Generating a 3D model typically takes a few hours when using comparable hardware. POINT-E by OpenAI will speed up this process.

According to the research group, the new AI system can also generate a 3D model in as little as one minute when running on an Nvidia V100 graphics card.

The AI system doesn’t directly generate a 3D model of the object, after receiving a user prompt describing an object. What it actually does is create a 2D drawing of that object. From that point, POINT-E turns the 2D drawing into a 3D point cloud, and this basic version simply works as an outline.

Different neural networks carry out each step of the process. The first step of turning object description into 2D drawing, is carried out by a neural network dubbed GLIDE that OpenAI released last year. This GLIDE version used in POINT-E has three billion parameters. The configuration settings that define how a neural network processes data are known as parameters.

After POINT-E generates a 2D drawing of an object, the drawing is turned into a point cloud by two different neural networks. The first neural network produces a basic, low-resolution point cloud with 1,000 pixels. In the second version, it adds 3,000 more pixels to increase the point cloud’s resolution and is the simpler version of the first.

OpenAI scientists explained POINT-E in a research paper by saying, “For image diffusion models, the best quality is typically achieved by using some form of hierarchy, where a low-resolution base model produces output which is then upsampled by another model. Our upsampler uses the same architecture as our base model.”

The neural networks mentioned above are based on diffusion, which is a machine-learning method. The method, introduced in 2015, also powers an image-generation AI that Google LLC launched earlier this year.

To build a diffusion model, engineers create images that consist of a type of error known as Gaussian noise. They then use the model to remove the noise. After repeating the process a few times, a neural network can learn techniques allowing it to make images from scratch.

POINT-E creates a point cloud of an object and then uses Blender, an open-source graphic design application, to convert the point cloud into a 3D model. Typically, an automated script manages the process of creating a 3D model in Blender.

OpenAI’s researchers added details, “While our method performs worse on this evaluation than state-of-the-art techniques, it produces samples in a small fraction of the time. This could make it more practical for certain applications, or could allow for the discovery of higher-quality 3D objects by sampling many objects and selecting the best one.”