OpenAI Is Back With Another Text-to-3D-Model Generator

OpenAI has released Shap-E, a new, AI-driven text-to-3D-model generator that at first glance appears similar to its recent Point-E.

Shap-E learns to generate 3D models by training an encoder to convert 3D objects into mathematical representations called implicit functions, then training a diffusion model to generate new 3D models based on those representations. The results are models with more-realistic textures and lighting effects – and that are created more quickly – than those generated by Point-E.

With Point-E, diffusion models are used to create 3D point clouds, which are then converted to mesh. A point cloud is a collection of individual points in 3D space that represent the shape of an object. These points are not connected, nor do they include explicit information about an object’s surface or structure. Here, the results are comparably rudimentary and low-resolution (though nonetheless impressive, in terms of a technological showcase).

Bottom-line, however: OpenAI claims Shap-E is capable of “comparable or better” performance than Point-E. The examples of the 3D models on Shap-E’s GitHub page – which houses model weights, inference code, and samples – seem to prove it, despite their general silliness (“an airplane that looks like a banana,” for instance).

Of course, as with Point-E, there still seems to be a long way to go until these types of model generators are useful to 3D printing without a human touch. Still, Shap-E is another exciting step toward realizing that achievement, with the implications of such innovative tech taking shape (no pun intended) being far-reaching and game-changing from multiple standpoints.

For those interested, an in-depth research paper dives further into the science behind Shap-E’s function, applications, limitations, and more is available for download.

You’ve read that; now read these:

• Snapmaker Releases J1 Controller Software As Open Source
• The New Race to the Bottom: Will the CoreXY Arms Race Pose a Problem to Open Source?
• This Redditor Commutes With Their Bambu Lab P1P