Researchers at MIT have proposed a simple fix to an existing technique that could help artists, designers, and engineers create better 3D models by leveraging 2D image generation models to create 3D shapes with improved quality. The new method infers the missing term from the current 3D shape rendering, rather than randomly sampling noise at each step, achieving results on par with or better than other approaches without additional training or complex postprocessing.
Researchers at MIT have proposed a simple fix to an existing technique that could help artists, designers, and engineers create better 3D models. The new method leverages 2D image generation models to create 3D shapes, but its output often ends up blurry or cartoonish.
The problem with Score Distillation is that it uses 2D image generation models to create 3D shapes, resulting in lower quality compared to the best model-generated 2D images. The root cause of this issue lies in the algorithms used to generate 2D images and 3D shapes, specifically the noise term in Score Distillation leading to blurry or cartoonish 3D shapes.
Instead of trying to solve the complex formula precisely, the researchers tested approximation techniques until they identified the best one. Their technique infers the missing term from the current 3D shape rendering, rather than randomly sampling noise at each step. The results show that the new method achieves 3D shape quality on par with or better than other approaches without additional training or complex postprocessing.
The MIT researchers’ technique is a significant improvement over existing methods for creating realistic 3D shapes using generative AI. By identifying the cause of the problem and applying approximation techniques, they were able to create smooth, realistic-looking 3D shapes without the need for costly retraining.
