Nvidia Research has created a new AI model for 3D reconstruction. Neuralangelo uses neural networks to convert 2D video clips into detailed 3D buildings, sculptures and other physical world objects. The AI model allows users to generate 3D structures with intricate levels of detail. The 3D objects can then be imported into design programs for further editing.
Neuralangelo can also translate complex textures like roof shingles and glass panes from 2D clips to 3D objects. With its high fidelity, the AI model’s reconstructions are easier for developers to quickly create viable virtual objects using smartphone video footage.
“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,” says senior director of research and co-author of the paper Ming-Yu Liu. “This tool will eventually enable developers to import detailed objects — whether small statues or massive buildings — into virtual environments for video games or industrial digital twins.”
Neuralangelo Can Recreate Masterpieces
Nvidia Researchers showcased a demo of how Neuralangelo can recreate iconic objects such as Michelangelo’s David. The model is also capable of reconstructing building interiors which Nvidia showcased with a 3D model of the park at Nvidia’s Bay Area campus.
In the past, AI models have failed to reconstruct repetitive texture patterns and strong colour variations accurately. However, Neuralangelo is able to do so by adopting instant neural graphics primitives to capture fine details. The model uses 2D videos of an object taken from different angles to create a 3D object. First, the AI model makes a rough 3D representation. Then it optimises it and renders fine details.
Nvidia Research will present Neuralangelo along with nearly 30 other projects at the Conference on Computer Vision and Pattern Recognition (CVPR). The event will take place Jun 18th to 22nd in Vancouver. Nvidia also recently announced Nvidia ACE, a new custom AI model foundry service for games.
