Disney Research, Z?rich has developed a new program that can build 3D models from 2D photographs.
Using hundreds of photographic images and a specially-designed algorithm, the program can create 3D models of complex, real-life scenes for movie, TV and games, and be used to make or print high-resolution models.
Building 3D models from multiple 2D images captured from a variety of viewing positions is nothing new, but doing so for highly detailed or cluttered environments at high resolution has proved difficult because of the large amounts of data involved.
The Disney Research, Z?rich team, however, developed an algorithm that can effectively leverage these amounts of data, and process them efficiently without the need to keep all of the input data in memory at one time.
The research is published in a paper called Scene Reconstruction from High Spatio-Angular Resolution Light Fields.
Many 3D models now are obtained using laser scanning. In complex, cluttered environments, however, a single laser scan misses a lot of detail because objects in the foreground can block the laser’s view. Photography makes it easier to capture the scene from multiple viewpoints, but combining photographs to build a 3D model is burdensome at high resolutions.
Disney Research’s algorithm first computes reliable depth estimates specifically around object boundaries instead of interior regions, by operating on individual light rays instead of image patches. More homogeneous interior regions are then processed in a fine-to-coarse procedure rather than the standard coarse-to-fine approaches. “This allows our algorithm to retain precise object contours while still ensuring smooth reconstructions in less detailed areas.” writes the paper.
According to Disney Research, this new method could be used to build and create accurate reconstructions of scenes for films and video games, or the models could be printed with all the advancements in 3D printing.
The researchers demonstrated their method by photographing a number of complex outdoor and indoor scenes with a standard DSLR camera, using 100 21-megapixel-resolution images to create each 3D reconstruction. Most existing stereo reconstruction techniques have been tailored for resolutions of just 1 or 2 megapixels.
The photos were captured along a linear path; this geometry provided structure that the researchers could leverage to make processing the data more efficient. However, the researchers also generalized their approach so that it can be applied even to a set of images taken with a hand-held camera.