Techniques for Fast and High-Quality 3D Reconstruction of General Scenes

Abstract: This thesis is a collection of techniques used for 3D reconstruction; the creation of 3D models from real world objects or scenes. Given the increase in accuracy, robustness and speed of modern methods and algorithms, new and exciting applications of this technology is constantly appearing. Asset creation for games and movies is a successful example, but there are numerous other applications in architecture, medicine, communication and more. The contribution of each paper in this thesis aims to make the use of 3D reconstruction even more ubiquitous by addressing problems such as performance, memory usage, ease-of-use, robustness and quality. Paper I presents a compression technique for volumetric video modeled with voxels. Memory consumption is an important issue when storing volume data, especially if the data is also varying with time. Paper II describes an end-to-end pipeline for recording and rendering volumetric video. A simple and readily available setup of webcams and a single desktop computer is used to record and render scenes in real-time. In Paper III , an interactive tool is developed that aims to help in modeling of real-world objects. Structured as a simple quad modeling program, the user can construct 3D models on top of a set of photographs of a chosen object. In the background, or after explicit activation, a multi-view stereo algorithm helps the user to align the geometry correctly to images in world space. This greatly simplifies the problem of modeling real world objects accurately, while levering the input from the user to help with topology and visibility. Paper IV implements a direct solver for the problem of neural rendering. The reconstruction is formulated as a non-linear least-squares problem which is solved efficiently with the Gauss Newton method and the Preconditioned Conjugate Gradient algorithm. This formulation achieves a significant improvement to reconstruction times compared to previous methods, while also being suitable for distributed computing due to needing three order of magnitudes fewer iterations until convergence. Paper V handles the shape-radiance ambiguity in neural rendering. Given infinite spatial resolution of view-dependent information, almost any shape can satisfy the incoming radiance to each camera, resulting in errors in the geometry. To address this problem, we propose a solution to separate Lambertian and view-dependent colors during reconstruction.