Progressive refinement imaging with depth-assisted disparity correction
Kluge M, Weyrich T, Kolb A (2023)
Publication Type: Journal article
Publication year: 2023
Journal
Book Volume: 115
Pages Range: 446-460
DOI: 10.1016/j.cag.2023.07.036
Abstract
In recent years, the increasing on-board compute power of mobile camera devices gave rise to a class of digitization algorithms that dynamically fuse a stream of camera observations into a progressively updated scene representation. Previous algorithms either obtain general 3D surface representations, often exploiting range maps from a depth camera, such as, Kinect Fusion, etc.; or they reconstruct planar (or distant spherical, respectively) 2D images with respect to a single (perspective or orthographic) reference view, such as, panoramic stitching or aerial mapping. Our work sets out to combine aspects of both, reconstructing a 2.5-D representation (color and depth) as seen from a fixed viewpoint, at spatially variable resolution. Inspired by previous work on “progressive refinement imaging”, we propose a hierarchical representation that enables progressive refinement of both colors and depths by ingesting RGB-D images from a handheld depth camera that is carried through the scene. We evaluate our system by comparing it against state-of-the-art methods in 2D progressive refinement and 3D scene reconstruction, using high-detail indoor and outdoor data sets comprising medium to large disparities. As we will show, the restriction to 2.5-D from a fixed viewpoint affords added robustness (particularly against self-localization drift, as well as backprojection errors near silhouettes), increased geometric and photometric fidelity, as well as greatly improved storage efficiency, compared to more general 3D reconstructions. We envision that our representation will enable scene exploration with realistic parallax from within a constrained range of vantage points, including stereo pair generation, visual surface inspection, or scene presentation within a fixed VR viewing volume.
Authors with CRIS profile
Involved external institutions
Germany (DE)How to cite
APA:
Kluge, M., Weyrich, T., & Kolb, A. (2023). Progressive refinement imaging with depth-assisted disparity correction. Computers & Graphics, 115, 446-460. https://dx.doi.org/10.1016/j.cag.2023.07.036
MLA:
Kluge, Markus, Tim Weyrich, and Andreas Kolb. "Progressive refinement imaging with depth-assisted disparity correction." Computers & Graphics 115 (2023): 446-460.
BibTeX: Download