NRMVS: Non-Rigid Multi-View Stereo

Innmann M, Kim K, Gu J, Nießner M, Loop C, Stamminger M, Kautz J (2020)

Publication Language: English

Publication Type: Conference contribution, Original article

Publication year: 2020

Publisher: IEEE

Pages Range: 2754 - 2763

Event location: Aspen, Colorado, USA US

URI: https://www.lgdv.tf.fau.de/?p=1730

DOI: 10.1109/WACV45572.2020.9093583

Open Access Link: http://openaccess.thecvf.com/content_WACV_2020/papers/Innmann_NRMVS_Non-Rigid_Multi-view_Stereo_WACV_2020_paper.pdf

Abstract

Multi-view Stereo (MVS) is a common solution in photogrammetry applications for the dense reconstruction of a static scene from images. The static scene assumption, however, limits the general applicability of MVS algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes, faces, or human bodies. In this paper, we open up a new challenging direction: Dense 3D reconstruction of scenes with non-rigid changes observed from a small number of images sparsely captured from different views with a single monocular camera, which we call non-rigid multi-view stereo (NRMVS) problem. We formulate this problem as a joint optimization of deformation and depth estimation, using deformation graphs as the underlying representation. We propose a new sparse 3D to 2D matching technique with a dense patch-match evaluation scheme to estimate the most plausible deformation field satisfying depth and photometric consistency. We show that a dense reconstruction of a scene with non-rigid changes from a few images is possible, and demonstrate that our method can be used to interpolate novel deformed scenes from various combinations of deformation estimates derived from the sparse views.

Authors with CRIS profile

Matthias Innmann Chair of Visual Computing Marc Stamminger Chair of Visual Computing

Involved external institutions

Nvidia Corporation US United States (USA) (US) Technische Universität München (TUM) DE Germany (DE)

How to cite

APA:

Innmann, M., Kim, K., Gu, J., Nießner, M., Loop, C., Stamminger, M., & Kautz, J. (2020). NRMVS: Non-Rigid Multi-View Stereo. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 2754 - 2763). Aspen, Colorado, USA, US: IEEE.

MLA:

Innmann, Matthias, et al. "NRMVS: Non-Rigid Multi-View Stereo." Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Aspen, Colorado, USA IEEE, 2020. 2754 - 2763.

BibTeX: Download