NRMVS: Non-Rigid Multi-View Stereo

Journal article

Publication Details

Author(s): Innmann M, Kim K, Gu J, Niessner M, Loop C, Stamminger M, Kautz J
Journal: arXiv
Publication year: 2019
ISSN: 2331-8442


Scene reconstruction from unorganized RGB images is an important task in many
computer vision applications. Multi-view Stereo (MVS) is a common solution in
photogrammetry applications for the dense reconstruction of a static scene. The
static scene assumption, however, limits the general applicability of MVS
algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes,
faces, or human bodies. In this paper, we open up a new challenging direction:
dense 3D reconstruction of scenes with non-rigid changes observed from
arbitrary, sparse, and wide-baseline views. We formulate the problem as a joint
optimization of deformation and depth estimation, using deformation graphs as
the underlying representation. We propose a new sparse 3D to 2D matching
technique, together with a dense patch-match evaluation scheme to estimate
deformation and depth with photometric consistency. We show that creating a
dense 4D structure from a few RGB images with non-rigid changes is possible,
and demonstrate that our method can be used to interpolate novel deformed
scenes from various combinations of these deformation estimates derived from
the sparse views.

FAU Authors / FAU Editors

Innmann, Matthias
Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)
Stamminger, Marc Prof. Dr.
Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)

External institutions
Technische Universität München (TUM)

How to cite

Innmann, M., Kim, K., Gu, J., Niessner, M., Loop, C., Stamminger, M., & Kautz, J. (2019). NRMVS: Non-Rigid Multi-View Stereo. arXiv.

Innmann, Matthias, et al. "NRMVS: Non-Rigid Multi-View Stereo." arXiv (2019).


Last updated on 2019-23-01 at 11:38