NRMVS: Non-Rigid Multi-View Stereo

Beitrag in einer Fachzeitschrift

Details zur Publikation

Autor(en): Innmann M, Kim K, Gu J, Niessner M, Loop C, Stamminger M, Kautz J
Zeitschrift: arXiv
Jahr der Veröffentlichung: 2019
ISSN: 2331-8442


Scene reconstruction from unorganized RGB images is an important task in many
computer vision applications. Multi-view Stereo (MVS) is a common solution in
photogrammetry applications for the dense reconstruction of a static scene. The
static scene assumption, however, limits the general applicability of MVS
algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes,
faces, or human bodies. In this paper, we open up a new challenging direction:
dense 3D reconstruction of scenes with non-rigid changes observed from
arbitrary, sparse, and wide-baseline views. We formulate the problem as a joint
optimization of deformation and depth estimation, using deformation graphs as
the underlying representation. We propose a new sparse 3D to 2D matching
technique, together with a dense patch-match evaluation scheme to estimate
deformation and depth with photometric consistency. We show that creating a
dense 4D structure from a few RGB images with non-rigid changes is possible,
and demonstrate that our method can be used to interpolate novel deformed
scenes from various combinations of these deformation estimates derived from
the sparse views.

FAU-Autoren / FAU-Herausgeber

Innmann, Matthias
Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)
Stamminger, Marc Prof. Dr.
Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)

Autor(en) der externen Einrichtung(en)
Technische Universität München (TUM)


Innmann, M., Kim, K., Gu, J., Niessner, M., Loop, C., Stamminger, M., & Kautz, J. (2019). NRMVS: Non-Rigid Multi-View Stereo. arXiv.

Innmann, Matthias, et al. "NRMVS: Non-Rigid Multi-View Stereo." arXiv (2019).


Zuletzt aktualisiert 2019-23-01 um 11:38