Real-time Non-rigid Reconstruction using an RGB-D Camera

Beitrag in einer Fachzeitschrift

Details zur Publikation

Autor(en): Izadi S, Rhemann C, Zach C, Fisher M, Wu C, Fitzgibbon A, Loop C, Theobald C, Stamminger M, Nießner M, Zollhöfer M
Zeitschrift: ACM Transactions on Graphics
Jahr der Veröffentlichung: 2014
Band: 33
Heftnummer: 4
Seitenbereich: 156:1-12
ISSN: 1557-7368


We present a combined hardware and software solution for marker-less reconstruction of non-rigidly deforming physical objects with arbitrary shape in real-time. Our system uses a single self-contained stereo camera unit built from off-the-shelf components and consumer graphics hardware to generate spatio-temporally coherent 3D models at 30 Hz. A new stereo matching algorithm estimates real-time RGB-D data. We start by scanning a smooth template model of the subject as they move rigidly. This geometric surface prior avoids strong scene assumptions, such as a kinematic human skeleton or a parametric shape model. Next, a novel GPU pipeline performs non-rigid registration of live RGB-D data to the smooth template using an extended non-linear as-rigid-as-possible (ARAP) framework. High-frequency details are fused onto the final mesh using a linear deformation model. The system is an order of magnitude faster than state-of-the-art methods, while matching the quality and robustness of many offline algorithms. We show precise real-time reconstructions of diverse scenes, including: large deformations of users' heads, hands, and upper bodies; fine-scale wrinkles and folds of skin and clothing; and non-rigid interactions performed by users on flexible objects such as toys. We demonstrate how acquired models can be used for many interactive scenarios, including re-texturing, online performance capture and preview, and real-time shape and motion re-targeting. Copyright © ACM.

FAU-Autoren / FAU-Herausgeber

Nießner, Matthias
Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)
Stamminger, Marc Prof. Dr.
Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)
Zollhöfer, Michael
Lehrstuhl für Informatik 9 (Graphische Datenverarbeitung)

Autor(en) der externen Einrichtung(en)
Microsoft Corporation
Stanford University


Izadi, S., Rhemann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A.,... Zollhöfer, M. (2014). Real-time Non-rigid Reconstruction using an RGB-D Camera. ACM Transactions on Graphics, 33(4), 156:1-12.

Izadi, Sharam, et al. "Real-time Non-rigid Reconstruction using an RGB-D Camera." ACM Transactions on Graphics 33.4 (2014): 156:1-12.


Zuletzt aktualisiert 2018-11-08 um 00:24