I3DMM: Deep implicit 3D morphable model of human heads

Yenamandra T, Tewari A, Bernard F, Seidel HP, Elgharib M, Cremers D, Theobalt C (2021)

Publication Type: Conference contribution

Publication year: 2021

Journal

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition IEEE Computer Society; 1999

Publisher: IEEE Computer Society

Pages Range: 12798-12808

Conference Proceedings Title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Event location: Virtual, Online, USA

ISBN: 9781665445092

DOI: 10.1109/CVPR46437.2021.01261

Abstract

We present the first deep implicit 3D morphable model (i3DMM) of full heads. Unlike earlier morphable face models it not only captures identity-specific geometry, texture, and expressions of the frontal face, but also models the entire head, including hair. We collect a new dataset consisting of 64 people with different expressions and hairstyles to train i3DMM. Our approach has the following favorable properties: (i) It is the first full head morphable model that includes hair. (ii) In contrast to mesh-based models it can be trained on merely rigidly aligned scans, without requiring difficult non-rigid registration. (iii) We design a novel architecture to decouple the shape model into an implicit reference shape and a deformation of this reference shape. With that, dense correspondences between shapes can be learned implicitly. (iv) This architecture allows us to semantically disentangle the geometry and color components, as color is learned in the reference space. Geometry is further disentangled as identity, expressions, and hairstyle, while color is disentangled as identity and hairstyle components. We show the merits of i3DMM using ablation studies, comparisons to state-of-the-art models, and applications such as semantic head editing and texture transfer. We will make our model publicly available.

Involved external institutions

Max Planck Institute for Informatics / Max-Planck-Institut für Informatik DE Germany (DE) Technische Universität München (TUM) DE Germany (DE)

How to cite

APA:

Yenamandra, T., Tewari, A., Bernard, F., Seidel, H.-P., Elgharib, M., Cremers, D., & Theobalt, C. (2021). I3DMM: Deep implicit 3D morphable model of human heads. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 12798-12808). Virtual, Online, USA: IEEE Computer Society.

MLA:

Yenamandra, Tarun, et al. "I3DMM: Deep implicit 3D morphable model of human heads." Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021, Virtual, Online, USA IEEE Computer Society, 2021. 12798-12808.

BibTeX: Download