Scan&Paint: Image-based Projection Painting

Klein V, Leuschner M, Langen T, Kurth P, Stamminger M, Bauer F (2021)

Publication Language: English

Publication Type: Conference contribution, Conference Contribution

Publication year: 2021

Series: IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Book Volume: 20

Pages Range: 517-525

Conference Proceedings Title: 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

Event location: Bari IT

URI: https://www.lgdv.tf.fau.de/?p=2253

DOI: 10.1109/ISMAR52148.2021.00069

Abstract

We present a pop-up projection painting system that projects onto an unknown three-dimensional surface, while the user creates the projection content on the fly. The digital paint is projected immediately and follows the object if it is moved. If unexplored surface areas are thereby exposed, an automated trigger system issues new depth recordings that expand and refine the surface estimate. By intertwining scanning and projection painting we scan the exposed surface at the appropriate time and only if needed. Like image-based rendering, multiple automatically recorded depth maps are fused in screen space to synthesize novel views of the object, making projection poses independent from the scan positions. Since the user’s digital paint is also stored in images, we eliminate the need to reconstruct and parametrize a single full mesh, which makes geometry and color updates simple and fast.

Authors with CRIS profile

Vanessa Klein Chair of Visual Computing Markus Leuschner Chair of Visual Computing Tobias Langen Chair of Visual Computing Philipp Kurth Professur für Mittlere und Neuere Kunstgeschichte Marc Stamminger Chair of Visual Computing Frank Bauer Chair of Visual Computing

Related research project(s)

Projection Mapping Oct. 1, 2014 - Oct. 1, 2025

How to cite

APA:

Klein, V., Leuschner, M., Langen, T., Kurth, P., Stamminger, M., & Bauer, F. (2021). Scan&Paint: Image-based Projection Painting. In IEEE (Eds.), 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 517-525). Bari, IT.

MLA:

Klein, Vanessa, et al. "Scan&Paint: Image-based Projection Painting." Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bari Ed. IEEE, 2021. 517-525.

BibTeX: Download