SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion

Wu SC, Tateno K, Navab N, Tombari F (2020)

Publication Type: Conference contribution

Publication year: 2020

Publisher: Institute of Electrical and Electronics Engineers Inc.

Pages Range: 801-810

Conference Proceedings Title: Proceedings - 2020 International Conference on 3D Vision, 3DV 2020

Event location: Virtual, Fukuoka, JPN

ISBN: 9781728181288

DOI: 10.1109/3DV50981.2020.00090

Abstract

Real-time scene reconstruction from depth data inevitably suffers from occlusion, thus leading to incomplete 3D models. Partial reconstructions, in turn, limit the performance of algorithms that leverage them for applications in the context of, e.g., augmented reality, robotic navigation, and 3D mapping. Most methods address this issue by predicting the missing geometry as an offline optimization, thus being incompatible with real-time applications. We propose a framework that ameliorates this issue by performing scene reconstruction and semantic scene completion jointly in an incremental and real-time manner, based on an input sequence of depth maps. Our framework relies on a novel neural architecture designed to process occupancy maps and leverages voxel states to accurately and efficiently fuse semantic completion with the 3D global model. We evaluate the proposed approach quantitatively and qualitatively, demonstrating that our method can obtain accurate 3D semantic scene completion in real-time.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Google Inc. US United States (USA) (US)

How to cite

APA:

Wu, S.-C., Tateno, K., Navab, N., & Tombari, F. (2020). SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion. In Proceedings - 2020 International Conference on 3D Vision, 3DV 2020 (pp. 801-810). Virtual, Fukuoka, JPN: Institute of Electrical and Electronics Engineers Inc..

MLA:

Wu, Shun-Cheng, et al. "SCFusion: Real-time Incremental Scene Reconstruction with Semantic Completion." Proceedings of the 8th International Conference on 3D Vision, 3DV 2020, Virtual, Fukuoka, JPN Institute of Electrical and Electronics Engineers Inc., 2020. 801-810.

BibTeX: Download