Semantics-Controlled Gaussian Splatting for Outdoor Scene Reconstruction and Rendering in Virtual Reality
Schieber H, Young J, Langlotz T, Zollmann S, Roth D (2025)
Publication Type: Conference contribution
Publication year: 2025
Publisher: Institute of Electrical and Electronics Engineers Inc.
Pages Range: 318-328
Conference Proceedings Title: Proceedings - 2025 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2025
Event location: Saint-Malo, FRA
ISBN: 9798331536459
DOI: 10.1109/VR59515.2025.00056
Abstract
Advancements in 3D rendering like Gaussian Splatting (GS) allow novel view synthesis and real-time rendering in virtual reality (VR). However, GS-created 3D environments are often difficult to edit. For scene enhancement or to incorporate 3D assets, segmenting Gaussians by class is essential. Existing segmentation approaches are typically limited to certain types of scenes, e.g., "circular"scenes, to determine clear object boundaries. However, this method is ineffective when removing large objects in non-"circling"scenes such as large outdoor scenes.We propose Semantics-Controlled GS (SCGS), a segmentation-driven GS approach, enabling the separation of large scene parts in uncontrolled, natural environments. SCGS allows scene editing and the extraction of scene parts for VR. Additionally, we introduce a challenging outdoor dataset, overcoming the "circling"setup. We outperform the state-of-the-art in visual quality on our dataset and in segmentation quality on the 3D-OVS dataset. We conducted an exploratory user study, comparing a 360-video, plain GS, and SCGS in VR with a fixed viewpoint. In our subsequent main study, users were allowed to move freely, evaluating plain GS and SCGS. Our main study results show that participants clearly prefer SCGS over plain GS. We overall present an innovative approach that surpasses the state-of-the-art both technically and in user experience.
Involved external institutions
Germany (DE)
New Zealand (NZ)How to cite
APA:
Schieber, H., Young, J., Langlotz, T., Zollmann, S., & Roth, D. (2025). Semantics-Controlled Gaussian Splatting for Outdoor Scene Reconstruction and Rendering in Virtual Reality. In Proceedings - 2025 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2025 (pp. 318-328). Saint-Malo, FRA: Institute of Electrical and Electronics Engineers Inc..
MLA:
Schieber, Hannah, et al. "Semantics-Controlled Gaussian Splatting for Outdoor Scene Reconstruction and Rendering in Virtual Reality." Proceedings of the 32nd IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2025, Saint-Malo, FRA Institute of Electrical and Electronics Engineers Inc., 2025. 318-328.
BibTeX: Download