PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DOF Object Pose Dataset Generation

Meyer L, Erich F, Yoshiyasu Y, Stamminger M, Ando N, Domae Y (2024)

Publication Type: Conference contribution, Conference Contribution

Publication year: 2024

Event location: Abu Dhabi AE

URI: https://arxiv.org/abs/2401.02281

Abstract

We introduce Physically Enhanced Gaussian Splatting
Simulation System (PEGASUS) for 6DOF object pose dataset
generation, a versatile dataset generator based on 3D
Gaussian Splatting.

Environment and object representations can be easily
obtained using commodity cameras to reconstruct with
Gaussian Splatting. PEGASUS allows the    omposition of new
scenes by merging the respective underlying Gaussian
Splatting point cloud of an environment with one or
multiple objects. Leveraging a physics engine enables the
simulation of natural object placement within a scene
through interaction between meshes extracted for the
objects and the environment. Consequently, an extensive
amount of new scenes - static or dynamic - can be created
by combining different environments and objects. By
rendering scenes from various perspectives, diverse data
points such as RGB images, depth maps, semantic masks, and
6DoF object poses can be extracted.

Our study demonstrates that training on data generated by
PEGASUS enables pose estimation networks to successfully
transfer from synthetic data to real-world data. Moreover,
we introduce the Ramen dataset, comprising 30 Japanese cup
noodle items. This dataset includes spherical scans that
captures images from both object hemisphere and the
Gaussian Splatting reconstruction, making them compatible
with PEGASUS.

Authors with CRIS profile

Lukas Meyer Chair of Visual Computing Marc Stamminger Chair of Visual Computing

Involved external institutions

National Institute of Advanced Industrial Science and Technology / 産業技術総合研究所 (AIST) JP Japan (JP)

How to cite

APA:

Meyer, L., Erich, F., Yoshiyasu, Y., Stamminger, M., Ando, N., & Domae, Y. (2024). PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DOF Object Pose Dataset Generation. In IEEE/RSJ (Eds.), Proceedings of the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024). Abu Dhabi, AE.

MLA:

Meyer, Lukas, et al. "PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DOF Object Pose Dataset Generation." Proceedings of the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024), Abu Dhabi Ed. IEEE/RSJ, 2024.

BibTeX: Download