One Prompt, Many Rooms: A Force-Directed Approach to 3D Scene Generation for Robotics Simulation

May C, Hetzner P, Kalenberg M, Sajith Nambiar A, Franke J, Reitelshöfer S (2026)

Publication Type: Conference contribution

Publication year: 2026

Original Authors: Christopher May, Peter Hetzner, Matthias Kalenberg, Ashwin Sajith Nambiar, Jörg Franke, Sebastian Reitelshöfer

Pages Range: 9274-9281

Event location: Wien AT

DOI: 10.1109/ICRA57385.2026.11696816

Abstract

Training generalizable robotic agents requires large datasets of diverse, physically consistent 3D scenes, yet their generation remains a critical bottleneck. Current text-to-scene methods are inefficient for this task; generating diverse layouts requires repeated, expensive sampling that fails to maintain the semantic consistency required for robust policy learning. In this paper, we address this with our "one-plan-to-many-layouts" method. A Large Language Model (LLM) generates a single declarative plan, which a force-directed physics simulation then realizes into multiple layouts that share semantics but differ in geometry. We validate our method by transfer to photorealistic 3D reconstructions of real environments (Replica) within simulation, where a navigation agent trained on our scenes attains a Success Rate of 0.84. These results establish our pipeline as a scalable method for producing the controlled, diverse data required for embodied AI training.

Authors with CRIS profile

Christopher May Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik (FAPS) Matthias Kalenberg Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik (FAPS) Ashwin Sajith Nambiar Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik (FAPS) Jörg Franke Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik (FAPS) Sebastian Reitelshöfer Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik (FAPS)

How to cite

APA:

May, C., Hetzner, P., Kalenberg, M., Sajith Nambiar, A., Franke, J., & Reitelshöfer, S. (2026). One Prompt, Many Rooms: A Force-Directed Approach to 3D Scene Generation for Robotics Simulation. In Proceedings of the 2026 IEEE International Conference on Robotics and Automation (ICRA) (pp. 9274-9281). Wien, AT.

MLA:

May, Christopher, et al. "One Prompt, Many Rooms: A Force-Directed Approach to 3D Scene Generation for Robotics Simulation." Proceedings of the 2026 IEEE International Conference on Robotics and Automation (ICRA), Wien 2026. 9274-9281.

BibTeX: Download