Henningson JO, Veltrup R, Semmler M, Döllinger M, Stamminger M (2025)
Publication Type: Conference contribution
Publication year: 2025
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 15187 LNCS
Pages Range: 102-112
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Marrakesh, MAR
ISBN: 9783031732805
DOI: 10.1007/978-3-031-73281-2_10
There exist various approaches for the 3D reconstruction of dynamic scenes. In medicine, particularly in endoscopy, single-shot structured light systems are frequently explored, as they allow for the reconstruction of dynamic, feature-less surfaces. Design and manufacturing of structured light endoscopes, however, implies high initial costs that significantly hinder the availability and development of these systems. To streamline this process, simulation systems are necessary that allow researchers to not only model the intricacies of medical domains, but also of structured light systems themselves. To address this, we propose Fireflies, a differentiable framework for the physically-based simulation and domain randomization of structured light endoscopy. Based on the differentiable Mitsuba renderer, Fireflies facilitates and simplifies the development of domain-specific algorithms for endoscopic procedures. In this paper, we demonstrate the effectiveness of our framework by jointly optimizing domain-specific laser-based projection pattern for Structured Light Endoscopy, and generating large-scale synthetic training data for efficient supervised learning without manual labeling. We show that a) an optimized projection pattern can increase the reconstructability of a target domain and b) the synthetic data generated by Fireflies lowers the labeling effort required for endoscopic machine learning tasks. The source code is available at: https://github.com/Henningson/Fireflies
APA:
Henningson, J.-O., Veltrup, R., Semmler, M., Döllinger, M., & Stamminger, M. (2025). Fireflies: Photorealistic Simulation and Optimization of Structured Light Endoscopy. In Virginia Fernandez, Jelmer M. Wolterink, David Wiesner, Samuel Remedios, Lianrui Zuo, Adrià Casamitjana (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 102-112). Marrakesh, MAR: Springer Science and Business Media Deutschland GmbH.
MLA:
Henningson, Jann-Ole, et al. "Fireflies: Photorealistic Simulation and Optimization of Structured Light Endoscopy." Proceedings of the 9th International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2024, held in conjunction with the 27th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2024, Marrakesh, MAR Ed. Virginia Fernandez, Jelmer M. Wolterink, David Wiesner, Samuel Remedios, Lianrui Zuo, Adrià Casamitjana, Springer Science and Business Media Deutschland GmbH, 2025. 102-112.
BibTeX: Download