Addressing data scarcity in nanomaterial segmentation networks with differentiable rendering and generative modeling

Possart D, Mill L, Vollnhals F, Hildebrand T, Suter P, Hoffmann M, Utz J, Augsburger D, Thies M, Gu M, Wagner F, Sarau G, Christiansen S, Breininger K (2025)

Publication Type: Journal article

Publication year: 2025

Journal

npj Computational Materials Nature Research

Book Volume: 11

Article Number: 197

Journal Issue: 1

DOI: 10.1038/s41524-025-01702-6

Abstract

Nanomaterials’ properties, influenced by size, shape, and surface characteristics, are crucial for their technological, biological, and environmental applications. Accurate quantification of these materials is essential for advancing research. Deep learning segmentation networks offer precise, automated analysis, but their effectiveness depends on representative annotated datasets, which are difficult to obtain due to the high cost and manual effort required for imaging and annotation. To address this, we present DiffRenderGAN, a generative model that produces annotated synthetic data by integrating a differentiable renderer into a Generative Adversarial Network (GAN) framework. DiffRenderGAN optimizes rendering parameters to produce realistic, annotated images from non-annotated real microscopy images, reducing manual effort and improving segmentation performance compared to existing methods. Tested on ion and electron microscopy datasets, including titanium dioxide (TiO2), silicon dioxide (SiO2), and silver nanowires (AgNW), DiffRenderGAN bridges the gap between synthetic and real data, advancing the quantification and understanding of complex nanomaterial systems.

Authors with CRIS profile

Leonid Mill Lehrstuhl für Informatik 5 (Mustererkennung) Mathis Hoffmann Lehrstuhl für Informatik 5 (Mustererkennung) Jonas Utz Department Artificial Intelligence in Biomedical Engineering (AIBE) Mareike Thies Lehrstuhl für Informatik 5 (Mustererkennung) Mingxuan Gu Lehrstuhl für Informatik 5 (Mustererkennung) Fabian Wagner Lehrstuhl für Informatik 5 (Mustererkennung) Katharina Breininger Department Artificial Intelligence in Biomedical Engineering (AIBE)

Involved external institutions

Fraunhofer-Institut für Keramische Technologien und Systeme (IKTS) DE Germany (DE) Innovations-Institut für Nanotechnologie und korrelative Mikroskopie e.V. (INAM) / Institute for Nanotechnology and Correlative Microscopy DE Germany (DE) Lucid Concepts AG CH Switzerland (CH)

How to cite

APA:

Possart, D., Mill, L., Vollnhals, F., Hildebrand, T., Suter, P., Hoffmann, M.,... Breininger, K. (2025). Addressing data scarcity in nanomaterial segmentation networks with differentiable rendering and generative modeling. npj Computational Materials, 11(1). https://doi.org/10.1038/s41524-025-01702-6

MLA:

Possart, Dennis, et al. "Addressing data scarcity in nanomaterial segmentation networks with differentiable rendering and generative modeling." npj Computational Materials 11.1 (2025).

BibTeX: Download