Hofmann C, May C, Ziegler P, Ghotbiravandi I, Franke J, Reitelshöfer S (2025)
Publication Type: Conference contribution
Publication year: 2025
Publisher: Science and Technology Publications, Lda
Book Volume: 2
Pages Range: 851-862
Conference Proceedings Title: Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
Event location: Porto, PRT
Large Language Models (LLMs) and Vision Language Models (VLMs) enable robots to perform complex tasks. However, many of today’s mobile robots cannot carry the computing hardware required to run these models on board. Furthermore, access via communication systems to external computers running these models is often impractical. Therefore, lightweight object detection models are often utilized to enable mobile robots to semantically perceive their environment. In addition, mobile robots are used in different environments, which also change regularly. Thus, an automated adaptation of object detectors would simplify the deployment of mobile robots. In this paper, we present a method for automated environment-specific individualization and adaptation of lightweight object detectors using LLMs and VLMs, which includes the automated identification of relevant object classes. We comprehensively evaluate our method and show its successful application in principle, while also pointing out shortcomings regarding semantic ambiguities and the application of VLMs for pseudo-labeling datasets with bounding box annotations.
APA:
Hofmann, C., May, C., Ziegler, P., Ghotbiravandi, I., Franke, J., & Reitelshöfer, S. (2025). Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots. In Thomas Bashford-Rogers, Daniel Meneveaux, Mehdi Ammi, Mounia Ziat, Stefan Jänicke, Helen Purchase, Petia Radeva, Antonino Furnari, Kadi Bouatouch, A. Augusto Sousa (Eds.), Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (pp. 851-862). Porto, PRT: Science and Technology Publications, Lda.
MLA:
Hofmann, Christian, et al. "Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots." Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2025, Porto, PRT Ed. Thomas Bashford-Rogers, Daniel Meneveaux, Mehdi Ammi, Mounia Ziat, Stefan Jänicke, Helen Purchase, Petia Radeva, Antonino Furnari, Kadi Bouatouch, A. Augusto Sousa, Science and Technology Publications, Lda, 2025. 851-862.
BibTeX: Download