Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots - Hofmann, Christian; May, Christopher; Ziegler, Patrick; Ghotbiravandi, Iliya; Franke, Jörg; Reitelshöfer, Sebastian (Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2025)
Large Language Models (LLMs) and Vision Language Models (VLMs) enable robots to perform complex tasks. However, many of today’s mobile robots cannot carry the computing hardware required to run these models on board. Furthermore, access via communication systems to external computers running these models is often impractical. Therefore, lightweight object detection models are often utilized to enable mobile robots to semantically perceive their environment. In addition, mobile robots are used in different environments, which also change regularly. Thus, an automated adaptation of object detectors would simplify the deployment of mobile robots. In this paper, we present a method for automated environment-specific individualization and adaptation of lightweight object detectors using LLMs and VLMs, which includes the automated identification of relevant object classes. We comprehensively evaluate our method and show its successful application in principle, while also pointing out shortcomings regarding semantic ambiguities and the application of VLMs for pseudo-labeling datasets with bounding box annotations.
CRIS-Team Scopus Importer:2025-04-11
Keywords: Large Language Model; Open Vocabulary Object Detection; Pseudo-Labeling; Vision Language Model