Aerial Grasping and Transport Using an Unmanned Aircraft (UA) Equipped with an Industrial Suction Gripper - Lieret, Markus; Kreis, Benedikt; Hofmann, Christian; Zwingel, Maximilian; Franke, Jörg (Annals of Scientific Society for Assembly, Handling and Industrial Robotics 2021, 2022)
Due to the availability of highly efficient unmanned aircraft (UA) and the advancement of the necessary technologies, the use of UA for object manipulation and cargo transport is becoming a more and more relevant research area. A reliable identification and localization of cargo and interaction objects as well as maintaining the required flight precision are essential to guarantee a successful object handling. Within this paper we demonstrate the successful application of an autonomous UA equipped with a lightweight suction gripper for object interaction. We discuss the approach used for precise localization as well as the identification and pose estimation of individual gripping objects. Concluding, the overall system performance is evaluated within an industrial-oriented use case.<br
Keywords: Autonomous unmanned aircrafts; Flying robots; Aerial grasping; Object detection; Localization

A Multimodal A∗ Algorithm to Solve the Two-Dimensional Optimization Problem of Accompanying a Person for an Intelligent Wheelchair - Kalenberg, Matthias; Lieret, Markus; Hofmann, Christian; Franke, Jörg (45th Annual International Conference of the IEEE Engineering in Medicine and Biology Conference, EMBC 2023, 2023)

An overhead docking and charging station for autonomous unmanned aircraft - Lieret, Markus; Wurmer, Florens; Hofmann, Christian; Franke, Jörg (IEEE International Conference on Automation Science and Engineering, 2021)
Autonomous unmanned aircraft (UA) are currently gaining importance in various areas of application. In addition to their use in the field of surveying, inspection or fluid application, the focus is also on the transport of goods and the monitoring of factory premises. However, decisive disadvantage of commonly used UA is their short flight time, which typically ranges between approx. 10 and 45 minutes, depending on the design and payload. For the automated use of UA, it is not practical to manually change the battery between individual flights but automated solutions are required. The automated charging methods presented so far have in common that the UA approaches the landing platform vertically or at an angle from above and stands on the landing platform after completion. In this paper, we present a novel solution for the overhead charging of UA. Thereby the UA docks vertically to the charging station from below and is locked in the station. This saves floor space and allows the use of sensors attached to UA for information acquisition and environmental monitoring while charging. In addition to the charging station and the associated docking device attached to the UA, a camera-based method for localizing the charging station and for precise docking is presented. Finally, the docking and undocking process is evaluated in a series of experiments.
CRIS-Team Scopus Importer:2021-10-29

Automated Exploration, Capture and Photogrammetric Reconstruction of Interiors Using An Autonomous Unmanned Aircraft - Lieret, Markus; Kogan, Vladyslav; Hofmann, Christian; Franke, Jörg (2021 IEEE International Conference on Mechatronics and Automation, ICMA 2021, 2021)
Digital environment models are gaining importance in various fields of application. High-resolution digital models of the current environment structure can facilitate planning manufacturing plants or allow monitoring construction processes precisely. While systems for the fully automated capturing of terrain structures are already in use in outdoor areas, the digitisation process in indoor areas is usually carried out manually. Since global navigation satellite systems are not available in complex indoor areas, other solutions for robust and precise autonomous localization and navigation have to be used and validated. Furthermore, it is more intricate to ensure that all required areas and structures are taken into account with sufficient accuracy during the capture process. Within this paper, we present a fully automated system for data acquisition and digitisation of complex indoor environments. Using an autonomous unmanned aircraft equipped with a suitable sensor system, intelligent algorithms for navigation, exploration and photogrammetric reconstruction, a previously unknown environment is autonomously explored and a three-dimensional model is reconstructed. Extending the related research, we demonstrate that also within indoor environments unmanned aircraft are applicable for the autonomous acquisition of images for photogrammetric reconstruction. Within the evaluation, we validate the functionality of the overall system and the accuracy of the resulting point clouds of the environment.
CRIS-Team Scopus Importer:2021-10-01
Keywords: autonomous unmanned aircraft; environment perception; photogrammetry; robot navigation

Automated Individualization of Object Detectors for the Semantic Environment Perception of Mobile Robots - Hofmann, Christian; May, Christopher; Ziegler, Patrick; Ghotbiravandi, Iliya; Franke, Jörg; Reitelshöfer, Sebastian (Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2025)
Large Language Models (LLMs) and Vision Language Models (VLMs) enable robots to perform complex tasks. However, many of today’s mobile robots cannot carry the computing hardware required to run these models on board. Furthermore, access via communication systems to external computers running these models is often impractical. Therefore, lightweight object detection models are often utilized to enable mobile robots to semantically perceive their environment. In addition, mobile robots are used in different environments, which also change regularly. Thus, an automated adaptation of object detectors would simplify the deployment of mobile robots. In this paper, we present a method for automated environment-specific individualization and adaptation of lightweight object detectors using LLMs and VLMs, which includes the automated identification of relevant object classes. We comprehensively evaluate our method and show its successful application in principle, while also pointing out shortcomings regarding semantic ambiguities and the application of VLMs for pseudo-labeling datasets with bounding box annotations.
CRIS-Team Scopus Importer:2025-04-11
Keywords: Large Language Model; Open Vocabulary Object Detection; Pseudo-Labeling; Vision Language Model

Automatische Identifikation mittels RFID - Rückverfolgbarkeit von Fertigungshilfsmitteln in der Produktion von Hochleistungsröntgenstrahlern - Herbert, Meike; Stirm, Julian; Hofmann, Christian; Fürst, Jens; Franke, Jörg (Zeitschrift für wirtschaftlichen Fabrikbetrieb, 2022)
Modern technologies are a key enabler for the automatic identification of objects in production to ensure quality-related traceability of components or manufacturing equipment. This article presents a possible solution for the traceability of equipment, tools and measuring devices by applying a RFID system in the production of high-performance X-ray tubes and discusses the results of durability analysis.
CRIS-Team Scopus Importer:2023-03-07
Keywords: Identification Systems; RFID; Lifetime; Traceability; X-Ray

Efficient Semantic Mapping in Dynamic Environments - Hofmann, Christian; Fichtner, Mathias; Lieret, Markus; Franke, Jörg (PROCEEDINGS OF THE 17TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS (VISAPP), VOL 4, 2022)
CRIS-Team WoS Importer:2022-05-06

Entropy-Based Intention Change Detection with a Multi-Hypotheses Filter - Particke, Florian; Hofmann, Christian; Hiller, Markus; Feist, Christian; Thielecke, Jörn (2018 21st International Conference on Information Fusion (FUSION), 2018)
In the future, pedestrians and fully automated vehicles have to operate in an environment they share. To minimize the risk for pedestrians, it is very important to predict precisely their future movement. One important information source is the intention of the pedestrian. For the integration of the intention information, a Multi-Hypotheses filter is used, where different hypotheses for the intention of the pedestrian are considered. An intention change detector based on the Multi-Hypotheses filter utilizing an entropy-based confidence score is developed. With this contribution, critical real-world situations like a pedestrian crossing the street instead of following the sidewalk are tackled. The evaluation of the intention change detector is performed in simulation and for real-world data. Firstly, the proposed approach is evaluated using simulated trajectory data, where trajectories with intention changes are generated by a self-made trajectory generator (open source). Secondly, the course of the confidence score is evaluated for a real-world scenario, where the detection of the pedestrians is performed by the combination of a deep learning network (Tiny YOLO) and background subtraction. It is shown that the mean distance into the road from the sidewalk edge at the detection of the intention change is below 1.5 m, even in the case of high sensor noise. For lower sensor noise level, the intention change of the pedestrian is even detected before entering the street. Key contributions are the proposal of the Multi-Hypotheses filter, the derivation of the confidence score, the proposal of the intention detector based on the confidence score and the detection of the pedestrians and other obstacles by the fusion of background subtraction and a deep learning networ
Keywords: Kalman filters; Detectors; Position measurement; Entropy; Trajectory; Machine learning; Noise measurement

Environment representation and path planning for unmanned aircraft in industrial indoor applications - Lieret, Markus; Kalenberg, Matthias; Hofmann, Christian; Franke, Jörg (FAIM 2021, 2021)
CRIS-Team WoS Importer:2022-10-28

Human Comfort Factors in People Navigation: Literature Review, Taxonomy and Framework - Kalenberg, Matthias; Hofmann, Christian; Martin, Sina; Franke, Jörg (Robotics, Computer Vision and Intelligent Systems. 4th International Conference, ROBOVIS 2024, Rome, Italy, February 25–27, 2024, Proceedings, 2024)
Due to demographic shifts and improvements in medical care, person navigation systems (PNS) for people with disabilities are becoming increasingly important. So far, PNS have received less attention than mobile robots. However, the work on mobile robots cannot always be transferred to PNS because there are important differences in navigating people. In this paper, we address these differences by providing a comprehensive literature review on human comfort factors in people navigation, presenting a unified taxonomy for PNS and proposing a framework for integrating these factors into a navigation stack. Based on the results, we extract the key differences and human comfort factors that have been addressed in current literature. Furthermore, the literature review shows that there is no unified taxonomy in this field. To address this, we introduce the term people navigation and a taxonomy to categorize existing systems. Finally, we summarize the human comfort factors that have been considered so far and provide an outlook on their implementation. Our survey serves as a foundation for a comprehensive research in people navigation and identifies open challenges.<br
Keywords: People navigation; Person navigation system; Human comfort factors

Human Detection and Gesture Recognition for the Navigation of Unmanned Aircraft - Lieret, Markus; Hübner, Maximilian; Hofmann, Christian; Franke, Jörg (PROCEEDINGS OF THE 17TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS (VISAPP), VOL 4, 2022)
CRIS-Team WoS Importer:2022-05-06

Learning Topometric Semantic Maps from Occupancy Grids - Hiller, Markus; Qiu, Chen; Particke, Florian; Hofmann, Christian; Thielecke, Jörn (2019 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2019)
CRIS-Team WoS Importer:2020-08-21

Neural Network Aided Potential Field Approach for Pedestrian Prediction - Particke, Florian; Zhou, Jiaren; Hiller, Markus; Hofmann, Christian; Thielecke, Jörn (2019 Symposium on Sensor Data Fusion: Trends, Solutions, Applications, SDF 2019, 2019)
Autonomous driving is one of the key challenges in recent time. As pedestrians are the most vulnerable traffic participants, collisions with pedestrians have to be avoided under all circumstances. Hence, prediction of pedestrian trajectories is of high interest for automated vehicles. For this purpose, a plethora of algorithms has been proposed to model the pedestrian in the last decades, reaching from simple kinematic models to advanced microscopic models. In addition, the machine learning community started to learn the behavior of pedestrians and showed major improvements in complex scenarios or unexpected situations. However, as most of the machine learning algorithms are treated as black boxes, the safeguarding of the software is one key challenge which has to be solved. This contribution proposes to combine classic modeling of pedestrians with machine learning algorithms by learning the model errors between a simple physical model and real data. In particular, it is proposed to combine a physical model based on potential fields with a neural network to predict the future behavior of pedestrians. It is shown that the combined approach outperforms the physical model in learnable areas, whereas the physical model without the neural network is more robust in areas where almost no training data is available. In addition, different structures of neural networks are analyzed.
CRIS-Team Scopus Importer:2019-12-31
Keywords: Autonomous driving; machine learning; neural network; pedestrian; prediction

Object detection, classification and localization by infrastructural stereo cameras - Hofmann, Christian; Particke, Florian; Hiller, Markus; Thielecke, Jörn (VISIGRAPP 2019 - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2019)
In the future, autonomously driving vehicles have to navigate in challenging environments. In some situations, their perception capabilities are not able to generate a reliable overview of the environment, by reason of occlusions. In this contribution, an infrastructural stereo camera system for environment perception is proposed. Similar existing systems only detect moving objects by background subtraction algorithms and monocular cameras. In contrast, the proposed approach fuses three different algorithms for object detection and classification and uses stereo vision for object localization. The algorithmic concept is composed of a background subtraction algorithm based on Gaussian Mixture Models, the convolutional neural network”You only look once” as well as a novel algorithm for detecting salient objects in depth maps. The combination of these complementary object detection principles allows the reliable detection of dynamic as well as static objects. An algorithm for fusing the results of the three object detection methods based on bounding boxes is introduced. The proposed fusion algorithm for bounding boxes improves the detection results and provides an information fusion. We evaluate the proposed concept on real word data. The object detection, classification and localization in the real world scenario is investigated and discussed.
CRIS-Team Scopus Importer:2019-07-09
Keywords: Autonomous Driving; Deep Learning; Infrastructural Cameras; Object Detection; Robotics; Stereo Vision

Stochastic Modeling of Camera Errors for Stereo Image Processing - Particke, Florian; Hofmann, Christian; Hiller, Markus; Patino-Studencki, Lucila; Thielecke, Jörn (ICPRAI 2018, International Conference on Pattern Recognition and Artificial Intelligence, 2018)

Systematic Analysis of Direct Sparse Odometry - Particke, Florian; Kalisz, Adam; Hofmann, Christian; Hiller, Markus; Bey, Henrik; Thielecke, Jörn (2018 Digital Image Computing: Techniques and Applications (DICTA), 2018)
In the field of robotics and autonomous driving, the camera as a sensor gets more and more important, as the camera is cheap and robust against environmental influences. One challenging task is the localization of the robot on an unknown map. This leads to the so-called Simultaneous Localization and Mapping (SLAM) problem. For the Visual SLAM problem, a plethora of algorithms was proposed in the last years, but the algorithms were rarely evaluated regarding the robustness of the approaches. This contribution motivates the systematic analysis of Visual SLAMs in simulation by using heterogeneous environments in Blender. For this purpose, three different environments are used for evaluation ranging from very low detailed to high detailed worlds. In this contribution, the Direct Sparse Odometry (DSO) is evaluated as an exemplary Visual SLAM. It is shown that the DSO is very sensitive to rotations of the camera. In addition, it is presented that if the scene does not provide sufficient clues about the depth, an estimation of the trajectory is not possible. The results are complemented by real-world experiments.
Keywords: Cameras; Simultaneous localization and mapping; Three-dimensional displays; Urban areas; Calibration

Systematic comparison of ORB-SLAM2 and LDSO based on varying simulated environmental factors - Kalisz, Adam; Ling, Tong; Particke, Florian; Hofmann, Christian; Thielecke, Jörn (VISIGRAPP 2020 - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2020)
Although the number of outstanding but highly complex Visual SLAM systems which are published as open source has increased in recent years, they often lack a systematic evaluation of their weaknesses and failure cases. This work systematically discusses the key differences of two state-of-the-art Visual SLAM algorithms, the indirect ORB-SLAM2 and the direct LDSO, by extensive experiments in varying environments. The evaluation is principally focused to the trajectory accuracy and robustness of the algorithms in specific situations. However, details about individual components used for the estimation of trajectories in both systems are presented. In order to investigate crucial aspects, a custom dataset was created in a 3D modeling software, Blender, to acquire the data for all experiments. The experimental results demonstrate the strengths and weaknesses of the systems. In particular, this research contributes insight into: 1. The influence of moving objects in a usually static scene. 2. How both systems react on periodicly changing scene lighting, both local and global. 3. The role of initialization on the resistance to dynamic changes in the scene.
CRIS-Team Scopus Importer:2020-04-28
Keywords: And; Comparison; DSO; LDSO; Localization; Mapping; ORB; Simultaneous; SLAM; Visual

Towards Adaptive Environment Perception and Understanding for Autonomous Mobile Robots - Hofmann, Christian; Taliercio, Franca; Walter, Jonas; Franke, Jörg; Reitelshöfer, Sebastian (2023 IEEE Symposium Sensor Data Fusion and International Conference on Multisensor Fusion and Integration (SDF-MFI), 2023)
Autonomous mobile robots (AMRs) are increasingly used in various applications like intralogistics, hospitality and agriculture. Though, their software and abilities are still mainly designed and implemented for specific environments and tasks, impeding an even more widespread deployment of AMRs. Their environment perception and understanding lacks automated adaption to specific operating environments and conditions.In this paper, we propose a software architecture that allows the automated adaption of AMRs’ environment perception and understanding to different operating environments. Further, we provide a concise review of state-of-the-art approaches and technologies needed. We present an exemplary implementation and evaluate the proposed architecture. On this basis, advantages and open research questions of the architecture are derived.<br
Keywords: Software architecture; Semantics; Data integration; Computer architecture; Robot sensing systems; Software; Agriculture; Mobile Robot; Adaptive Perception; Semantic Environment Representation

World Modeling for Mobile Platforms Using a Contextual Object-Based Representation of the Environment - Hiller, Markus; Particke, Florian; Hofmann, Christian; Bey, Henrik; Thielecke, Jörn (2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), 2018)