An Approach to Progress Monitoring of Industrial Manual Processes Based on Camera Recordings and Object Interactions - Mühlbauer, Matthias; Kutzner, Korbinian; Sommer, Alexander; Würschinger, Hubert; Hanenkamp, Nico (Procedia CIRP, 2022)
Despite digitalization and automation, manual processes in factory operations still play a significant role. With the increasing complexity of tasks, it is advisable to support the employee during execution. Technological progress regarding the analysis of image data facilitates the use of camera-based systems for assistance functions. A core element for these assistance systems is the recognition of state or actions to achieve context-sensitive support. However, existing solutions are often developed for specific problems and are difficult to transfer to other processes. This is especially the case for manual processes with more and different sub-process steps such as setup and maintenance operations. High development effort even in case of minor process changes is the result. Therefore, the objective of this work is to develop a generic approach for progress monitoring of manual processes. The proposed approach is based on the understanding that manual processes can be considered as the manipulation of objects to achieve a defined state of their characteristic properties. Consequently, manual processes can be described by their features, like the positions or movements of objects or their relative position to each other. To realize a configurable approach a two-step procedure is proposed. The basis is the flexible recognition and localization of objects in camera recordings by machine learning methods. Subsequently, the information obtained can be utilized by a configurable logic to verify defined states. To develop the detailed concept, firstly, a method for analysis of complex manual processes in factory operation is conducted. The aim is to determine and categorize the objects relevant for optical recognition and state verification. Secondly, different machine learning approaches for object recognition and object localization are compared to evaluate their capability in capturing the feature states. The possible outputs of the machine learning approaches are used to create control elements for progress verification. Examples are the position of objects in relation to each other or the speed of movement. In the next step, a procedure for the process-specific selection of the control elements is developed. Finally, the approach is applied and discussed. A setup process of a turning machine and a deburring process serve as representative use cases.
Created from Fastlane, Scopus look-up
Keywords: assistance system; machine learning; machine vision; Manual process; progress monitoring; state validation

Audio Images: Analyzing impingement sounds of an air jet to develop a new tool wear measurement system - Würschinger, Hubert; Hanenkamp, Nico (Measurement, 2025)
Cutting tools are essential in machining processes, where knowing the condition of the tool is crucial to determining the optimal time for replacement. Various methods exist to assess tool condition, yet they are rarely applied due to challenges users face with these systems. This article provides background on a new approach for assessing and measuring tool wear using the impingement sound of an air jet. By scanning the surface with an air jet, an Audio Image is generated, enabling the reconstruction of surface details and the analysis of height variations, edges, and wear characteristics. Changes in the sound spectrum provide insights into surface variations, height differences, edge detection, and wear patterns. These findings open new possibilities for developing a Tool Condition Monitoring (TCM) system capable of reconstructing wear in three dimensions, offering more detailed and cost-effective analysis compared to existing methods.
CRIS-Team Scopus Importer:2025-02-17

Automated Data Labeling and Anomaly Detection Using Airborne Sound Analysis - Mühlbauer, Matthias; Würschinger, Hubert; Polzer, Dominik; Ju, Shu; Hanenkamp, Nico (Procedia CIRP, 2020)
Anomaly detection is facing the challenge of generating a maximum of information with a limited number of sensors in production machines and minimal effort of data analysis. Machine operators are often able to perceive changes acoustically and based on experience. In order to imitate this intelligent human capability, a systematic methodology has been developed in this work. Firstly, an introduction to this topic will be given and the acoustic sensor set up as well as the data preprocessing will be described. Secondly, an approach for automatic data labeling as an input for the intelligent anomaly detection will be presented. Using a model-based approach, anomaly detection is performed in the next step. This allows not only the detection of gradual and sudden process changes but also systematic problem solving. Finally, the anomaly detection will be validated for cutting processes.<br
Keywords: Anomaly Detection; Automated Data Labeling; Process Monitoring; Airborne Sound; Cyclic Production Processes; Cutting Processes

Demonstration of a new approach for measuring tools with the impingement sound of an air jet using machine learning - Würschinger, Hubert; Gross, Daniel; Stadler, Manuel; Mühlbauer, Matthias; Hanenkamp, Nico (MM Science Journal, 2021)
Monitoring the tool condition of machining processes is important but challenging. Several automated tool condition monitoring solutions are available, but often not used due to existing restrictions or disadvantages. A new approach can be the detection and measurement of tool conditions analyzing the sound of an air jet impingement on tools. Due to the availability of compressed air as a working and cleaning medium for many processes, this approach can be used for various condition monitoring and measuring tasks. In this paper the procedure and its functionality are first presented on simple shapes and then tested on the tool wear of inserts.<br
Keywords: Tool Condition Monitoring; Condition Monitoring; Impingement Sound; Jet Impingement; Air Jet; Flow Noise; Airborne Sound; Machine Learning

Deviation Detection in Production Processes based on Video Data using Unsupervised Machine Learning Approaches - Mühlbauer, Matthias; Epp, Henrik; Würschinger, Hubert; Hanenkamp, Nico (Procedia CIRP, 2022)
The detection of deviations within production processes is essential to ensure high productivity or avoid potential damage. Various approaches are available for this purpose. In the field of video surveillance, unsupervised machine learning methods have made significant progress in detecting deviations. In this paper, the transferability of these generic approaches to production processes is investigated. At first, an evaluation basis is created. Therefore, the variety of deviations, which can occur in an automated production process, is structured and covered as far as possible in video benchmark data sets. Subsequently, existing unsupervised approaches are selected, adapted and tested on the created data sets. In conclusion, the results show that the two chosen unsupervised autoencoder architectures can be partially used for generic deviation detection in the production domain. The main challenges identified are the large variety of different tasks and deviations in production processes. However, for further investigations, the development of even more detailed benchmark sets is essential.
CRIS-Team Scopus Importer:2022-12-09
Keywords: anomaly detection; autoencoder; computer vision; deviation detection; Production process; unsupervised learning; video data

Energy Profile Prediction of Milling Processes Using Machine Learning Techniques - Mühlbauer, Matthias; Würschinger, Hubert; Polzer, Dominik; Hanenkamp, Nico (Machine Learning for Cyber Physical Systems, 2020)
The prediction of the power consumption increases the transparency and the understanding of a cutting process, this delivers various potentials. Beside the planning and optimization of manufacturing processes, there are application areas in different kinds of deviation detection and condition monitoring. Due to the complicated stochastic processes during the cutting processes, analytical approaches quickly reach their limits. Since the 1980s, approaches for predicting the time or energy consumption use empirical models. Nevertheless, most of the existing models regard only static snapshots and are not able to picture the dynamic load fluctuations during the entire milling process. This paper describes a data-driven way for a more detailed prediction of the power consumption for a milling process using Machine Learning techniques. To increase the accuracy we used separate models and machine learning algorithms for different operations of the milling machine to predict the required time and energy. The merger of the individual models allows finally the accurate forecast of the load profile of the milling process for a specific machine tool. The following method introduces the whole pipeline from the data acquisition, over the preprocessing and the model building to the validation.<br
Keywords: energy prediction; time prediction; power load prediction; milling process; machine learning; regression

Implementation and potentials of a machine vision system in a series production using deep learning and low-cost hardware - Würschinger, Hubert; Mühlbauer, Matthias; Winter, Michael; Engelbrecht, Michael; Hanenkamp, Nico (Procedia CIRP, 2020)
For manufacturing processes there is a need to ensure an efficient production and to fulfill the increasing quality requirements. To handle these challenges, Machine Vision Systems can be used for process monitoring and quality control. In this paper the implementation and thereby the potentials of such a system in a series production using Transfer Learning with low cost hardware is introduced. The necessary steps, from the hardware implementation, the data acquisition, the preprocessing over the optimization and the application are depicted. Finally we show that the proposed solution can fulfill defined requirements and can compete with a professional Machine Vision System.<br
Keywords: Machine Vision; Manufacturing; Series Production; Image ClassificationTransfer Learning; Low Cost; Data Acquisition

Maschinelle Lernverfahren für die Praxis - Praxisnahes Qualifizierungskonzept für Industriesegmente wie Automotive und Maschinenbau - Mühlbauer, Matthias; Würschinger, Hubert; Hanenkamp, Nico (VDI-Z Integrierte Produktion, 2019)

Maschinelle Lernverfahren zur Digitalisierung manueller Ablese- und Messvorgänge - Mühlbauer, Matthias; Würschinger, Hubert; Hanenkamp, Nico; Funtikov, Svyatoslav (Industrie 4.0 Management : Gegenwart und Zukunft industrieller Geschäftsprozesse, 2023)
Im Fabrikbetrieb kommen häufig Messmittel zum Einsatz, bei denen noch keine automatische Abspeicherung oder Weiterverarbeitung des gemessenen Werts erfolgt. Mitarbeiter müssen in diesem Fall den Messwert manuell er-fassen und weiterverarbeiten. Folgend wird ein Ansatz zur optischen Erfassung und Digitalisierung von Messwerten mithilfe Maschineller Lernverfahren vorgestellt. Dies soll zur Entlastung der Mitarbeiter beitragen, Ablesefehler reduzieren sowie eine automatisierte Dokumentation ermöglichen.<br
Keywords: Digitalisierung; Bildverarbeitung; Maschinelles Lernen; Objekterkennung

SUPPORTING MANUAL CONTOUR AND SURFACE PROCESSING WITH THE HELP OF AUGMENTED REALITY - Mühlbauer, Matthias; Auer, Alexander; Würschinger, Hubert; Hanenkamp, Nico (MM Science Journal, 2023)
When manually processing complex contours or surfaces, there is a risk of incomplete or unnecessary multiple processing. The support of such processes with an augmented reality (AR) application will be presented in this work using the example of manual deburring. Firstly, the positions of the component and tool are determined based on camera recordings. For this purpose, approaches for 6 Degrees of Freedom (DoF) pose estimation will be utilized. Secondly, the correct and complete processing is concluded by further analyzing the extracted data. For example, the distance of the objects to each other. In the third step, the processing result is visualized to aid the worker. Finally, the system is validated through a study. The results show that with the help of image targets a robust recognition of the objects at a normal execution speed and thus process support can be realized.
CRIS-Team Scopus Importer:2023-12-22
Keywords: assistance system; Augmented reality; contour and surface processing; deburring; manual processes; object detection

Transfer Learning für visuelle Kontrollaufgaben - Würschinger, Hubert; Mühlbauer, Matthias; Hanenkamp, Nico (wt Werkstattstechnik - Online, 2020)
In der industriellen Praxis wird eine Vielzahl von Prozessund Qualitätskontrollaufgaben visuell von Mitarbeitern oder mithilfe von Kamerasystemen durchgeführt. Durch den Einsatz Künstlicher Intelligenz (KI) lässt sich die Programmierung und damit die Implementierung von Kamerasystemen effizienter gestalten. Im Bereich der Bildanalyse können dabei vortrainierte Künstliche Neuronale Netze verwendet werden. Das Anwenden dieser Netze auf neue Aufgaben wird dabei Transfer Learning genannt. <br
Keywords: Künstliche Intelligenz; Produktion; Qualitätssicherung

Vorhersage der Prozessstabilität mit Maschinellem Lernen - Potenziale Künstlicher Intelligenz zur frühzeitigen Erkennung von Abweichungen bei der pharmazeutischen Abfüllung - Mühlbauer, Matthias; Würschinger, Hubert; Hanenkamp, Nico; Schmehling, Moritz; Krause, Björn (Industrie 4.0 Management : Gegenwart und Zukunft industrieller Geschäftsprozesse, 2020)