Activity recognition in beach volleyball using a Deep Convolutional Neural Network - Kautz, Thomas; Groh, Benjamin; Hannink, Julius; Jensen, Ulf; Strubberg, Holger; Eskofier, Björn (Data Mining and Knowledge Discovery, 2017)
Many injuries in sports are caused by overuse. These injuries are a major cause for reduced performance of professional and non-professional beach volleyball players. Monitoring of player actions could help identifying and understanding risk factors and prevent such injuries. Currently, time-consuming video examination is the only option for detailed player monitoring in beach volleyball. The lack of a reliable automatic monitoring system impedes investigations about the risk factors of overuse injuries. In this work, we present an unobtrusive automatic monitoring system for beach volleyball based on wearable sensors. We investigate the possibilities of Deep Learning in this context by designing a Deep Convolutional Neural Network for sensor-based activity classification. The performance of this new approach is compared to five common classification algorithms. With our Deep Convolutional Neural Network, we achieve a classification accuracy of 83.2%, thereby outperforming the other classification algorithms by 16.0%. Our results show that detailed player monitoring in beach volleyball using wearable sensors is feasible. The substantial performance margin between established methods and our Deep Neural Network indicates that Deep Learning has the potential to extend the boundaries of sensor-based activity recognition.

A Robust Kalman Framework with Resampling and Optimal Smoothing - Kautz, Thomas; Eskofier, Björn (Sensors, 2015)
The Kalman filter (KF) is an extremely powerful and versatile tool for signal processing that has been applied extensively in various fields. We introduce a novel Kalman-based analysis procedure that encompasses robustness towards outliers, Kalman smoothing and real-time conversion from non-uniformly sampled inputs to a constant output rate. These features have been mostly treated independently, so that not all of their benefits could be exploited at the same time. Here, we present a coherent analysis procedure that combines the aforementioned features and their benefits. To facilitate utilization of the proposed methodology and to ensure optimal performance, we also introduce a procedure to calculate all necessary parameters. Thereby, we substantially expand the versatility of one of the most widely-used filtering approaches, taking full advantage of its most prevalent extensions. The applicability and superior performance of the proposed methods are demonstrated using simulated and real data. The possible areas of applications for the presented analysis procedure range from movement analysis over medical imaging, brain-computer interfaces to robot navigation or meteorological studies.
UnivIS-Import:2015-04-14:Pub.2015.tech.IMMD.IMMD5.arobus
Keywords: Kalman filter; fixed-lag smoothing; outlier detection; real-time filtering; non-uniform sampling; parameter estimation

Augmented Motion Models for Constrained Position Tracking with Kalman Filters - Kautz, Thomas; Groh, Benjamin; Eskofier, Björn (19th International Conference on Information Fusion (FUSION 2016), 2016)
Accurate position tracking is a crucial task in many applications ranging from car navigation over robot control to sports analysis. In order to improve the accuracy of position tracking, we introduce a novel method for constraining Kalman filters by incorporating prior knowledge in an augmented motion model. In contrast to previously reported methods, our approach does not require cumbersome tuning of additional filter parameters and causes less computational overhead. We demonstrate our method in the context of sports analysis in athletics. Using 34 data sets recorded during 400m and 800m runs, we compare our approach to unconstrained and pseudo-measurement filters.  The presented augmented motion model in conjunction with an Extended Kalman Filter (EKF) reduced the root mean square error of the filtered output by 60% compared to unconstrained filtering and by 50% compared to a pseudo-measurement EKF.

Automated Ski Velocity and Jump Length Determination in Ski Jumping Based on Unobtrusive and Wearable Sensors - Groh, Benjamin; Warschun, Frank; Deininger, Martin; Kautz, Thomas; Martindale, Christine; Eskofier, Björn (Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2017)

Ball impact localization on table tennis rackets using piezo-electric sensors - Blank, Peter; Kautz, Thomas; Eskofier, Björn (Proceedings of the 2016 ACM International Symposium on Wearable Computers, 2016)
In this paper, we present an approach for a ball impact localization on table tennis rackets using piezo-electric sensors. Three sensors were used as vibration sensors at the outer edge of the racket. We analyzed ball impact vibration appearances and measured time differences of arriving wave fronts for each sensor pair. A study was conducted comprising six different racket-rubber combinations. Firstly, we calculated a time difference distribution model based on training sets of impact data constructed by a grid of measuring points. Secondly, this model was used to estimate the impact position of test data sets using three different methods. Best performance for each racket-rubber combination was achieved with a linear regression based method yielding a RMSE in x-direction (transversal racket direction) of 22.1 mm and a RMSE of 19.8 mm in y-direction (longitudinal racket direction). The measuring system is small enough to be completely invisibly integrated into table tennis rackets. This contribution is the first attempt to address a ball impact localization system for table tennis rackets that has the potential to be used in real-world training exercises and competitions to assist table tennis players during matches and to support the athletes' training progress.
Keywords: piezo-electric sensors; vibration measurement; localization; table tennis

Classification and visualization of skateboard tricks using wearable sensors - Groh, Benjamin; Fleckenstein, Martin; Kautz, Thomas; Eskofier, Björn (Pervasive and Mobile Computing, 2017)
The application of wearables and customized signal processing methods offers new opportunities for motion analysis and visualization in skateboarding. In this work, we propose an automatic trick analysis and visualization application based on inertial-magnetic data. Skateboard tricks are detected and classified in real-time and visualized by means of an animated 3D-graphic. We achieved a trick detection recall of 96.4%, a classification accuracy of 89.1% (considering correctly performed tricks) and an error of the board orientation visualization of 2.2<span class="formulatext stixSupport mathImg" data-mathurl="/science?_ob=MathURL&_method=retrieve&_eid=1-s2.0-S1574119217302833&_mathId=si2.gif&_user=111111111&_pii=S1574119217302833&_rdoc=1&_issn=15741192&md5=56d34f072a0d9127210fb47b638caa9a" title="Click to view the MathML source">°<span class="formulatext stixSupport mathImg" data-mathurl="/science?_ob=MathURL&_method=retrieve&_eid=1-s2.0-S1574119217302833&_mathId=si1.gif&_user=111111111&_pii=S1574119217302833&_rdoc=1&_issn=15741192&md5=5e85dfbc7032305c4b31535405f35d91" title="Click to view the MathML source"> ± 1.9<span class="formulatext stixSupport mathImg" data-mathurl="/science?_ob=MathURL&_method=retrieve&_eid=1-s2.0-S1574119217302833&_mathId=si2.gif&_user=111111111&_pii=S1574119217302833&_rdoc=1&_issn=15741192&md5=56d34f072a0d9127210fb47b638caa9a" title="Click to view the MathML source">°. The system is extendable in its application and can be incorporated as support for skateboard training and competitions.

Enhancement of speed and accuracy trade-off for sports ball detection in videos—finding fast moving, small objects in real time - Hiemann, Alexander; Kautz, Thomas; Zottmann, Tino; Hlawitschka, Mario (Sensors, 2021)
The detection and localization of the ball in sport videos is crucial to better understand events and actions occurring in those sports. Despite recent advances in the field of object detection, the automatic detection of balls remains a challenging task due to the unsteady nature of balls in images. In this paper, we address the detection of small, fast-moving balls in sport video data and introduce a real-time ball detection approach based on the YOLOv3 object detection model. We apply specific adjustments to the network architecture and training process in order to enhance the detection accuracy and speed: We facilitate an efficient integration of motion information, avoiding a complex modification of the network architecture. Furthermore, we present a customized detection approach that is designed to primarily focus on the detection of small objects. We integrate domain-specific knowledge to adapt image pre-processing and a data augmentation strategy that takes advantage of the special features of balls in images in order to improve the generalization ability of the detection network. We demonstrate that the general trade-off between detection speed and accuracy of the YOLOv3 model can be enhanced in consideration of domain-specific prior knowledge.
CRIS-Team Scopus Importer:2021-05-28
Keywords: Data augmentation; Small object detection; Speed-accuracy trade-off; Sports ball detection; YOLO

Generic performance measure for multiclass-classifiers - Kautz, Thomas; Eskofier, Björn; Pasluosta, Cristian Federico (Pattern Recognition, 2017)
The evaluation of classification performance is crucial for algorithm and model selection. However, a performance measure for multiclass classification problems (i.e., more than two classes) has not yet been fully adopted in the pattern recognition and machine learning community. In this work, we introduce the multiclass performance score (MPS), a generic performance measure for multiclass problems. The MPS was designed to evaluate any multiclass classification algorithm for any arbitrary testing condition. This measure handles the case of unknown misclassification costs and imbalanced data, and provides confidence indicators of the performance estimation. We evaluated the MPS using real and synthetic data, and compared it against other frequently used performance measures. The results suggest that the proposed MPS allows capturing the performance of a classification with minimum influence from the training and testing conditions. This is demonstrated by its robustness towards imbalanced data and its sensitivity towards class separation in feature space.
Keywords: Classification; Multiclass; Performance; Evaluation; Model selection; Accuracy

IMU-based Trick Classification in Skateboarding - Groh, Benjamin; Kautz, Thomas; Schuldhaus, Dominik; Eskofier, Björn (Workshop on Large-Scale Sports Analytics (21st ACM SIGKDD Conference on Knowledge Discovery and Data Mining), 2015)

Mobile Stride Length Estimation with Deep Convolutional Neural Networks. - Hannink, Julius; Kautz, Thomas; Pasluosta, Cristian Federico; Barth, Jens; Schülein, Samuel; Gaßmann, Karl-Günter; Klucken, Jochen; Eskofier, Björn (IEEE Journal of Biomedical and Health Informatics, 2018)

Neuronal cell spike sorting using signal features extracted by PARAFAC - Just, Thomas; Kautz, Thomas; Weis, Martin; Williamson, Adam; Husar, Peter (6th International IEEE/EMBS Conference on Neural Engineering, 2013)

Recent Machine Learning Advancements in Sensor-Based Mobility Analysis: Deep Learning for Parkinson’s Disease Assessment - Eskofier, Björn; Lee, Sunghoon I.; Daneault, Jean-Francois; Golabchi, Fatemeh N.; Ferreira-Carvalho, Gabriela; Vergara-Diaz, Gloria; Sapienza, Stefano; Costante, Gianluca; Klucken, Jochen; Kautz, Thomas; Bonato, Paolo (Proceedings of the 38th IEEE Engineering in Medicine and Biology Society Conference (EMBC 2016), 2016)
The development of wearable sensors has opened the door for long-term assessment of movement disorders. However, there is still a need for developing methods suitable to monitor motor symptoms in and outside the clinic. The purpose of this paper was to investigate deep learning as a method for this monitoring. Deep learning recently broke records in speech and image classification, but it has not been fully investigated as a potential approach to analyze wearable sensor data. We collected data from ten patients with idiopathic Parkinson’s disease using inertial measurement units. Several motor tasks were expert-labeled and used for classification. We specifically focused on the detection of bradykinesia. For this, we compared standard machine learning pipelines with deep learning based on convolutional neural networks. Our results showed that deep learning outperformed other state-of-the-art machine learning algorithms by at least 4.6 % in terms of classification rate. We contribute a discussion of the advantages and disadvantages of deep learning for sensor-based movement assessment and conclude that deep learning is a promising method for this field.

Sensor-based Gait Parameter Extraction with Deep Convolutional Neural Networks. - Hannink, Julius; Kautz, Thomas; Pasluosta, Cristian Federico; Gaßmann, Karl-Günter; Klucken, Jochen; Eskofier, Björn (IEEE Journal of Biomedical and Health Informatics, 2017)
Measurement of stride-related, biomechanical parameters is the common rationale for objective gait impairment scoring. State-of-the-art double integration approaches to extract these parameters from inertial sensor data are, however, limited in their clinical applicability due to the underlying assumptions. To overcome this, we present a method to translate the abstract information provided by wearable sensors to context-related expert features based on deep convolutional neural networks. Regarding mobile gait analysis, this enables integration-free and data-driven extraction of a set of eight spatio-temporal stride parameters. To this end, two modelling approaches are compared: A combined network estimating all parameters of interest and an ensemble approach that spawns less complex networks for each parameter individually. The ensemble approach is outperforming the combined modelling in the current application. On a clinically relevant and publicly available benchmark dataset, we estimate stride length, width and medio-lateral change in foot angle up to -0.15 ±6.09 cm, -0.09 ±4.22 cm and 0.13 ±3.78 ° respectively. Stride, swing and stance time as well as heel and toe contact times are estimated up to ±0.07, ±0.05, ±0.07, ±0.07 and ±0.12 s respectively. This is comparable to and in parts outperforming or defining stateof- the-art. Our results further indicate that the proposed change in methodology could substitute assumption-driven doubleintegration methods and enable mobile assessment of spatiotemporal stride parameters in clinically critical situations as e.g. in the case of spastic gait impairments.

Sensor fusion for multi-player activity recognition in game-sports - Kautz, Thomas; Groh, Benjamin; Eskofier, Björn (Workshop on Large-Scale Sports Analytics (21st ACM SIGKDD Conference on Knowledge Discovery and Data Mining), 2015)
The use of wearable sensors for automatic recognition of human activities has pervaded both professional and recreational sports. While many activities involving only a single athlete can be classified robustly, the automatic classification of complex activities involving several athletes is still in its infancy. In this paper, we present a novel approach for the recognition of such multi-player activities in the context of game sports. Our method is based on the fusion of  position measurements with inertial measurements in a set of interaction features. We demonstrate the efficacy of our method in the recognition of tackles and scrums in Rugby Sevens. The results of our current work suggest that the proposed features can be leveraged to achieve classification accuracies of more than 97%.

Using wearable sensors for semiology-independent seizure detection - towards ambulatory monitoring of epilepsy - Heldberg, Beeke; Kautz, Thomas; Leutheuser, Heike; Hopfengärtner, Rüdiger; Kasper, Burkhard; Eskofier, Björn (Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual Conference, 2015)
Epilepsy is a disease of the central nervous system. Nearly 70% of people with epilepsy respond to a proper treatment, but for a successful therapy of epilepsy, physicians need to know if and when seizures occur. The gold standard diagnosis tool video-electroencephalography (vEEG) requires patients to stay at hospital for several days. A wearable sensor system, e.g. a wristband, serving as diagnostic tool or event monitor, would allow unobtrusive ambulatory long-term monitoring while reducing costs. Previous studies showed that seizures with motor symptoms such as generalized tonic-clonic seizures can be detected by measuring the electrodermal activity (EDA) and motion measuring acceleration (ACC). In this study, EDA and ACC from 8 patients were analyzed. In extension to previous studies, different types of seizures, including seizures without motor activity, were taken into account. A hierarchical classification approach was implemented in order to detect different types of epileptic seizures using data from wearable sensors. Using a k-nearest neighbor (kNN) classifier an overall sensitivity of 89.1% and an overall specificity of 93.1% were achieved, for seizures without motor activity the sensitivity was 97.1% and the specificity was 92.9%. The presented method is a first step towards a reliable ambulatory monitoring system for epileptic seizures with and without motor activity.

Wearable Real-Time Skateboard Trick Visualization and Its Community Perception - Groh, Benjamin; Flaschka, Jasmin; Wirth, Markus; Kautz, Thomas; Fleckenstein, Martin; Eskofier, Björn (IEEE Computer Graphics and Applications, 2016)

Workshop on wearables for sports - Martindale, Christine; Wirth, Markus; Schneegas, Stefan; Zrenner, Markus; Groh, Benjamin; Blank, Peter; Schuldhaus, Dominik; Kautz, Thomas; Eskofier, Björn (2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016)
Wearables are becoming mainstream technology, however there is still room for improvement in the sports domain of this field. Monitoring performance and collecting large scale data are of high interest among athletes - amateurs and professionals alike. The current state-of-the art wearable solutions for sports analysis are able to provide individual statistics to the user, however they have shortcomings in certain aspects, such as isolating and visualizing important information for the user, beyond statistics. This workshop focuses on the application of wearable technology in sports. We will explore novel ideas and application scenarios of how sensors and actuators are capable of supporting athletes in monitoring and improving their performance. We will discuss the design space of the domain by bringing together experts from various communities and exchanging ideas from different perspectives on wearables for sports applications. Participants will collaboratively produce sports related prototype applications.

xLength: Predicting Expected Ski Jump Length Shortly after Take-Off Using Deep Learning - Link, Johannes; Schwinn, Leo; Pulsmeyer, Falk; Kautz, Thomas; Eskofier, Björn (Sensors, 2022)
With tracking systems becoming more widespread in sports research and regular training and competitions, more data are available for sports analytics and performance prediction. We analyzed 2523 ski jumps from 205 athletes on five venues. For every jump, the dataset includes the 3D trajectory, 3D velocity, skis’ orientation, and metadata such as wind, starting gate, and ski jumping hill data. Using this dataset, we aimed to predict the expected jump length (xLength) inspired by the expected goals metric in soccer (xG). We evaluate the performance of a fully connected neural network, a convolutional neural network (CNN), a long short-term memory (LSTM), and a ResNet architecture to estimate the xLength. For the prediction of the jump length one second after take-off, we achieve a mean absolute error (MAE) of 5.3 (Formula presented.) for the generalization to new athletes and an MAE of 5.9 (Formula presented.) for the generalization to new ski jumping hills using ResNet architectures. Additionally, we investigated the influence of the input time after the take-off on the predictions’ accuracy. As expected, the MAE becomes smaller with longer inputs. Due to the real-time transmission of the sensor’s data, xLength can be updated during the flight phase and used in live TV broadcasting. xLength could also be used as an analysis tool for experts to quantify the quality of the take-off and flight phases.
CRIS-Team Scopus Importer:2022-11-25
Keywords: inertial measurement unit; performance analysis; performance prediction; sports analytics; ultra-wideband; wearable sensors