Accurate Continuous Prediction of 14 Degrees of Freedom of the Hand from Myoelectrical Signals through Convolutive Deep Learning - Simpetru, Raul; Oßwald, Marius; Braun, Dominik; Souza de Oliveira, Daniela; Cakici, Andre; Del Vecchio, Alessandro (44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022, 2022)
Natural control of assistive devices requires continuous positional encoding and decoding of the user's volition. Human movement is encoded by recruitment and rate coding of spinal motor units. Surface electromyography provides some information on the neural code of movement and is usually decoded into finger joint angles. However, the current approaches to mapping the electrical signal into joint angles are unsatisfactory. There are no methods that allow precise estimation of joint angles during natural hand movements within the large numbers of degrees of freedom of the hand. We propose a framework to train a neural network from digital cameras and high-density surface electromyography from the extrinsic (forearm and wrist) hand muscles. Furthermore, we show that our 3D convolutional neural network optimally predicted 14 functional flexion/extension joints of the hand. We found in our experiments (4 subjects; mean age of 26 +- 2.12 years) that our model can predict individual sinusoidal finger movement at different speeds (0.5 and 1.5 Hz), as well as two and three finger pinching, and hand opening and closing, covering 14 degrees of freedom of the hand. Our deep learning method shows a mean absolute error of 2.78 +- 0.28 degrees with a mean correlation coefficient between predicted and expected joint angles of 0.94, 95% confidence interval (CI) [0.81, 0.98] with simulated real-time inference times lower than 30 milliseconds. These results demonstrate that our approach is capable of predicting the user's volition similar to digital cameras through a non-invasive wearable neural interface. Clinical relevance— This method establishes a viable interface that can be used for both immersive virtual reality medical simulations environments and assistive devices such as exoskeleton and prosthetics

A Generalized Framework for the Study of Spinal Motor Neurons Controlling the Human Hand During Dynamic Movements - Cakici, Andre; Oßwald, Marius; Souza de Oliveira, Daniela; Braun, Dominik; Simpetru, Raul; Kinfe, Thomas Mehari; Eskofier, Björn; Del Vecchio, Alessandro (44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022, 2022)
The human hand possesses a large number of degrees of freedom. Hand dexterity is encoded by the discharge times of spinal motor units (MUs). Most of our knowledge on the neural control of movement is based on the discharge times of MUs during isometric contractions. Here we designed a noninvasive framework to study spinal motor neurons during dynamic hand movements with the aim to understand the neural control of MUs during sinusoidal hand digit flexion and extension at different rates of force development. The framework included 320 high-density surface EMG electrodes placed on the forearm muscles, with markerless 3D hand kinematics extracted with deep learning, and a realistic virtual hand that displayed the motor tasks. The movements included flexion and extension of individual hand digits at two different speeds (0.5 Hz and 1.5 Hz) for 40 seconds. We found on average 4.7 +- 1.7 MUs across participants and tasks. Most MUs showed a biphasic pattern closely mirroring the flexion and extension kinematics. Indeed, a factor analysis method (non-negative matrix factorization) was able to learn the two components (flexion/extension) with high accuracy at the individual MU level (R = 0.87 +- 0.12). Although most MUs were highly correlated with either flexion or extension movements, there was a smaller proportion of MUs that was not task-modulated and controlled by a different neural module (7.1% of all MUs with R < 0.3). This work shows a noninvasive visually guided framework to study motor neurons controlling the movement of the hand in human participants during dynamic hand digit movements.

ANALYSIS OF EMBEDDED GPU ARCHITECTURES FOR AI IN NEUROMUSCULAR APPLICATIONS - Pfenning, Simon; Simpetru, Raul; Pollak, Niklas; Del Vecchio, Alessandro; Fey, Dietmar (IADIS International Journal on Computer Science and Information Systems, 2024)
The advancements in deep neural network design have led to a significant increase in the possibilities and functioning of AI-assisted medical hardware. To make use of this progress in the field of mobile applications or even as wearable devices, a suitable hardware-software ecosystem must be identified to meet the high computation and memory demands of neural networks with minimal energy consumption. In this paper, we analyze an up-to-date heterogenous embedded platform employing a deep convolutional network for hand position recognition through electromyography signals. Our evaluation aimed to determine the optimization efforts required for the architecture to function as a human wearable device and identify the most suitable accelerators on the given platform for this task.<br
Keywords: EMG; GPU; Embedded Hardware; Deep Learning; Prosthesis; AI

Bridging the gap between experimental control and ecological validity in human sensorimotor science - Tsay, Jonathan; Steadman, Nathan; Fleming, Melanie; Gann, Mareike; Di Giulio, Irene; Graef, Cosima; Han, Jinpei; Jayasinghe, Kavindu; Mitchell, Matthew; Chen, Robert; Raheel, Kausar; Semedo, Edgar; et al. (The Journal of Physiology, 2024)
CRIS-Team Scopus Importer:2024-07-19
Keywords: motor systems; movement; sensorimotor control

EVALUATION OF GPU-CENTRIC EMBEDDED ARCHITECTURES IN THE FIELD OF NEUROMUSCULAR EMG-BASED AI - Pfenning, Simon; Simpetru, Raul; Pollak, Niklas; Del Vecchio, Alessandro; Fey, Dietmar (International Conferences on Applied Computing 2023, AC 2023 and WWW/Internet 2023, ICWI 2023, 2023)
The recent progress in the design of deep neural networks has led a huge step forward in inference accuracy and opened new opportunities also in the design of AI aided medical hardware. However deep neural models have a high demand in both computational as well as memory resources. To make these systems applicable in the field, it is necessary to find a suitable hardware-software ecosystem, which can meet these demands at a sufficiently low energy consumption. For this purpose, we evaluated a modern heterogeneous embedded platform with a deep convolutional network for hand position recognition, based on the collection of electromyography signals. The goal of our evaluation was to find out, how much effort must be spent for optimization, if the architecture itself would be sufficient for the usage as human wearable device and which of the available accelerators on the embedded platform are best suited for the task.
CRIS-Team Scopus Importer:2024-12-16
Keywords: CNN; Deep Learning; Embedded Hardware; EMG; GPU

Gaitmap – An Open Ecosystem for IMU-based Human Gait Analysis and Algorithm Benchmarking - Küderle, Arne; Ullrich, Martin; Roth, Nils; Ollenschläger, Malte; Ibrahim, Alzhraa; Moradi, Hamid; Richer, Robert; Seifer, Ann-Kristin; Zürl, Matthias; Simpetru, Raul; Herzer, Liv; Prossel, Dominik; et al. (IEEE Open Journal of Engineering in Medicine and Biology, 2024)
Goal: Gait analysis using inertial measurement units (IMUs) has emerged as a promising method for monitoring movement disorders. However, the lack of public data and easy-to-use open-source algorithms hinders method comparison and clinical application development. To address these challenges, this publication introduces the gaitmap ecosystem, a comprehensive set of open source Python packages for gait analysis using foot-worn IMUs. Methods: This initial release includes over 20 state-of-the-art algorithms, enables easy access to seven datasets, and provides eight benchmark challenges with reference implementations. Together with its extensive documentation and tooling, it enables rapid development and validation of new algorithm and provides a foundation for novel clinical applications. Conclusion: The published software projects represent a pioneering effort to establish an open-source ecosystem for IMU-based gait analysis. We believe that this work can democratize the access to high-quality algorithm and serve as a driver for open and reproducible research in the field of human gait analysis and beyond.<br
Keywords: Pipelines; Benchmark testing; Software algorithms; Ecosystems; Machine learning algorithms; Estimation; Trajectory; Accelerometer; Walking; Biomarker; Biomechanics; Movement Analysis

Identification of Spared and Proportionally Controllable Hand Motor Dimensions in Motor Complete Spinal Cord Injuries Using Latent Manifold Analysis - Simpetru, Raul; Souza de Oliveira, Daniela; Ponfick, Matthias; Del Vecchio, Alessandro (IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024)
The loss of bilateral hand function is a debilitating challenge for millions of individuals that suffered a motor-complete spinal cord injury (SCI). We have recently demonstrated in eight tetraplegic individuals the presence of highly functional spared spinal motor neurons in the extrinsic muscles of the hand that are still capable of generating proportional flexion and extension signals. In this work, we hypothesized that an artificial intelligence (AI) system could automatically learn the spared electromyographic (EMG) patterns that encode the attempted movements of the paralyzed digits. We constrained the AI to continuously output the attempted movements in the form of a digital hand so that this signal could be used to control any assistive system (e.g. exoskeletons, electrical stimulation). We trained a convolutional neural network using data from 13 uninjured (control) participants and 8 tetraplegic participants (7 motor-complete, 1 incomplete) to study the latent space learned by the AI. Our model can automatically differentiate between eight different hand movements, including individual finger flexions, grasps, and pinches, achieving a mean accuracy of 98.3% within the SCI group. Analysis of the latent space of the model revealed that proportionally controllable movements exhibited an elliptical path, while movements lacking proportional control followed a chaotic trajectory. We found that proportional control of a movement can only be correctly estimated if the latent space embedding of the movement follows an elliptical path (correlation =0.73; p <0.001). These findings emphasize the reliability of the proposed system for closed-loop applications that require an accurate estimate of spinal cord motor output.

Influence of spatio-temporal filtering on hand kinematics estimation from high-density EMG signals - Simpetru, Raul; Cnejevici, Vlad; Farina, Dario; Del Vecchio, Alessandro (Journal of Neural Engineering, 2024)
Objective. Surface electromyography (sEMG) is a non-invasive technique that records the electrical signals generated by muscles through electrodes placed on the skin. sEMG is the state-of-the-art method used to control active upper limb prostheses because of the association between its amplitude and the neural drive sent from the spinal cord to muscles. However, accurately estimating the kinematics of a freely moving human hand using sEMG from extrinsic hand muscles remains a challenge. Deep learning has been recently successfully applied to this problem by mapping raw sEMG signals into kinematics. Nonetheless, the optimal number of EMG signals and the type of pre-processing that would maximize performance have not been investigated yet. Approach. Here, we analyze the impact of these factors on the accuracy in kinematics estimates. For this purpose, we processed monopolar sEMG signals that were originally recorded from 320 electrodes over the forearm muscles of 13 subjects. We used a previously published deep learning method that can map the kinematics of the human hand with real-time resolution. Main results. While myocontrol algorithms essentially use the temporal envelope of the EMG signal as the only EMG feature, we show that our approach requires the full bandwidth of the signal in the temporal domain for accurate estimates. Spatial filtering however, had a smaller impact and low-order spatial filters may be suitable. Moreover, reducing the number of channels by ablation resulted in large performance losses. The highest accuracy was reached with the highest number of available sensors (n = 320). Importantly and unexpected, our results suggest that increasing the number of channels above those used in this study may further enhance the accuracy in predicting the kinematics of the human hand. Significance. We conclude that full bandwidth high-density EMG systems of hundreds of electrodes are needed for accurate kinematic estimates of the human hand.<br

Learning a Hand Model from Dynamic Movements Using High-Density EMG and Convolutional Neural Networks - Simpetru, Raul; Arkudas, Andreas; Braun, Dominik; Oßwald, Marius; Souza de Oliveira, Daniela; Eskofier, Björn; Kinfe, Thomas Mehari; Del Vecchio, Alessandro (IEEE Transactions on Biomedical Engineering, 2024)
Objective: Surface electromyography (sEMG) can sense the motor commands transmitted to the muscles. This work presents a deep learning method that can decode the electrophysiological activity of the forearm muscles into the movements of the human hand. Methods: We have recorded the kinematics and kinetics of the hand during a wide range of grasps and individual digit movements that cover 22 degrees of freedom of the hand at slow (0.5 Hz) and comfortable (1.5 Hz) movement speeds in 13 healthy participants. The input of the model consists of 320 non-invasive EMG sensors placed on the extrinsic hand muscles. Results: Our network achieves accurate continuous estimation of both kinematics and kinetics, surpassing the performance of comparable networks reported in the literature. By examining the latent space of the network, we find evidence that it mapped EMG activity into the anatomy of the hand at the individual digit level. In contrast to what is observed from the low-pass filtered EMG and linear decoding approaches, we found that the full-bandwidth EMG (monopolar unfiltered) signals during synergistic and individual digit movements contain distinct neural embeddings that encode each movement of the human hand. These manifolds consistently represent the anatomy of the hand and are generalized across participants. Moreover, we found a task-specific distribution of the embeddings without any presence of correlated activations during multi- and individual-digit tasks. Conclusion/Significance: The proposed method could advance the control of assistive hand devices by providing a robust and intuitive interface between muscle signals and hand movement
Keywords: Thumb; Task analysis; Deep learning; EMG; Force; Deep Learning; Electromyography; Muscles; Recording; Kinematics; Kinematics

MyoGestic: EMG interfacing framework for decoding multiple spared motor dimensions in individuals with neural lesions - Simpetru, Raul; Braun, Dominik; Simon, Arndt; März, Michael; Cnejevici, Vlad; Souza de Oliveira, Daniela; Weber, Nico; Walter, Jonas; Franke, Jörg; Höglinger, Daniel; Prahm, Cosima; Ponfick, Matthias; et al. (Science Advances, 2025)
Restoring motor function in individuals with spinal cord injuries (SCIs), strokes, or amputations is a crucial challenge. Recent studies show that spared motor neurons can still be voluntarily controlled using surface electromyography (EMG), even without visible movement. To harness these signals, we developed a wireless, high-density EMG bracelet and a software framework, MyoGestic. Our system enables rapid adaptation of machine learning models to users’ needs, allowing real-time decoding of spared motor dimensions. In our study, we successfully decoded motor intent from two participants with traumatic SCI, two with spinal stroke, and three with amputations in real time, achieving multiple controllable motor dimensions within minutes. The decoded neural signals could control a digitally rendered hand, an orthosis, a prosthesis, or a two-dimensional cursor. MyoGestic’s participant-centered approach allows a collaborative and iterative development of myocontrol algorithms, bridging the gap between researcher and participant, to advance intuitive EMG interfaces for neural lesion

Proportional and Simultaneous Real-Time Control of the Full Human Hand From High-Density Electromyography - Simpetru, Raul; März, Michael; Del Vecchio, Alessandro (IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023)
Surface electromyography (sEMG) is a non-invasive technique that measures the electrical activity generated by the muscles using sensors placed on the skin. It has been widely used in the field of prosthetics and other assistive systems because of the physiological connection between muscle electrical activity and movement dynamics. However, most existing sEMG-based decoding algorithms show a limited number of detectable degrees of freedom that can be proportionally and simultaneously controlled in real-time, which limits the use of EMG in a wide range of applications, including prosthetics and other consumer-level applications (e.g., human/machine interfacing). In this work, we propose a new deep learning method that can decode and map the electrophysiological activity of the forearm muscles into proportional and simultaneous control of > 20 degrees of freedom of the human hand with real-time resolution and with latency within the neuromuscular delays (< 50 ms). We recorded the kinematics of the human hand during grasping, pinching, individual digit movements and three gestures at slow (0.5 Hz) and fast (0.75 Hz) movement speeds in healthy participants. We demonstrate that our neural network can predict the kinematics of the hand in real-time at a constant 32 predictions per second. To achieve this, we employed transfer learning and created a prediction smoothing algorithm for the output of the neural network that reconstructed the full geometry of the hand in three-dimensional Cartesian space in real-time. Our results demonstrate that high-density EMG signals from the forearm muscles contain almost all the information that is needed to predict the kinematics of the human hand. The proposed method has the capability of predicting the full kinematics of the human hand with real-time resolution with immediate translational impact in subjects with motor impairments.

Simultaneous Control of Human Hand Joint Positions and Grip Force via HD-EMG and Deep Learning - Rahimi, Farnaz; Badamchizadeh, Mohammad Ali; Simpetru, Raul; Ghaemi, Sehraneh; Eskofier, Björn; Del Vecchio, Alessandro (2025)
In myoelectric control, simultaneous control of multiple degrees of freedom can be challenging due to the dexterity of the human hand. Numerous studies have focused on hand functionality, however, they only focused on a few degrees of freedom. In this paper, a 3DCNN-MLP model is proposed that uses high-density sEMG signals to estimate 20 hand joint positions and grip force simultaneously. The deep learning model maps the muscle activity to the hand kinematics and kinetics. The proposed models' performance is also evaluated in estimating grip forces with real-time resolution. This paper investigated three individual dynamic hand movements (2pinch, 3pinch, and fist closing and opening) while applying forces in 10% and 30% of the maximum voluntary contraction (MVC). The results demonstrated significant accuracy in estimating kinetics and kinematics. The average Euclidean distance across all joints and subjects was 11.01 ± 2.22 mm and the mean absolute error for offline and real-time force estimation were found to be 0.8 ± 0.33 N and 2.09 ± 0.9 N respectively. The results demonstrate that by leveraging high-density sEMG and deep learning, it is possible to estimate human hand dynamics (kinematics and kinetics), which is a step forward to practical prosthetic hands.<br
https://cris.fau.de/converis/publicweb/Publication/335935342
Keywords: Deep learning; surface Electromyography; real-time; multi-DoF prediction; grip force

tpcp: Tiny Pipelines for Complex Problems - A set of framework independent helpers for algorithms development and evaluation - Küderle, Arne; Richer, Robert; Simpetru, Raul; Eskofier, Björn (Journal of Open Source Software, 2023)
During algorithm development and analysis researchers regularly use software libraries developed for their specific domain. With such libraries, complex analysis tasks can often be reduced to a couple of lines of code. This not only reduces the amount of implementation required but also prevents errors. The best developer experience is usually achieved when the entire analysis can be represented with the tools provided by a single library. For example, when an entire machine learning pipeline is represented by a scikit-learn pipeline (Pedregosa et al., 2018), it is extremely easy to switch out and train algorithms. Furthermore, train/test leaks and other methodological errors at various stages in the analysis are automatically prevented – even if the user might not be aware of these issues. However, if the performed analysis gets too complex, too specific to an application domain, or requires the use of tooling and algorithms from multiple frameworks, developers lose a lot of the benefits provided by individual libraries. In turn, the required skill level and the chance of methodological errors rise. With tpcp we attempt to overcome the issue by providing higher-level tooling and structure for algorithm development and evaluation that is independent of the frameworks required for the algorithm implementatio