A deep learning enhanced novel software tool for laryngeal dynamics analysis - Kist, Andreas; Gómez, Pablo; Dubrovskiy, Denis; Schlegel, Patrick; Kunduk, Melda; Echternach, Matthias; Patel, Rita; Semmler, Marion; Bohr, Christopher; Durr, Stephan; Schützenberger, Anne; Döllinger, Michael (Journal of Speech Language and Hearing Research, 2021)
Purpose: High-speed videoendoscopy (HSV) is an emerging, but barely used, endoscopy technique in the clinic to assess and diagnose voice disorders because of the lack of dedicated software to analyze the data. HSV allows to quantify the vocal fold oscillations by segmenting the glottal area. This challenging task has been tackled by various studies; however, the proposed approaches are mostly limited and not suitable for daily clinical routine. Method: We developed a user-friendly software in C# that allows the editing, motion correction, segmentation, and quantitative analysis of HSV data. We further provide pretrained deep neural networks for fully automatic glottis segmentation. Results: We freely provide our software Glottis Analysis Tools (GAT). Using GAT, we provide a general threshold-based region growing platform that enables the user to analyze data from various sources, such as in vivo recordings, ex vivo recordings, and high-speed footage of artificial vocalfolds. Additionally, especially for in vivo recordings, we provide three robust neural networks at various speed and quality settings to allow a fully automatic glottis segmentation needed for application by untrained personnel. GAT further evaluates video and audio data in parallel and is able to extract various features from the video data, among others the glottal area waveform, that is, the changing glottal area over time. In total, GAT provides 79 unique quantitative analysis parameters for video-and audio-based signals. Many of these parameters have already been shown to reflect voice disorders, highlighting the clinical importance and usefulness of the GAT software. Conclusion: GAT is a unique tool to process HSV and audio data to determine quantitative, clinically relevant parameters for research, diagnosis, and treatment of laryngeal disorders.
CRIS-Team Scopus Importer:2021-07-02

BAGLS, a multihospital Benchmark for Automatic Glottis Segmentation - Gómez, Pablo; Kist, Andreas; Schlegel, Patrick; Berry, David A.; Chhetri, Dinesh K.; Dürr, Stephan; Echternach, Matthias; Johnson, Aaron M.; Kniesburges, Stefan; Kunduk, Melda; Maryn, Youri; Schützenberger, Anne; et al. (Scientific Data, 2020)
Laryngeal videoendoscopy is one of the main tools in clinical examinations for voice disorders and voice research. Using high-speed videoendoscopy, it is possible to fully capture the vocal fold oscillations, however, processing the recordings typically involves a time-consuming segmentation of the glottal area by trained experts. Even though automatic methods have been proposed and the task is particularly suited for deep learning methods, there are no public datasets and benchmarks available to compare methods and to allow training of generalizing deep learning models. In an international collaboration of researchers from seven institutions from the EU and USA, we have created BAGLS, a large, multihospital dataset of 59,250 high-speed videoendoscopy frames with individually annotated segmentation masks. The frames are based on 640 recordings of healthy and disordered subjects that were recorded with varying technical equipment by numerous clinicians. The BAGLS dataset will allow an objective comparison of glottis segmentation methods and will enable interested researchers to train their own models and compare their methods.
CRIS-Team Scopus Importer:2020-06-26

Biomechanical simulation of vocal fold dynamics in adults based on laryngeal high-speed videoendoscopy - Döllinger, Michael; Gómez, Pablo; Patel, Rita R.; Alexiou, Christoph; Bohr, Christopher; Schützenberger, Anne (PLoS ONE, 2017)
MOTIVATION: Human voice is generated in the larynx by the two oscillating vocal folds. Owing to the limited space and accessibility of the larynx, endoscopic investigation of the actual phonatory process in detail is challenging. Hence the biomechanics of the human phonatory process are still not yet fully understood. Therefore, we adapt a mathematical model of the vocal folds towards vocal fold oscillations to quantify gender and age related differences expressed by computed biomechanical model parameters. METHODS: The vocal fold dynamics are visualized by laryngeal high-speed videoendoscopy (4000 fps). A total of 33 healthy young subjects (16 females, 17 males) and 11 elderly subjects (5 females, 6 males) were recorded. A numerical two-mass model is adapted to the recorded vocal fold oscillations by varying model masses, stiffness and subglottal pressure. For adapting the model towards the recorded vocal fold dynamics, three different optimization algorithms (Nelder-Mead, Particle Swarm Optimization and Simulated Bee Colony) in combination with three cost functions were considered for applicability. Gender differences and age-related kinematic differences reflected by the model parameters were analyzed. RESULTS AND CONCLUSION: The biomechanical model in combination with numerical optimization techniques allowed phonatory behavior to be simulated and laryngeal parameters involved to be quantified. All three optimization algorithms showed promising results. However, only one cost function seems to be suitable for this optimization task. The gained model parameters reflect the phonatory biomechanics for men and women well and show quantitative age- and gender-specific differences. The model parameters for younger females and males showed lower subglottal pressures, lower stiffness and higher masses than the corresponding elderly groups. Females exhibited higher subglottal pressures, smaller oscillation masses and larger stiffness than the corresponding similar aged male groups. Optimizing numerical models towards vocal fold oscillations is useful to identify underlying laryngeal components controlling the phonatory process.
EVALuna2:33166

Degrees of freedom in a vocal fold inverse problem - Gómez, Pablo; Kniesburges, Stefan; Schützenberger, Anne; Bohr, Christopher; Döllinger, Michael (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2017)
Experimental research on human phonation is negatively affected by the complexity of the process and the limiting anatomy of the larynx. Numerical simulation of the vocal folds and the formulation of an inverse problem is one way to remedy this. Several studies have explored this choosing different degrees of freedom (DOFs) in a biomechanical Two-Mass-Model (2MM). The selection of the DOFs in an inverse problem has a critical impact on the quality of possible solutions, but also affects the complexity of the problem and convergence speed in solving it. This work compares previous DOF configurations with several new extended configurations in solving the inverse problem for vocal fold recordings of 20 healthy female subjects. Results indicate that, for the 2MM, uncoupled mass and stiffness, and variable collision strength and damping coefficients improve the matching capabilities. They match physiology and lead to up to 50% smaller errors even for a low number of model evaluations.
CRIS-Team Scopus Importer:2022-09-20
Keywords: High-speed videoendoscopy; Inverse problem; Two-mass-model; Vocal fold oscillation

Intersegmenter Variability in High-Speed Laryngoscopy-Based Glottal Area Waveform Measures - Maryn, Youri; Verguts, Monique; Demarsin, Hannelore; Van Dinther, Joost; Gómez, Pablo; Schlegel, Patrick; Döllinger, Michael (Laryngoscope, 2019)
Objectives/Hypothesis High-speed videoendoscopy (HSV) has potential to objectively quantify vibratory vocal fold characteristics during phonation. Glottal Analysis Tools (GAT) version 2018, developed in Erlangen, Germany, is software for determining various glottal area waveform (GAW) quantities. Before having GAT analyze HSV videos, segmenters have to define glottis manually across videos in a semiautomatic segmentation protocol. Such interventions are hypothesized to induce variability of subsequent GAW measure computation across segmenters and may attenuate GAT measures' reliability to a certain point. This study explored intersegmenter variability in GAT's GAW measures based on semiautomatic image processing. Study Design Cohort study of rater reliability. Methods In total, 20 HSV videos from normophonic and dysphonic subjects with various laryngeal disorders were selected for this study and segmented by three trained segmenters. They separately segmented glottis areas in the same frame sets of the videos. Upon analysis of GAW, GAT offers 46 measures related to topologic GAW dynamic characteristics, GAW periodicity and perturbation characteristics, and GAW harmonic components. To address GAT's reliability, intersegmenter-based variability in these measures was examined with intraclass correlation coefficient (ICC). Results In general, ICC behavior of the 46 GAW measures across three raters was highly acceptable. ICC of one parameter was moderate (0.5 < ICC < 0.75), good for seven parameters (0.75 < ICC < 0.9), and excellent for 38 parameters (0.9 < ICC). Conclusions Overall, high ICC values confirm clinical applicability of GAT for objective and quantitative assessment of HSV. Small intersegmenter differences with actual small parameter differences suggest that manual or semiautomatic segmentation in GAT does not noticeably influence clinical assessment outcome. To guarantee the software's performance, we suggest segmentation training before clinical application. Level of Evidence 2b Laryngoscope, 2019
CRIS-Team WoS Importer:2019-12-27

Laryngeal pressure estimation with a recurrent neural network - Gómez, Pablo; Schützenberger, Anne; Semmler, Marion; Döllinger, Michael (IEEE Journal of Translational Engineering in Health and Medicine, 2019)
Quantifying the physical parameters of voice production is essential for understanding the process of phonation and can aid in voice research and diagnosis. As an alternative to invasive measurements, they can be estimated by formulating an inverse problem using a numerical forward model. However, high-fidelity numerical models are often computationally too expensive for this. This paper presents a novel approach to train a long short-term memory network to estimate the subglottal pressure in the larynx at massively reduced computational cost using solely synthetic training data. We train the network on synthetic data from a numerical two-mass model and validate it on experimental data from 288 high-speed ex vivo video recordings of porcine vocal folds from a previous study. The training requires significantly fewer model evaluations compared with the previous optimization approach. On the test set, we maintain a comparable performance of 21.2% versus previous 17.7% mean absolute percentage error in estimating the subglottal pressure. The evaluation of one sample requires a vanishingly small amount of computation time. The presented approach is able to maintain estimation accuracy of the subglottal pressure at significantly reduced computational cost. The methodology is likely transferable to estimate other parameters and training with other numerical models. This improvement should allow the adoption of more sophisticated, high-fidelity numerical models of the larynx. The vast speedup is a critical step to enable a future clinical application and knowledge of parameters such as the subglottal pressure will aid in diagnosis and treatment selection.
CRIS-Team Scopus Importer:2019-07-16
Keywords: High-speed video; inverse problem; recurrent neural networks; vocal fold dynamics; voice physiology

Low-light image enhancement of high-speed endoscopic videos using a convolutional neural network - Gómez, Pablo; Semmler, Marion; Schützenberger, Anne; Bohr, Christopher; Döllinger, Michael (Medical & Biological Engineering & Computing, 2019)
Laryngeal endoscopy is one of the primary diagnostic tools for laryngeal disorders. The main techniques are videostroboscopy and lately high-speed video endoscopy. Unfortunately, due to the restricting anatomy of the larynx and technical limitations of the recording equipment, many videos suffer from insufficient illumination, which complicates clinical examination and analysis. This work presents an approach to enhance low-light images from high-speed video endoscopy using a convolutional neural network. We introduce a new technique to generate realistically darkened training samples using Perlin noise. Extensive data augmentation is employed to cope with the limited training data allowing training with just 55 videos. The approach is compared against four state-of-the-art low-light enhancement methods and statistically significantly outperforms each on a no-reference (NIQE) and two full-reference (PSNR, SSIM) image quality metrics. The presented approach can be run on consumer-grade hardware and is thereby directly applicable in a clinical context. It is likely transferable to similar techniques such as videostroboscopy.
CRIS-Team WoS Importer:2019-07-12

Physical parameter estimation from porcine ex vivo vocal fold dynamics in an inverse problem framework - Gómez, Pablo; Schützenberger, Anne; Kniesburges, Stefan; Bohr, Christopher; Döllinger, Michael (Biomechanics and Modeling in Mechanobiology, 2017)
This study presents a framework for a direct comparison of experimental vocal fold dynamics data to a numerical two-mass-model (2MM) by solving the corresponding inverse problem of which parameters lead to similar model behavior. The introduced 2MM features improvements such as a variable stiffness and a modified collision force. A set of physiologically sensible degrees of freedom is presented, and three optimization algorithms are compared on synthetic vocal fold trajectories. Finally, a total of 288 high-speed video recordings of six excised porcine larynges were optimized to validate the proposed framework. Particular focus lay on the subglottal pressure, as the experimental subglottal pressure is directly comparable to the model subglottal pressure. Fundamental frequency, amplitude and objective function values were also investigated. The employed 2MM is able to replicate the behavior of the porcine vocal folds very well. The model trajectories' fundamental frequency matches the one of the experimental trajectories in [Formula: see text] of the recordings. The relative error of the model trajectory amplitudes is on average [Formula: see text]. The experiments feature a mean subglottal pressure of 10.16 (SD [Formula: see text]) [Formula: see text]; in the model, it was on average 7.61 (SD [Formula: see text]) [Formula: see text]. A tendency of the model to underestimate the subglottal pressure is found, but the model is capable of inferring trends in the subglottal pressure. The average absolute error between the subglottal pressure in the model and the experiment is 2.90 (SD [Formula: see text]) [Formula: see text] or [Formula: see text]. A detailed analysis of the factors affecting the accuracy in matching the subglottal pressure is presented.
EVALuna2:33371

Rethinking glottal midline detection - Kist, Andreas; Zilker, Julian; Gómez, Pablo; Schützenberger, Anne; Döllinger, Michael (Scientific Reports, 2020)
A healthy voice is crucial for verbal communication and hence in daily as well as professional life. The basis for a healthy voice are the sound producing vocal folds in the larynx. A hallmark of healthy vocal fold oscillation is the symmetric motion of the left and right vocal fold. Clinically, videoendoscopy is applied to assess the symmetry of the oscillation and evaluated subjectively. High-speed videoendoscopy, an emerging method that allows quantification of the vocal fold oscillation, is more commonly employed in research due to the amount of data and the complex, semi-automatic analysis. In this study, we provide a comprehensive evaluation of methods that detect fully automatically the glottal midline. We used a biophysical model to simulate different vocal fold oscillations, extended the openly available BAGLS dataset using manual annotations, utilized both, simulations and annotated endoscopic images, to train deep neural networks at different stages of the analysis workflow, and compared these to established computer vision algorithms. We found that classical computer vision perform well on detecting the glottal midline in glottis segmentation data, but are outperformed by deep neural networks on this task. We further suggest GlottisNet, a multi-task neural architecture featuring the simultaneous prediction of both, the opening between the vocal folds and the symmetry axis, leading to a huge step forward towards clinical applicability of quantitative, deep learning-assisted laryngeal endoscopy, by fully automating segmentation and midline detection.
CRIS-Team Scopus Importer:2020-12-04