New entries to the SPL EDICS for audio and acoustic signal processing - Christensen, Mads Graesboll; Rabenstein, Rudolf (IEEE Signal Processing Letters, 2012)
This letter describes some of the new entries to the Signal Processing Letters (SPL) Editor's Information Classification Scheme (EDICS) for the topic Audio and Acoustic Signal Processing. © 1994-2012 IEE

New multichannel algorithms for signal enhancement in hearing aids - Kellermann, Walter; Lombard, Anthony; Aichner, Robert; Wouters, J.; Eneman, K. (Conf. Rec. 11. Jahrestagung Deutsche Gesellschaft für Audiologie, 2008)
LMS::774

New results for feature-domain reverberation modeling - Sehr, Armin; Kellermann, Walter (2008 Hands-free Speech Communication and Microphone Arrays, HSCMA 2008, 2008)
To achieve robust distant-talking automatic speech recognition in reverberant environments, the effect of reverberation on the speech feature sequences has to be modeled as accurately as possible. A convolution in the feature domain has been proposed recently in [1, 2, 3, 4] to capture the dispersion of the feature vectors caused by reverberation. These publications use a fixed representation of the acoustic path between speaker and microphone or an elementary statistical reverberation model based on simplifying assumptions. In this contribution, we propose a Monte-Carlo approach that allows for an explicit determination of the joint probability density function of a feature-domain reverbeation model. ©2008 IEE
Keywords: Distant-talking speech recognition; Feature-domain processing; Monte-Carlo method; Reverberation modeling; Robust speech recognition

New results on automatic speech recognition in extremely reverberant environments - Hofmann, Christian; Sehr, Armin; Sahoo, Soyuj Kumar; Maas, Roland; Kellermann, Walter (AIA-DAGA Conference on Acoustics, 2013)
Today’s life offers a plethora of devices equipped with an acoustic human-machine interface. These interfaces, providing comfortable access to smartphones, navigation devices, entertainment systems, and many more, require a high recognition accuracy of the underlying Automatic Speech Recognition (ASR) system. Due to the interfering signals recorded by distant microphones, reliable ASR is in general hard to achieve in real-world acoustic environments. While outdoor applications suffer mainly from noise, indoor applications are additionally impaired by reverberation. The reverberation of typical office or lecture-room environments already renders small-vocabulary recognizers trained on clean speech unusable. Nevertheless, ASR technology would also be highly desirable in very large and reverberant environments, such as large entrance halls or exhibition rooms in museums, e.g., for interaction with a voice-controlled information retrieval system. This contribution explains why such environments do not necessarily preclude the use of ASR systems and provides experimental verification of ASR in an extremely reverberant environment along with a dedicated and efficient ASR training method known as ICEWIN

Noise Residual Learning for Noise Modeling in Distributed Video Coding - van Luong, Huynh; Forchhammer, S. (Picture Coding Symposium, 2012)
Distributed video coding (DVC) is a coding paradigm which exploits the source statistics at the decoder side to reduce the complexity at the encoder. The noise model is one of the inherently difficult challenges in DVC. This paper considers Transform Domain Wyner-Ziv (TDWZ) coding and proposes noise residual learning techniques that take residues from previously decoded frames into account to estimate the decoding residue more precisely. Moreover, the techniques calculate a number of candidate noise residual distributions within a frame to adaptively optimize the soft side information during decoding. A residual refinement step is also introduced to take advantage of correlation of DCT coefficients. Experimental results show that the proposed techniques robustly improve the coding efficiency of TDWZ DVC and for GOP=2 bit-rate savings up to 35% on WZ frames are achieved compared with DISCOVE
LMS::1542
Keywords: distributed video coding; noise residual learning; adaptive noise model

Noise-Robust Adaptation Control for Supervised Acoustic System Identification Exploiting a Noise Dictionary - Haubner, Thomas; Brendel, Andreas; Elminshawi, Mohamed; Kellermann, Walter (IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), 2021)
Keywords: System Identification; Adaptation Control; Nonnegative Matrix Factorization; Acoustic Echo Cancellation

Nonlinear Acoustic Echo Cancellation Based on Pipelined Hermite Filters - Yin, Kai-Li; Halimeh, Mhd Modar; Pu, Yi-Fei; Lu, Lu; Kellermann, Walter (Signal Processing, 2024)
This paper introduces a new class of nonlinear filters for nonlinear acoustic echo cancellation (NLAEC) based on Hermite nonlinear filters (HNFs), which is a sub-class of linear-in-the-parameters nonlinear filters (LIPNFs). Specifically, the basis functions of HNFs include cross-terms of the expanded inputs at different time instants, and are mutually orthogonal for white normally distributed input signals. Although HNFs yield good performance for NLAEC, they suffer from high computational complexity. To tackle this problem, a computationally efficient pipelined variant of HNFs is introduced. The pipelined HNFs (PHNFs) include a nonlinear part followed by a linear one such that the input space is first expanded nonlinearly and then cast by a linear mapping to the output space. Experimental results using both synthesized and real data, and involving several nonlinear scenarios show the effectiveness of the proposed approach. <br
Keywords: Nonlinear adaptive filter; Nonlinear acoustic echo cancellation (NLAEC); linear-in-the-parameters nonlinear filters (LIPNFs); pipelined architecture

Nonlinear Acoustic Echo Cancellation - Küch, Fabian; Kellermann, Walter (Topics in Acoustic Echo and Noise Control, 2006)

Nonlinear acoustic echo cancellation using adaptive orthogonalized power filters - Küch, Fabian; Mitnacht, A.; Kellermann, Walter (2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05, 2005)
In acoustic echo cancellation as, e.g., for mobile communication receivers, loudspeakers and their amplifiers cause significant non-linear distortion in the echo path, resulting in a degradation of the performance of linear echo cancelers. In order to cope with this type of nonlinear echo paths, we discuss a orthogonalized version power filters that can be considered as a parallelized realization of the cascade of a memoryless polynomial followed by a linear filter. As, in the echo cancellation context, the statistics of the speech input are non-stationary and not known in advance, the orthogonalization follows the signal statistics. The performance of the resulting novel nonlinear structure is evaluated by experiments using real hardware. © 2005 IEEE.

Nonlinear acoustic echo cancellation using elitist resampling particle filter - Halimeh, Mhd Modar; Hümmer, Christian; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2018)
This paper considers an effective method for nonlinear acoustic echo cancellation (NL-AEC). More specifically, we model the nonlinear echo path by a latent state vector capturing the coefficients of a memoryless processor and a linear finite impulse response filter. To estimate the posterior probability distribution of the state vector, an elitist particle filter based on evolutionary strategies (EPFES) has been proposed, which evaluates realizations of the latent state vector based on long-term fitness measures. This method includes a manually-tuned recursive calculation of the probabilities that the observation has been produced by the state-vector realizations. For avoiding this manual tuning, we introduce a new approach denoted as Elitist Resampling Particle Filtering (ERPF) which can also be shown to combine the advantages of the Sequential Importance Sampling Particle Filter (SIS-PF) and the Sequential Importance Sampling/Resampling Particle Filter (SIR-PF). This new approach allows universal use and leads to superior system identification performance compared to both the original EPFES as well as the SIR-PF, as verified for a simulated scenario and a real smartphone recording. <br
Keywords: Nonlinear System Identification; Particle Filters; Bayesian Estimation; Nonlinear Acoustic Echo Cancellation.

Nonlinear acoustic echo cancellation with 2nd order adaptive Volterra filters - Stenger, Alexander; Trautmann, Lutz; Rabenstein, Rudolf (Proceedings of the 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-99), 1999)
Acoustic echo cancellers in today's speakerphones or video conferencing systems rely on the assumption of a linear echo path. Low-cost audio equipment or constraints of portable communication systems cause nonlinear distortions, which limit the echo return loss enhancement achievable by linear adaptation schemes. These distortions are a super-position of different effects, which can be modelled either as memoryless nonlinearities or as nonlinear systems with memory. Proper adaptation schemes for both cases of nonlinearities are discussed. An echo canceller for nonlinear systems with memory based on an adaptive second order Volterra filter is presented. Its performance is demonstrated by measurements with small loudspeakers. The results show an improvement in the echo return loss enhancement of 7 dB over a conventional linear adaptive filter. The additional computational requirement for the presented Volterra filter is comparable to that of existing acoustic echo cancellers.

Nonlinear acoustic echo cancellation with fast converging memoryless preprocessor - Stenger, Alexander; Kellermann, Walter (2000 IEEE Interntional Conference on Acoustics, Speech, and Signal Processing, 2000)
Low-cost audio components in hands-free telephone applications call for nonlinear adaptive echo cancellation (AEC). It has been demonstrated that a cascade of a polynomial and an FIR filter can cancel such non-linear echoes. Another technique employs a hard-clipping curve with LMS adapted saturation parameter. For both cascaded systems we derive an LMS-type adaptation using a common framework, and propose stepsize normalizations for both approaches. To achieve sufficiently fast convergence for practical use, an RLS-type adaptation for the polynomial is derived and experimentally verified. Both techniques are compared using real hardware and speech signals, and show robust convergence behaviour and an echo reduction gain of up to 10 dB compared to a linear AEC.

Non-linear dark current fixed pattern noise compensation for variable frame rate moving picture cameras - Schöberl, Michael; Senel, Cihan; Fößel, Siegfried; Bloss, Hans; Kaup, André (17th European Signal Processing Conference, EUSIPCO 2009, 2009)
CMOS image sensors are used in most of the camera systems today. For achieving a high image quality it is essential to compensate for fixed pattern noise. Compensation can be carried out by subtracting an estimated noise value per pixel, either directly on the sensor or in the digital processing. Unfortunately these values are different for each camera and will vary for different exposure times, camera mode settings and temperature. This poses additional challenges for high-end moving picture camera systems. We present a new algorithm for improved fixed pattern noise compensation that extends the currently available linear models. Measurements of a real world camera system and a simulation are used to show the improvements with our algorithm. Significant improvement of the compensated fixed pattern noise over a wide exposure range is shown. This allows the operation of the camera system at a much wider range of frame rates and especially long exposures are now possible. Our algorithm can be implemented without increasing the required memory bandwidth which saves power, size and cost. © EURASIP, 200

Nonlinear line echo cancellation using a simplified second order Volterra filter - Küch, Fabian; Kellermann, Walter (2002 IEEE International Conference on Acoustic, Speech and Signal Processing, 2002)
This paper deals with nonlinearities in line echo cancellation. If a high attenuation of the echo signal is required, the nonlinear behaviour of the actual echo path has to be taken into account. As the echo path inherently has memory, the nonlinear approaches have to be capable to model nonlinear systems with memory. Adaptive Volterra filters represent a common method to deal with such systems. Unfortunately, they suffer from a high computational complexity. Therefore, we propose an adaptive structure representing a simplified realization of a special second order Volterra filter (SVF) and provide an efficient step-size control for speech input. For the given application this structure achieves the same performance as second order Volterra filters at remarkably reduced computational cost.

Nonlinear residual echo suppression using a power filter model of the acoustic echo path - Küch, Fabian; Kellermann, Walter (2007 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '07, 2007)
Loudspeakers and amplifiers of mobile communication receivers may cause significant nonlinear distortion in the acoustic echo path, resulting in a limitation of the performance of linear echo cancelers. In this contribution, we present a nonlinear acoustic echo suppressor in order to increase the attenuation of the nonlinearly distorted residual echo. The proposed approach is based on a power filter model of the acoustic echo path which is applied to the estimation of the power spectral density of the nonlinear residual echo. These time-variant estimates are used to appropriately adjust the frequency-dependent gain values of the echo suppressor. The performance of the proposed approach is evaluated in realistic experimental set-ups. © 2007 IEE
Keywords: Echo suppression; Nonlinear filters

Non-Recursive System Identification and Fault Detection in LVDC and ELVDC Grids - Strobl, Christian; Schäfer, Maximilian; Rabenstein, Rudolf (IEEE Int. Symp. on Circuits and Systems (ISCAS), 2018)
Low end extra low voltage direct current grids require selective fault protection designed for the specific application and system voltage. System identification and machine learning methods are helpful to identify, to localize and to classify occurring fault events. A category of non-recursive large-signal methods in the time domain for system identification and for refined fault detection and analysis is introduced.

Novel Consistency Check for Fast Recursive Reconstruction of Non-Regularly Sampled Video Data - Grosche, Simon; Seiler, Jürgen; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2021)

Novel features for the detection of bearing faults in railway vehicles - Kreuzer, Matthias; Schmidt, Alexander; Kellermann, Walter (Inter-Noise 2021 - The 50th International Congress and Exposition on Noise Control Engineering, 2021)
In this paper, we address the challenging problem of detecting bearing faults from vibration signals. For this, several time- and frequency-domain features have been proposed already in the past. However, these features are usually evaluated on data originating from relatively simple scenarios and a significant performance loss can be observed if more realistic scenarios are considered. To overcome this, we introduce Mel Frequency Cepstral Coefficients (MFCCs) and features extracted from the Amplitude Modulation Spectrogram (AMS) as features for the detection of bearing faults. Both AMS and MFCCs were originally introduced in the context of audio signal processing but it is demonstrated that a significantly improved classification performance can be obtained by using these features. Furthermore, to tackle the characteristic data imbalance problem in the context of bearing fault detection, i.e., typically much more data from healthy bearings than from damaged bearings is available, we propose to train a One-class Support Vector Machine (SVM) with data from healthy bearings only. Bearing faults are then classified by the detection of outliers. Our approach is evaluated with data measured in a highly challenging scenario comprising a state-of-the-art commuter railway engine which is supplied by an industrial power converter and coupled to a load machin

Novel schemes for nonlinear acoustic echo cancellation based on filter combinations - Azpicueta-Ruiz, Luis A.; Zeller, Marcus; Arenas-García, Jerónimo; Kellermann, Walter (2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009, 2009)
Nonlinear acoustic echo cancellers (NLAEC) are becoming increasingly important in hands-free applications. However, in some situations, an NLAEC is inferior to a linear AEC, especially when the channel generates a negligible (or no) nonlinear echo. In general, the ratio of the linear to nonlinear echo signal power is unknown a priori, and will vary over time, thus making it difficult to know if an NLAEC would improve or degrade the cancellation. In this paper, we present two novel solutions to this problem based on the adaptive combination of linear and nonlinear echo cancellers. Both solutions perform efficiently regardless of the level of nonlinear echo. The benefits and robustness of both schemes are illustrated by experiments using Laplacian colored noise and speech input signals. ©2009 IEE
Keywords: Adaptive filters; Combination of filters; Nonlinear acoustic echo cancellation; Volterra filters

Novel Techniques for Void Filling in Glacier Elevation Change Data Sets - Seehaus, Thorsten; Morgenshtern, Veniamin; Hübner, Fabian; Bänsch, Eberhard; Braun, Matthias (Remote Sensing, 2020)
The increasing availability of digital elevation models (DEMs) facilitates the monitoring of glacier mass balances on local and regional scales. Geodetic glacier mass balances are obtained by differentiating DEMs. However, these computations are usually affected by voids in the derived elevation change data sets. Different approaches, using spatial statistics or interpolation techniques, were developed to account for these voids in glacier mass balance estimations. In this study, we apply novel void filling techniques, which are typically used for the reconstruction and retouche of images and photos, for the first time on elevation change maps. We selected 6210 km2 of glacier area in southeast Alaska, USA, covered by two void-free DEMs as the study site to test different inpainting methods. Different artificially voided setups were generated using manually defined voids and a correlation mask based on stereoscopic processing of Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) acquisition. Three “novel” (Telea, Navier–Stokes and shearlet) as well as three “classical” (bilinear interpolation, local and global hypsometric methods) void filling approaches for glacier elevation data sets were implemented and evaluated. The hypsometric approaches showed, in general, the worst performance, leading to high average and local offsets. Telea and Navier–Stokes void filling showed an overall stable and reasonable quality. The best results are obtained for shearlet and bilinear void filling, if certain criteria are met. Considering also computational costs and feasibility, we recommend using the bilinear void filling method in glacier volume change analyses. Moreover, we propose and validate a formula to estimate the uncertainties caused by void filling in glacier volume change computations. The formula is transferable to other study sites, where no ground truth data on the void areas exist, and leads to higher accuracy of the error estimates on void-filled areas. In the spirit of reproducible research, we publish a software repository with the implementation of the novel void filling algorithms and the code reproducing the statistical analysis of the data, along with the data sets themselves.<br
Keywords: glacier mass balance; elevation change; void filling

Numerical integration of nonlinear multidimensional systems - Rabenstein, Rudolf; et al. (Proc. Int. Conf. Acoustics, Speech, and Signal Proc. (ICASSP 97), 1997)
The suitability of methods from multidimensional systems theory and digital signal processing for the efficient simulation of time and space dependent problems has already been demonstrated. Properly chosen functional transformations for the time and space coordinates turn a partial differential equation into a transfer function description of a multidimensional continuous system. It serves as the starting point for the derivation of a discrete system which closely models the behaviour of the given continuous system and which is suitable for computer implementation. This concept is extended here to the simulation of nonlinear multidimensional systems. The essence of the presented method is a systematic way to turn a nonlinear partial differential equation into a set of ordinary differential equations, for which standard methods for the numerical integration exist. This paper briefly reviews the linear case, points out the various difficulties arising from nonlinearity and shows how to overcome them. Numerical results demonstrate the effectiveness of the metho

Numerical iterative methods and repetitive processes - Rabenstein, Rudolf; Steffen, Peter (Multidimensional Systems and Signal Processing, 2012)
Implicit schemes are a popular approach to the discretization of linear partial differential equations by finite differences. They require to solve a set of linear equations in each time step. Since finite difference discretizations lead to a local coupling, these systems of equations are sparse and can be effectively solved by iterative methods. Numerical procedures of this type are known in control theory as repetitive processes. They have mostly been used to solve problems in control like processes where passes of finite length are repeated over and over. This paper shows how the implicit discretization of partial differential equations can be cast into the framework of repetitive processes. Thus it establishes a link between yet unrelated results in numerical mathematics and control theory. © 2010 Springer Science+Business Media, LL
Keywords: Iterative methods; Numerical mathematics; Partial differential equations; Repetitive processes

Numerical Simulation of Wave Fields Created by Loudspeaker Arrays - Horbach, Ulrich; Karamustafaoglu, A.; Rabenstein, Rudolf; Runze, Gerhard; Steffen, Peter (Proc. 107th Convention of the Audio Engineering Society, 1999)
LMS::309

Numerical Solution of Physics-based Models by Functional Transformations - Rabenstein, Rudolf (Ninth Workshop on Image and Multidimensional Signal Processing, 1996)

Numerical Solution of the Telegraph Equation by Multidimensional Digital Filters - Rabenstein, Rudolf; Krauss, Herbert (Proceedings of the International Conference on Digital Signal Processing and II. International Conference on Computer Applications to Engineering Systems, 1993)

Oberflächenstrukturierter Aliasfilter für digitale Kameras - Ernst, J.; Schöberl, Michael (Fachzeitschrift für Fernsehen, Film und elektronische Medien, 2011)
In hochwertigen elektronischen Kameras sind Aliasfilter ein obligatorisches optisches Bauelement. Sie werden zur Unterdrückung von Bildstörungen eingesetzt, die bei feinen Strukturen und scharfen Kanten im Bild entstehen. Im Artikel werden zunächst die Hintergründe für die Entstehung von Aliaseffekten erläutert und die prinzipielle Methode zu deren Unterdrückung beschrieben. Es wird erklärt, wie die heute üblichen Aliasfilter funktionieren, welche Nachteile sie haben und wie sie optimiert werden können. Schließlich wird ein neuartiger Ansatz für die Realisierung von Aliasfiltern auf der Basis von Oberflächenstrukturen vorgestell
LMS::1102

Object-Based Texture Coding of Moving Video in MPEG-4 - Kaup, André (IEEE Transactions on Circuits and Systems For Video Technology, 1999)
LMS::457

Object Localization by Joint Audio-Video Signal Processing - Wang, Guangyu; Rabenstein, Rudolf; Strobel, Norbert; Spors, Sascha (Proceedings of Vision, Modeling and Visualization (VMV), 2000)
LMS::213

Object Localization using Audio and Video Signals - Strobel, Norbert; Rabenstein, Rudolf (Principles of 3D Image Analysis and Synthesis, 2000)

Object Reconstruction Using Spatially Extended Voxels - Steinbach, Eckehard; Eisert, Peter; Betz, A.; Girod, Bernd (ICPR 2000, 2000)
LMS::166

Objektbasierte Codierung von bewegungskompensierten Prädiktionsfehlerbildern in MPEG-4 - Kaup, André; Hundt, E. (2. ITG-Fachtagung Codierung für Quelle, Kanal und Übertragung, ITG-Fachbericht 146, 1998)
LMS::469

On 2D localization of reflectors using robust beamforming techniques - Mabande, Edwin; Sun, Haohai; Kowalczyk, Konrad; Kellermann, Walter (36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 2011)
This paper presents a method for the localization of reflectors in an acoustic environment, using robust beamforming techniques and a cylindrical microphone array, for which an intuitive and highly efficient three-step procedure is proposed. First, the directions of arrival (DOAs) corresponding to the sound source and reflectors are estimated by a robust Minimum Variance Distortionless Response (MVDR) beamformer. Next, signals originating from the estimated DOAs are extracted by a robust superdirective beamformer, from which time differences of arrival (TDOAs) of major reflections are estimated by crosscorrelation analysis. Finally, by using additional information about the position of the direct sound source relative to the array, the positions of the reflective boundaries of the room can be inferred. Experiments based on real measurements carried out in a moderately reverberant room show the effectiveness of this method. © 2011 IEE
Keywords: Beamforming; Room Inference; Source Localization; TDOA Estimation

On Annotation-free Optimization of Video Coding for Machines - Windsheimer, Marc; Brand, Fabian; Kaup, André (International Conference on Image Processing (ICIP), 2024)
Today, image and video data is not only viewed by humans, but also automatically analyzed by computer vision algorithms. However, current coding standards are optimized for human perception. Emerging from this, research on video coding for machines tries to develop coding methods designed for machines as information sink. Since many of these algorithms are based on neural networks, most proposals for video coding for machines build upon neural compression. So far, optimizing the compression by applying the task loss of the analysis network, for which ground truth data is needed, is achieving the best coding performance. But ground truth data is difficult to obtain and thus an optimization without ground truth is preferred. In this paper, we present an annotation-free optimization strategy for video coding for machines. We measure the distortion by calculating the task loss of the analysis network. Therefore, the predictions on the compressed image are compared with the predictions on the original image, instead of the ground truth data. Our results show that this strategy can even outperform training with ground truth data with rate savings of up to 7.5 %. By using the non-annotated training data, the rate gains can be further increased up to 8.2
Keywords: Video Coding for Machines; Neural Network Compression; Computer Vision; Learned Image Coding; Machine-to-Machine Communication

On Bayesian networks in speech signal processing - Maas, Roland; Hümmer, Christian; Hofmann, Christian; Kellermann, Walter (11. ITG Symposium on Speech Communication, 2014)
This paper describes a class of relevant speech signal processing algorithms as probabilistic inference problems. Starting with an observation model that relates all involved random variables, we convert the respective joint probability density function into its Bayesian network representation in order to infer the desired signal estimates. After recalling the well-known Bayesian network descriptions of Wiener filtering and adaptive filtering, we show how the proportionate normalized least mean square (PNLMS) algorithm arises under certain restrictive assumptions on the covariance matrices of the latent random variables. In this context, the inherent relation of the Kalman filter, the normalized least mean square (NLMS), and the PNLMS algorithm is moreover outlined. We finally recall that also unsupervised signal estimation problems, such as dereverberation, can be considered from the same point of vie

On Benefits and Challenges of Conditional Interframe Video Coding in Light of Information Theory - Brand, Fabian; Seiler, Jürgen; Kaup, André (2022 PICTURE CODING SYMPOSIUM (PCS), 2022)
CRIS-Team WoS Importer:2023-03-03

On blocking matrix-based dereverberation for automatic speech recognition - Schwarz, Andreas; Reindl, Klaus; Kellermann, Walter (International Workshop on Acoustic Signal Enhancement (IWAENC), 2012)
We investigate a two-channel reverberation suppression scheme comprising a blocking matrix for estimating late reverberation components by canceling the direct path and early reflections, and spectral enhancement filters for suppressing the late reverberation components. For idealized blocking matrices, we analyze the influence of the length of the blocking matrix filters and the impact of the coherence between the microphones on the resulting estimate. We show that blocking matrices that cancel more than the direct path of the desired signal can be of advantage for robust speech recognition in highly reverberant environment
Keywords: Dereverberation; Distant-talking speech recognition; Reverberation suppression; Signal enhancement

On Differential Beamforming with Nonuniform Linear Microphone Arrays - Jin, Jilu; Benesty, Jacob; Huang, Gongping; Chen, Jingdong (IEEE/ACM Transactions on Audio, Speech and Language Processing, 2022)
While differential beamforming with uniform linear arrays (ULAs) has been widely studied, there is little work so far regarding the design of differential beamformers with nonuniform linear arrays (NULAs). This article attempts to shed some light on the principles of differential beamforming with NULAs. We define spatial difference operators with NULAs, where any order of the spatial difference of the observation signals can be represented as the product of a nonuniform spatial difference operator matrix and the observation vector. Consequently, the design of differential beamformers is performed in two stages. In the first one, a nonuniform spatial difference operator matrix is applied to the array observations, thereby yielding differential signals. In the second stage, beamformers are designed and applied to the obtained differential signals to optimize the array performance. Based on the defined spatial difference operators, we derive from some performance metrics a family of differential beamformers with NULAs, which include the maximum directivity factor (DF), the maximum white noise gain (WNG), and the maximum front-to-back ratio (FBR) differential beamformers. To compromise between the DF and array robustness, we also derive the parameterized maximum DF and parameterized maximum FBR differential beamformers. The null-constraint maximum DF and WNG differential beamformers are also developed so that some nulls can be placed in specified directions for interference suppression. Simulation results validate the theoretical analysis and justify the properties of the proposed methods.
CRIS-Team Scopus Importer:2022-06-24
Keywords: Differential beamforming; directivity factor; front-to-back ratio; nonuniform linear arrays; spatial difference operator; white noise gain

One- and Two-Dimensional Window Sequences with Few Coefficients - Rabenstein, Rudolf (Mediterranien Electrical Engineering Conference (MELECON'85), 1985)
LMS::1587

One-Pass Frame Level Budget Allocation Based on inter-frame Dependency - Li, Xiang; Hutter, Andreas; Kaup, André (2009 IEEE International Workshop on Multimedia Signal Processing, MMSP '09, 2009)
In this paper, a one-pass budget allocation algorithm is proposed for hybrid video coding. Taking the percentage of skipped MBs as the measure of inter-frame dependency, the optimal budget allocation is first modeled for a two-frame case. Then this model is extended to a practical method in slow movement scenario, where the information of inter-frame dependency is predicted based on the previously coded frames. Simulations show that the proposed algorithm outperforms the recommended MB and frame level rate control algorithms in H.264/AVC reference software JM 15.1. For eight CIF sequences with slow movement, significant gains of 0.91 dB and 0.43 dB on average (1.66 dB and 1.49 dB at maximum) were obtained over the reference MB and frame level rate control, respectively. Considering that the computational complexity by the proposed algorithm is quite low (less than 1% of the total coding time when fast motion estimation algorithm is enabled), it is quite appealing for real-time video applications. © 2009 IEE

One-Pass Multi-Layer Rate-Distortion Optimization for H.264/SVC Quality Scalable Video Coding - Li, Xiang; Amon, Peter; Hutter, Andreas; Kaup, André (2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009, 2009)
In this paper, a one-pass multi-layer rate-distortion optimization algorithm is proposed for quality scalable video coding. To improve the overall coding efficiency, the MB mode in the base layer is selected not only based on its rate-distortion performance relative to this layer but also according to its impact on the enhancement layer. Moreover, the optimization module for residues is also improved to benefit inter-layer prediction. Simulations show that the proposed algorithm outperforms the most recent SVC reference software. For eight test sequences, a gain of 0.35 dB on average and 0.75 dB at maximum is achieved at a cost of less than 8% increase of the total coding time. ©2009 IEE
Keywords: H.264/AVC; Multi-layer RDO; Quality scalable video coding; SVC

One-Shot Object Detection in Heterogeneous Artwork Datasets - Madhu, Prathmesh; Meyer, Anna; Zinnen, Mathias; Mührenberg, Lara; Suckow, Dirk; Bendschus, Torsten; Reinhardt, Corinna; Bell, Peter; Verstegen, Ute; Kosti, Ronak Vijaypal; Maier, Andreas; Christlein, Vincent (2022 Eleventh International Conference on Image Processing Theory, Tools and Applications (IPTA), 2022)
Christian archeologists face many challenges in understanding visual narration through artwork images. This understanding is essential to access underlying semantic in-formation. Therefore, narrative elements (objects) need to be labeled, compared, and contextualized by experts, which takes an enormous amount of time and effort. Our work aims to reduce labeling costs by using one-shot object detection to generate a labeled database from unannotated images. Novel object categories can be defined broadly and annotated using visual examples of narrative elements without training exclusively for such objects. In this work, we propose two ways of using contextual information as data augmentation to improve the detection performance. Furthermore, we introduce a multi-relation detector to our framework, which extracts global, local, and patch-based relations of the image. Additionally, we evaluate the use of contrastive learning. We use data from Christian archeology (CHA) and art history - IconArt-v2 (IA). Our context encoding approach improves the typical fine-tuning approach in terms of mean average precision (mAP) by about 3.5 % (4 %) at 0.25 intersection over union (IoU) for UnSeen categories, and 6 % (1.5 %) for Seen categories in CHA (IA). To the best of our knowledge, our work is the first to explore few shot object detection on heterogeneous artistic data by investigating evaluation methods and data augmentation strategies. We will release the code and models after acceptance of the wor
Keywords: one-shot; object detection; digital humanities; computational humanities; data augmentation

On Interpolation of Subjective Rate-Distortion Curves for Video Coder Comparison - Brand, Fabian; Herglotz, Christian; Kaup, André (2023 15th International Conference on Quality of Multimedia Experience (QoMEX), 2023)
When comparing two video coders, in order to obtain meaningful and unambiguous results, it is necessary to interpolate the rate-distortion curves. Typical methods like the BjØntegaard delta rate use piece-wise cubic interpolation for that purpose. This works well for the unbounded quality metric PSNR. For fitting the curve of saturating quality metrics, which are common when measuring subjective or perceptual quality, the use of a logistic fitting function was proposed with SCENIC earlier. However, the interpolation accuracy has not been validated. In this paper, we compare different methods for interpolating rate-distortion curves of subjective metrics. We come to the conclusion that SCENIC is in fact better than piece-wise cubic interpolation for four supporting points. With more supporting points, however it is restricted by the number of free parameters, and piece-wise cubic interpolation should be used. We furthermore show that logarithmic rescaling has a small benefit for the interpolation accuracy of MOS values.<br
Keywords: Measurement; Interpolation; Fitting; Rate-distortion; Streaming media; Logistics; video compression; codec; subjective quality; VMAF; rate-distortion

On Intra Video Coding and In-loop Filtering for Neural Object Detection Networks - Fischer, Kristian; Herglotz, Christian; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2020)
Classical video coding for satisfying humans as the final user is a widely investigated field of studies for visual content, and common video codecs are all optimized for the human visual system (HVS). But are the assumptions and optimizations also valid when the compressed video stream is analyzed by a machine? To answer this question, we compared the performance of two state-of-the-art neural detection networks when being fed with deteriorated input images coded with HEVC and VVC in an autonomous driving scenario using intra coding. Additionally, the impact of the three VVC in-loop filters when coding images for a neural network is examined. The results are compared using the mean average precision metric to evaluate the object detection performance for the compressed inputs. Throughout these tests, we found that the Bjøntegaard Delta Rate savings with respect to PSNR of 22.2 % using VVC instead of HEVC cannot be reached when coding for object detection networks with only 13.6 % in the best case. Besides, it is shown that disabling the VVC in-loop filters SAO and ALF results in bitrate savings of 6.4 % compared to the standard VTM at the same mean average precision. <
Keywords: machine to machine communication; video coding for machines (VCM); neural object detection; versatile video coding (VVC); in-loop filtering

Online Acoustic System Identification Exploiting Kalman Filtering and an Adaptive Impulse Response Subspace Model - Haubner, Thomas; Brendel, Andreas; Kellermann, Walter (Journal of Signal Processing Systems, 2022)
We introduce a novel algorithm for online estimation of Acoustic Impulse Responses (AIRs) which allows for fast convergence by exploiting prior knowledge about the fundamental structure of AIRs. The proposed method assumes that the variability of AIRs of an acoustic scene is confined to a low-dimensional manifold which is embedded in a high-dimensional space of possible AIR estimates. We discuss various approaches which exploit a training data set of AIRs, e.g., high-accuracy AIR estimates from the acoustic scene, to learn a local affine subspace approximation of the AIR manifold. The model is motivated by the idea of describing the generally nonlinear AIR manifold locally by tangential hyperplanes and its validity is verified for simulated data. Subsequently, we describe how the manifold assumption can be used to enhance online AIR estimates by projecting them onto an adaptively estimated subspace. Motivated by the assumption of manifolds being locally Euclidean, the parameters determining the adaptive subspace are learned from the nearest neighbor AIR training samples to the current AIR estimate. To assess the proximity of training data AIRs to the current AIR estimate, we introduce a probabilistic extension of the Euclidean distance which improves the performance for applications with non-white excitation signals. Furthermore, we describe how model imperfections can be tackled by a soft projection of the AIR estimates. The proposed algorithm exhibits significantly faster convergence properties in comparison to a high-performance state-of-the-art algorithm. Furthermore, we show an improved steady-state performance for speech-excited system identification scenarios suffering from high-level interfering noise and nonunique solution
Keywords: Online Acoustic System Identification; Kalman Filter; Subspace Model; Adaptation Control

Online decomposition of compressive streaming data using n-l1 cluster-weighted minimization - van Luong, Huynh; Deligiannis, Nikos; Forchhammer, Soren; Kaup, André (Data Compression Conference, 2018)

Online environmental adaptation of CNN-based acoustic models using spatial diffuseness features - Hümmer, Christian; Delcroix, Marc; Ogawa, Atsunori; Kinoshita, Keisuke; Nakatani, Tomohiro; Kellermann, Walter (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017)
We propose a new concept for adapting CNN-based acoustic models using spatial diffuseness features as auxiliary information about the acoustic environment: the spatial diffuseness features are simultaneously employed as acoustic-model input features and to estimate environmental cues for context adaptation, where one convolutional layer is factorized into several sub-layers to represent different acoustic conditions. This context-adaptive CNN-based acoustic model facilitates an online environmental adaptation and is experimentally verified for the real-world recordings provided by the CHiME-3 task. The best performing setup reduces the average word error rate scores achieved by the baseline system (without using spatial diffuseness features) from 19.4% to 15.9% and 12.2% to 10.7% considering two experimental setups with and without front-end signal enhancement, respectivel
LMS::1711
Keywords: Context adaptation; spatial diffuseness features; CNN-based acoustic model; environmental robustness

Online estimation of the optimum quadratic kernel size of second-order Volterra filters using a convex combination scheme - Zeller, Marcus; Azpicueta-Ruiz, Luis A.; Kellermann, Walter (2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009, 2009)
This paper presents a method for estimating the optimum memory size for identification of an unknown second-order Volterra kernel. As these structures may imply considerable computational demands, it is highly desirable to design adaptive realizations with a minimum number of coefficients. Therefore, we propose a combination scheme comprising two Volterra filters with time-variant sizes of the actually used quadratic kernels. By following some simple rules, the number of diagonals in the quadratic kernels is increased or decreased in order to find the optimum memory configuration in parallel to the coefficient adaptation. Thus, the arbitrary choice of the nonlinear system size is overcome by a dynamically growing/ shrinking system. Experimental results for various signals and nonlinear scenarios demonstrate the effectiveness of the proposed method. ©2009 IEE
Keywords: Adaptive Volterra filter; Convex combination; Nonlinear system identification; Structure selection

Online estimation of time-variant microphone utility in wireless acoustic sensor networks using single-channel signal features - Günther, Michael; Brendel, Andreas; Kellermann, Walter (29th European Signal Processing Conference (EUSIPCO 2021), 2021)

Online secondary path modeling in wave-domain active noise control - Zhang, Wen; Hofmann, Christian; Bürger, Michael; Kellermann, Walter (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017)
The performance of an ANC system largely depends on the availability of an accurate secondary path model. This is however a major challenge in multichannel ANC where the computational complexity increases significantly with the number of secondary sources and error sensors. This paper proposes wave-domain adaptive processing algorithm for multichannel ANC with online secondary modelling to cancel a tonal noise over a region of space within a reverberant room. The design is based on exploiting a special property of the secondary path model in the wave domain. A feedback control system is implemented, where a single microphone array is placed at the boundary of the control region to measure the residual signals, and a loudspeaker array reproduces secondary sources to generate the anti-noise signals and auxiliary noise for secondary path modelling. Through experimental verification in comparison with existing methods the proposed algorithm demonstrates more efficient adaptation with low-level auxiliary noise
Keywords: Active noise control; secondary path modelling; wave domain; adaptive processing; feedback system

Online Supervised Acoustic System Identification exploiting Prelearned Local Affine Subspace Models - Haubner, Thomas; Brendel, Andreas; Kellermann, Walter (IEEE International Workshop on Machine Learning in Signal Processing (MLSP), 2020)

On-line Time-Domain Blind Source Separation of Nonstationary Convolved Signals - Aichner, Robert; Buchner, Herbert; Araki, S.; Makino, Shoji (Proc. Int. Symposium on Independent Component Analysis and Blind Signal Separation (ICA), 2003)
LMS::370

On Prefilter Computation for Reduced-State Equalization - Gerstacker, Wolfgang; Obernosterer, Frank; Meyer, Raimund; Huber, Johannes (IEEE Transactions on Wireless Communications, 2002)
In advanced time-division multiple-access (TDMA) mobile communications systems, reduced-state equalization algorithms have to be employed because high-level modulation is used in order to improve spectral efficiency. Reduced-state equalizers yield only high performance, if the overall discrete-time system to be equalized is minimum-phase. Therefore, in general, a discrete-time prefllter has to be inserted in front of equalization. For prefllter computation, several approaches are investigated in this paper. For the finite impulse response (FIR) prefilter case, which seems to be more relevant for practical applications than the infinite impulse response case, we discuss a method based on minimum mean-squared error decision-feedback equalization and a novel approach based on linear prediction (LP). The LP method seems to be very robust and requires an only moderate amount of computational complexity. Here, the prefilter consists of the cascade of a channel-matched filter and a prediction-error filter, which may be viewed as a finite-length approximation to the noise whitening part of the ideal prefilter transfer function. A key observation of the paper is that the proposed cascaded structure enables a very efficient prefilter computation because a prediction-error filter can be calculated via the Levinson-Durbin algorithm. Simulation results are given, which demonstrate that the performance of reduced-state equalization with proper FIR prefiltering is close to that of equalization combined with ideal all-pass prefiltering. Furthermore, it is shown that high performance can be obtained for TDMA mobile communications systems, if the LP scheme is employed for prefiltering. © 2002 IEEE.
UnivIS-Import:2015-03-09:Pub.2002.tech.IE.mobilk.onpref
Keywords: EDGE; Minimum-phase systems; Preflltering; Reduced-state equalization; Time-division multiple-access mobile communications

On real and complex valued L1-norm minimization for overcomplete blind source separation - Winter, Stefan; Sawada, H.; Makino, Shoji (2005 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2005)
LMS::595

On requantization in intra-frame video transcoding with different transform block sizes - Bialkowski, Jens; Barkowsky, Marcus; Kaup, André (2005 IEEE 7th Workshop on Multimedia Signal Processing, MMSP 2005, 2005)
Transcoding is a technique to convert one video bit-stream into another. While homogeneous transcoding is done at the same coding standard, inhomogeneous transcoding converts from one standard format to another standard. Inhomogeneous transcoding between MPEG-2, MPEG-4 or H.263 was performed using the same transform. With the standardisation of H.264 also a new transform basis and different block size was defined. For requantization from block size 8 × 8 to 4 × 4 this leads to the effect that the quantization error of one coefficient in a block of size 8 × 8 is distributed over multiple coefficients in blocks of size 4 × 4. In our work, we analyze the requantization process for inhomogeneous transcoding with different transforms. The deduced equations result in an expression for the correlation of the error contributions from the coefficients of block size 8 x 8 at each coefficient of block size 4×4. We then compare the mathematical analysis to simulations on real sequences. The reference to the requantization process is the direct quantization of the undistorted signal. It will be shown that the loss is as high as 3 dB PSNR at equivalent step size for input and output bitstream. Also an equation for the choice of the second quantization step size in dependency of the requantization loss is deduced. The model is then extended from the DCT to the integer-based transform as defined in H.264.

On Semi-Blind Source Separation-Based Approaches to Nonlinear Echo Cancellation Based on Bilinear Alternating Optimization - Wang, Xianrui; Yang, Yichen; Brendel, Andreas; Ueda, Tetsuya; Makino, Shoji; Benesty, Jacob; Kellermann, Walter; Chen, Jingdong (IEEE/ACM Transactions on Audio, Speech and Language Processing, 2024)
Acoustic echo cancellation (AEC) is a crucial task in full duplex communications. As conventional linear filtering approaches are ineffective to deal with double-talk, various semi-blind source separation (SBSS)-based AEC algorithms are deceived, most of which are formulated and implemented in the frequency domain based on the multiplicative transfer function (MTF) model for computational efficiency. To avoid large latency and in order to deal with loudspeaker nonlinearities, the convolutive transfer function (CTF) model and odd power series expansion are leveraged, which are employed by numerous SBSS-based nonlinear AEC (SBSS-NAEC) algorithms. Conventional SBSS-NAEC methods estimate the series expansion coefficients and the CTF filter simultaneously making the number of free parameters to estimate large. Hence, the corresponding algorithms are computationally expensive and are difficult to optimize. In this work, we propose to decouple the series expansion coefficients and the CTF filters into a bilinear form and present a bilinear alternating optimization framework for estimating the model parameters. An alternating iterative projection (AIP) algorithm and an alternating element-wise iterative source steering (AEISS) algorithm are proposed. As the bilinear representation consists of less parameters compared to the conventional methods, the proposed algorithms not only improve the AEC performance but also reduce the computational complexity, which is validated by comprehensive simulations and experiments.
CRIS-Team Scopus Importer:2024-06-28
Keywords: alternating optimization; bilinear; convolutive transfer function model; nonlinear acoustic echo cancellation; odd power series expansion; Semi-blind source separation

On Texture Analysis: Local Energy Transforms Versus Quadrature Filters - Aach, T.; Kaup, André; Mester, R. (Signal Processing, 1995)
LMS::1050

On the Analysis and Synthesis of Local Sound Fields for Personal Audio - Bürger, Michael (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2019)
The aim of multizone sound reproduction is to independently synthesize multiple acoustic scenes in different areas of the reproduction environment. Thereby, individual audio contents may be simultaneously provided to multiple persons. In this thesis, a novel algorithm for this purpose is derived, and practical aspects, such as imperfections of the utilized transducer arrays or sensor noise, are investigated. Furthermore, an iterative filter design technique with a spatiotemporal weighting function is presented, which is especially beneficial in the case of reverberant reproduction environments. As high-quality sound reproduction is closely related to active noise control, this work also contributes to the analysis of noise fields evoked by spatially extended sources.
Keywords: sound reproduction; sound synthesis

On the Application of Multivariate Kernel Density Estimation to Image Error Concealment - Koloda, Jan; Peinado, A.M.; Sánchez, V. (IEEE International Conference on Acoustics, Speech and Signal Processing, 2013)
This paper proposes a methodology for the application of multivariate kernel density estimation (KDE) to MMSE-based image/video error concealment (EC). We show that the estimation of the kernel bandwidth matrix for EC must follow a criterion different from that of typical KDE problems. In particular, we propose a bandwidth built as the product of a structure matrix and a scale factor obtained with a minimum square error criterion. We show that our proposal can achieve average PSNR improvements larger than 1 dB with respect to other state-of-the-art technique
LMS::1537
Keywords: kernel estimation; error concealment

On the application of reverberation suppression to robust speech recognition - Maas, Roland; Habets, Emanuël; Sehr, Armin; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2012)
In this paper, we study the effect of the design parameters of a single-channel reverberation suppression algorithm on reverberation-robust speech recognition. At the same time, reverberation compensation at the speech recognizer is investigated. The analysis reveals that it is highly beneficial to attenuate only the reverberation tail after approximately 50 ms while coping with the early reflections and residual late-reverberation by training the recognizer on moderately reverberant data. It will be shown that the overall system at its optimum configuration yields a very promising recognition performance even in strongly reverberant environments. Since the reverberation suppression algorithm is evidenced to significantly reduce the dependency on the training data, it allows for a very efficient training of acoustic models that are suitable for a wide range of reverberation conditions. Finally, experiments with an "ideal" reverberation suppression algorithm are carried out to cross-check the inferred guidelines. © 2012 IEE
Keywords: automatic speech recognition; reverberation robustness; reverberation suppression; Signal enhancement

On the causality problem in time-domain blind source separation and deconvolution algorithms - Aichner, Robert; Buchner, Herbert; Kellermann, Walter (2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05, 2005)
Based on a recently presented generic framework for multichannel blind signal processing for convolutive mixtures we investigate in this paper the problem of incorporating acausal delays which are necessary with certain geometric constellations. Starting from a generic update equation which is applicable to blind source separation (BSS), multichannel blind deconvolution (MCBD), and multichannel blind partial deconvolution (MCBPD) for dereverberation of speech signals, two formulations of the natural gradient are derived. It is shown that one expression is applicable to mere causal filters whereas the other one also allows an implementation of noncausal filters. Moreover, proper initialization methods for both cases are given. For the implementation of the aforementioned algorithms cross-relation estimation techniques known from linear prediction are discussed. Based on these results, relationships between traditional MCBD algorithms can be established. Experimental results of different acoustic scenarios show the applicability of the presented algorithms. © 2005 IEEE.

On the Design of Unitary Filterbanks for the Construction of Orthonormal Wavelets - Brandhuber, Wolfgang (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2009)
<

On the Efficiency of Inter-Layer Prediction in H.264/AVC Quality Scalable Video Coding - Kaup, André; Li, Xiang; Amon, P.; Hutter, A. (IEEE International Symposium on Recent Advances in Communication Engineering, 2008)
LMS::1042

On the estimation of the number of acoustic wideband sources using eigen-beam processing for circular apertures - Teutsch, Heinz; Kellermann, Walter (2005 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, 2005)
This paper addresses the problem of determining the number of wideband sources in an acoustic wave field. By using eigen-beam processing based on circular apertures, this method takes the physics of wave propagation into account. The eigen-beam method decomposes the sound field into eigen-solutions of the acoustic wave equation. This idea is in contrast to existing techniques that are mainly based on examining the eigenvalues of the covariance matrix estimated from measurements of an array of sensors. It is shown that by re-synthesizing these eigen-solutions, a method can be derived that is capable of adaptively estimating the number of acoustic sources in real acoustic environments. © 2005 IEEE.

On the Extrapolation of Room Impulse Responses from Circular Measurements for Data Based Wave Field Synthesis - Kuntz, Achim; Rabenstein, Rudolf (Proc. 34th Annual German Conf. on Acoustics (DAGA), 2008)
LMS::735

On the impact of localization errors on HRTF-based robust least-squares beamforming - Barfuß, Hendrik; Kellermann, Walter (Jahrestagung für Akustik (DAGA), 2016)
In this work, a recently proposed Head-Related Transfer Function HRTF)-based Robust Least-Squares Frequency-Invariant (RLSFI) beamformer design is analyzed with respect to its robustness against localization errors, which lead to a mismatch between the HRTFs corresponding to the actual target source position and the HRTFs which have been used for the beamformer design. The impact of this mismatch on the performance of the HRTF-based RLSFI beamformer is evaluated, including a comparison to the free-field-based beamformer design, using signal-based measures and word error rates for an off-the-shelf speech recognize

On the impact of signal preprocessing for robust distant speech recognition in adverse acoustic environments - Reindl, Klaus; Zheng, Yuanhang; Meier, Stefan; Schwarz, Andreas; Kellermann, Walter (IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), 2012)
In this contribution, a two-channel acoustic front-end for robust automatic speech recognition (ASR) in adverse acoustic environments is analyzed. The source signal extraction scheme combines a blocking matrix based on semi-blind source separation, which provides a continuously updated reference of all undesired components separated from the desired signal and its reflections, and a single-channel Wiener postfilter. The postfilter is directly derived from the obtained noise and interference reference signal and hence, generalizes well-known postfilter realizations. The proposed front-end and its integration into an ASR system are analyzed and evaluated with respect to keyword accuracy under reverberant conditions with unpredictable and nonstationary interferences, and for different target source distances. Evaluating a simplified front-end based on free-field assumptions, an ideal front-end, where knowledge about the true undesired components is assumed, and comparing the proposed scheme with the competitive approach of solely using multistyle training, demonstrates the importance of an adequate signal preprocessing for robust distant speech recognitio
Keywords: Blind source extraction; speech enhancement; robust distant speech recognition

On The Importance of Exact Synchronization for Distributed Audio Signal Processing - Lienhart, R.; Kozintsev, I.; Wehr, Stefan; Yeung, M. (ICASSP 2003 Proceedings, 2003)
LMS::494

On the influence of clipping in lossless predictive and wavelet coding of noisy images - Schnurrer, Wolfgang; Seiler, Jürgen; Schoberl, M.; Kaup, André (29th Picture Coding Symposium, PCS 2012, 2012)
Especially in lossless image coding the obtainable compression ratio strongly depends on the amount of noise included in the data as all noise has to be coded, too. Different approaches exist for lossless image coding. We analyze the compression performance of three kinds of approaches, namely direct entropy, predictive and wavelet-based coding. The results from our theoretical model are compared to simulated results from standard algorithms that base on the three approaches. As long as no clipping occurs with increasing noise more bits are needed for lossless compression. We will show that for very noisy signals it is more advantageous to directly use an entropy coder without advanced preprocessing steps. © 2012 IEE

On the influence of loudspeaker and microphone array geometries on the limits for listening room equalization in a spatial continuum - Schneider, Martin; Kellermann, Walter (AIA-DAGA Conference on Acoustics, 2013)
In audio reproduction, loudspeaker signals are often optimized to be ideally rendered under free-field conditions. In real reverberant listening rooms a listening room equalization (LRE) system may be used to improve to reproduction quality by removing the undesired reverberation. To this end, additional microphones are placed in the listening room and the resulting loudspeakerenclosure-microphone system (LEMS) is then identified to determine equalizing pre-filters for the loudspeaker signals. As the listener position may vary, an equalization in a spatially extended zone is desired as it was already considered in early work on LRE [1]. Later work involved systematically placed loudspeakers and microphones to improve the LRE in a spatially extended zone [2, 3]. It turned out that loudspeaker arrays used for wave field synthesis (WFS) [4] or Higher-Order Ambisonics (HOA) [5] allow for a sufficient control over the reproduced wave field to equalize a spatially extended zone in a corresponding frequency area. Microphone arrays designed for wave field analysis allow for a description of a wave field in an extended zone and are therefore attractive candidates to complement the WFS or HOA loudspeaker arrays in an LRE system. However, a spatially extended equalization can only be achieved for wave field components which may be measured and controlled at the same time. In this contribution, some combinations of differently shaped loudspeaker and microphone arrays are analyzed with respect to their suitability for LRE.

On the limitations of binaural reproduction of monaural blind source separation output signals - Reindl, Klaus; Zhang, Mengqiu; Kellermann, Walter (European Signal Processing Conference (EUSIPCO), 2012)
In this contribution we analyze the binaural reproduction of an acoustic scene from monaural blind source separation (BSS) outputs for headphone applications. In practice, a perfect separation of the individual sources cannot be achieved, there are always residual components in the BSS outputs which constitute a major problem for binaural reproduction of an acoustic scene. We derive a necessary condition for the required signal-to-interference-ratio (SIR) at the BSS outputs so that a binaural reproduction of the source signals is possible up to a given tolerable error based on a simplified free-field model. The theoretical findings are verified by simulations with speech signals. © 2012 EURASI
Keywords: binaural reproduction; headphone rendering; HRTF models; Source separation

On the Performance of the Shape Adaptive DCT in Object-Based Coding of Motion Compensated Difference Images - Kaup, André; Panis, S. (Picture Coding Symposium, ITG-Fachbericht 143, 1997)
LMS::481

On the Robustness and Imperceptibility of Digital Fingerprints - Su, Jonathan; Girod, Bernd (Proc. IEEE Intl. Conf. Multimedia & Computing Systems (ICMCS 99), 1999)
LMS::289

On the Stability of Certain Time-Varying Digital Filters - Czarnach, Roland; Rabenstein, Rudolf; Schüßler, Hans Wilhelm (1983)
LMS::1450

On the Statistical Properties of Reverberant Speech Feature Vector Sequences - Sehr, Armin; Kellermann, Walter (Proc. International Workshop on Acoustic Echo and Noise Control (IWAENC), 2010)
The statistical properties of reverberant logarithmic mel-spectral feature vector sequences that are relevant for acoustic mod-eling in robust speech recognition are analyzed and compared to the corresponding properties of clean-speech feature vector sequences. The investigation focuses on probability densities of feature vector elements and the correlation between feature vectors of different frames. A Monte-Carlo method is used for a quantitative analysis of the density changes and the increase in inter-frame correlation due to reverberation. As an example for the insights that can be obtained from this analysis, the densities and inter-frame correlations of reverberant features are compared to the modeling capabilities of reverberantly-trained and adapted HMMs. Thus, limitations of these approaches can be clearly identifie

On Versatile Video Coding at UHD with Machine-Learning-Based Super-Resolution - Fischer, Kristian; Herglotz, Christian; Kaup, André (12th International Conference on Quality of Multimedia Experience (QoMEX), 2020)
Coding 4K data has become of vital interest in recent years, since the amount of 4K data is significantly increasing. We propose a coding chain with spatial down- and upscaling that combines the next-generation VVC codec with machine learning based single image super-resolution algorithms for 4K. The investigated coding chain, which spatially downscales the 4K data before coding, shows superior quality than the conventional VVC reference software for low bitrate scenarios. Throughout several tests, we find that up to 12 % and 18 % Bjøntegaard delta rate gains can be achieved on average when coding 4K sequences with VVC and QP values above 34 and 42, respectively. Additionally, the investigated scenario with up- and downscaling helps to reduce the loss of details and compression artifacts, as it is shown in a visual example.
Keywords: VVC; video compression; CNN super-resolution; 4K; spatial-resolution scaling; VDSR

Open(G)PIAS: An open-source solution for the construction of a high-precision acoustic startle response setup for tinnitus screening and threshold estimation in rodents - Gerum, Richard; Rahlfs, Hinrich; Streb, Matthias; Krauß, Patrick; Grimm, Jannik; Metzner, Claus; Tziridis, Konstantin; Günther, Michael; Schulze, Holger; Kellermann, Walter; Schilling, Achim (Frontiers in Behavioral Neuroscience, 2019)
The modulation of the acoustic startle reflex (ASR) by a pre-stimulus called pre-pulse inhibition (PPI, for gap of silence pre-stimulus: GPIAS) is a versatile tool to, e.g., estimate hearing thresholds or identify subjective tinnitus percepts in rodents. A proper application of these paradigms depends on a reliable measurement of the ASR amplitudes and an exact stimulus presentation in terms of frequency and intensity. Here, we introduce a novel open-source solution for the construction of a low-cost ASR setup. The complete software for data acquisition and stimulus presentation is written in Python 3.6 and is provided as an Anaconda package. Furthermore, we provide a construction plan for the sensor system based on low-cost hardware components. Exemplary GPIAS data from two animal models (Mus musculus, Meriones unguiculatus) show that the ratio histograms (1-GPIAS) of the gap-pre-stimulus and no pre-stimulus ASR amplitudes can be well described by a log-normal distribution being in good accordance to previous studies with already established setups. Furthermore, it can be shown that the PPI as a function of pre-stimulus intensity (threshold paradigm) can be approximated with a hard-sigmoid function enabling a reproducible sensory threshold estimation. Thus, we show that the open-source solution could help to further establish the ASR method in many laboratories and, thus, facilitate and standardize research in animal models of tinnitus and/or hearing loss.
CRIS-Team Scopus Importer:2019-07-16
Keywords: 3D acceleration sensor; Anaconda package; Animal model; Low-cost setup; Startle; Tinnitus

Open source HEVC analyzer for rapid prototyping (HARP) - Springer, Dominic; Schnurrer, Wolfgang; Weinlich, Andreas; Heindel, Andreas; Seiler, Jürgen; Kaup, André (IEEE International Conference on Image Processing, 2014)
The design of new HEVC extensions comes with the need for careful analysis of internal HEVC codec decisions. Several bitstream analyzers have evolved for this purpose and provide a visualization of encoder decisions as seen from a decoder viewpoint. None of the existing solutions is able to provide actual insight into the encoder and its RDO decision process. With one exception, all solutions are closed source and make adaption of their code to specific implementation needs impossible. Overall, development with the HM code base remains a time-consuming task. Here, we present the HEVC Analyzer for Rapid Prototyping (HARP), which directly addresses the above issues and is freely available under www.lms.lnt.de/HAR

Optical filter structures based on the self-imaging phenomenon - Jahns, J.; Kuppertz, H.; Lohmann, Adolf; Minguez-Vega, G. (ICTON 2004, Wroclaw, Poland, 2004)
LMS::662

Optimal Filter Selection for Multispectral Object Classification Using Fast Binary Search - Sippel, Frank; Seiler, Jürgen; Kaup, André (24th International Workshop on Multimedia Signal Processing, 2022)

Optimality of SCS Watermarking - Bäuml, R.; Tzschoppe, Roman; Kaup, André; Huber, Johannes (Security and Watermarking of Multimedia Contents V, 2003)
Blind spread-spectrum watermarking schemes with correlation based detection algorithms suffer significantly from host signal interference. Improvements are possible when considering the host signal as side-information to the watermark encoder. Costa showed that, in a specific case, host signal interference can be avoided completely. The scalar Costa scheme, which operates on structured codebooks based on scalar quantizers, performs relatively close to the ideal Costa scheme in terms of maximum rate of reliable communication (capacity) over a channel with AWGN attack. The concept of amplitude limited channels in conjunction with repetition of the signal constellation is also known in the communications community by the name modulo channel, which has been studied into great detail. Starting from the modulo channel, which implies a constraint on the amplitude in the receiver, we model the watermarking scenario based on scalar quantizers. The capacity of this model is presented and compared to the theoretical results for SCS watermarking in the case of AWGN attacks. A final analysis of SCS in terms of optimality, based on the results with the modulo channel, shows that even binary SCS is near optimum for one-dimensional (symbol-by-symbol) embedding in terms of capacity for an i.i.d. Gaussian host signal and an appropriate AWGN attack.

Optimization of Probability Distributions for Residual Coding of Screen Content - Och, Hannah; Strutz, Tilo; Kaup, André (2021 IEEE International Conference on Visual Communications and Image Processing (VCIP), 2021)

Optimized and parallelized processing order for improved frequency selective signal extrapolation - Seiler, Jürgen; Kaup, André (19th European Signal Processing Conference, EUSIPCO 2011, 2011)
In the recent years, multi-core processor designs have found their way into many computing devices. To exploit the capabilities of such devices in the best possible way, signal processing algorithms have to be adapted to an operation in parallel tasks. In this contribution an optimized processing order is proposed for Frequency Selective Extrapolation, a powerful signal extrapolation algorithm. Using this optimized order, the extrapolation can be carried out in parallel. The algorithm scales very good, resulting in an acceleration of a factor of up to 7.7 for an eight core computer. Additionally, the optimized processing order aims at reducing the propagation of extrapolation errors over consecutive losses. Thus, in addition to the acceleration, a visually noticeable improvement in quality of up to 0.5 dB PSNR can be achieved. © 2011 EURASI

Optimized Anisotropic Spatial Transforms for Wavelet-Based Scalable Multi-View Video Coding - Garbas, Jens-Uwe; Pesquet-Popescu, Béatrice; Kaup, André (Visual Communications and Image Processing 2009, 2009)
A 4-D wavelet-based transform is used for efficient and scalable compression of multi-view video data. It is composed of a 1-D temporal wavelet transform, namely Motion Compensated Temporal Filtering (MCTF), a 1-D view-directional wavelet transform, namely Disparity Compensated View Filtering (DCVF), and a 2-D spatial transform. The latter is the subject of study in this paper. Whereas usually fixed isotropic wavelet or wavelet packet transforms have been used in the past for the spatial decomposition of the temporal-view- directional highpass bands, we now introduce the usage of adaptive anisotropic wavelet packet transforms as a generalization of wavelet and wavelet packet transforms. An efficient algorithm to adaptively find the rate-distortion optimal joint anisotropic basis for temporal-view-directional multi-view video subbands is derived. It is shown that the adaptive anisotropic transform performs best, compared with conventional wavelet or wavelet packet transforms. © 2009 SPIE-IS&
Keywords: Multi-view; Scalability; Video coding; Wavelet packet transform

Optimized Decoding-Energy-Aware Encoding In Practical VVC Implementations - Kränzler, Matthias; Wieckowski, Adam; Ramasubbu, Geetha; Bross, Benjamin; Kaup, André; Marpe, Detlev; Herglotz, Christian (29th IEEE International Conference on Image Processing (IEEE ICIP), 2022)
The optimization of the energy demand is crucial for modern video codecs. Previous studies show that the energy demand of VVC decoders can be improved by more than 50% if specific coding tools are disabled in the encoder. However, those approaches increase the bit rate by over 20% if the concept is applied to practical encoder implementations such as VVenC. Therefore, in this work, we investigate VVenC and study possibilities to reduce the additional bit rate, while still achieving low-energy decoding at reasonable encoding times. We show that encoding using our proposed coding tool profiles, the decoding energy efficiency is improved by over 25% with a bit rate increase of less than 5% with respect to standard encoding. Furthermore, we propose a second coding tool profile targeting maximum energy savings, which achieves 34% of energy savings at bitrate increases below 15
Keywords: Energy-Efficiency; Optimization; VVC; Decoder; Practical implementation

Optimized Processing Order for 3D Hole Filling in Video Sequences Using Frequency Selective Extrapolation - Seiler, Jürgen; Schöll, Susanne; Schnurrer, Wolfgang; Kaup, André (Picture Coding Symposium (PCS), 2016)
A problem often arising in video communication is the reconstruction of missing or distorted areas in a video sequence. Such holes of unavailable pixels may be caused for example by transmission errors of coded video data or undesired objects like logos. In order to close the holes given neighboring available content, a signal extrapolation has to be performed. The best quality can be achieved, if spatial as well as temporal information is used for the reconstruction. However, the question always is in which order to process the extrapolation to obtain the best result. In this paper, an optimized processing order is introduced for improving the extrapolation quality of Three-dimensional Frequency Selective Extrapolation. Using the proposed optimized order, holes in video sequences can be closed from the outer margin to the center, leading to a higher reconstruction quality, and visually noticeable gains of more than 0.5 dB PSNR are possibl

Optimizing frame structure with real-time computation for interactive multiview video streaming - Gao, Y.; Cheung, G.; Liang, Jie; Kaup, André (2012 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video, 3DTV-CON 2012, 2012)
Interactive multiview video streaming (IMVS) is an application where a network client requests from server a single video view at a time but can periodically switch to other views as the video is played back uninterrupted. Existing IMVS algorithms output pre-computed frame structures that facilitate permissible view-switching while minimizing the expected transmission rate given a storage constraint. In this paper, we use real-time computation (available at a remote powerful server or media cloud) to assist the pre-computed frame structure to satisfy users' view-switch requests. In particular, we first propose a new frame type called uni-merge frame that is computed in real-time for view-switching from one single view to one target view with low transmission rate and reasonable computation cost. Then, to enable permissible view-switches to a particular target picture, we find the optimal combination of pre-computed frames and real-time computed frames - one that minimizes streaming rate subject to both storage and real-time computation constraints - using a greedy combinatorial algorithm. Experimental results show that with real-time computation, the expected streaming rate of the IMVS system can be further decreased by 50% compared to pre-encoded frame structures without real-time computation. © 2012 IEE
Keywords: Multiview videos; optimization; real-time computing; video streaming

Optimizing Rate-Distortion Performance of Motion Compensated Wavelet Lifting with Denoised Prediction and Update - Wokusch, Daniela; Kaup, André (IEEE Int. Workshop on Multimedia Signal Processing (MMSP), 2020)

Optimum Attack on Digital Watermarks and its Defense - Su, Jonathan; Eggers, Joachim; Girod, Bernd (34th Asilomar Conference Signals, Systems, Computers, 2000)
LMS::201

Orthogonality Deficiency Compensation for Improved Frequency Selective Image Extrapolation - Seiler, Jürgen; Meisinger, Katrin; Kaup, André (Picture Coding Symposium, 2007)
This paper describes a very efficient algorithm for image signal extrapolation. It can be used for various applications in image and video communication, e. g. the concealment of data corrupted by transmission errors or prediction in video coding. The extrapolation is performed on a limited number of known samples and extends the signal beyond these samples. Therefore the signal from the known samples is iteratively projected onto different basis functions in order to generate a model of the signal. As the basis functions are not orthogonal with respect to the area of the known samples we propose a new extension, the orthogonality deficiency compensation, to cope with the non-orthogonality. Using this extension, very good extrapolation results for structured as well as for smooth areas are achievable. This algorithm improves PSNR up to 2 dB and gives a better visual quality for concealment of block losses compared to extrapolation algorithms existent so far.©EURAS
LMS::718
Keywords: Signal extrapolation; Error concealment; Prediction; Image processing

Orthogonalized power filters for nonlinear acoustic echo cancellation - Küch, Fabian; Kellermann, Walter (Signal Processing, 2006)
Standard approaches for the cancellation of acoustic echoes in telecommunication systems assume that the echo path to be identified can be modeled by a linear system. However, in applications such as echo cancellation for mobile communication terminals, non-negligible nonlinear distortions are introduced by loudspeakers and their amplifiers, resulting in a significant degradation of the performance of purely linear approaches. In this contribution we consider so-called power filters in order to cope with these kinds of nonlinear echo paths. By combining time-domain orthogonalization methods with a DFT-domain implementation of power filters we derive corresponding quasi-complete orthogonalized versions. As the statistical properties of speech input are non-stationary, the orthogonalization must follow this time-variance, too. Furthermore, we introduce a step-size control for the adaptation of the equivalent orthogonalized structure. Experiments with real hardware show that, with the proposed nonlinear approach, an increase in echo attenuation can be achieved, if the loudspeaker system introduces saturation-type nonlinearities in the echo path. © 2005 Elsevier B.V. All rights reserve
Keywords: Acoustic echo cancellation; Nonlinear adaptive filter; Power filter

Örtliche Fehlerverschleierung von gestört empfangenen Bilddaten durch frequenzselektive Extrapolation - Meisinger, Katrin; Kaup, André (Elektronische Medien: 10. Dortmunder Fernsehseminar, ITG-Fachbericht 179, 2003)
LMS::445

OSLO-IC: On-the-Sphere Learned Omnidirectional Image Compression with Attention Modules and Spatial Context - Wawerek-López, Paul; Mahmoudian Bidgoli, Navid; Fossard, Pascal; Kaup, André; Maugey, Thomas (ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025)
Developing effective 360-degree (spherical) image compression techniques is crucial for technologies like virtual reality and automated driving. This paper advances the state-of-the-art in on-the-sphere learning (OSLO) for omnidirectional image compression framework by proposing spherical attention modules, residual blocks, and a spatial autoregressive context model. These improvements achieve a 23.1% bit rate reduction in terms of WS-PSNR BD rate. Additionally, we introduce a spherical transposed convolution operator for upsampling, which reduces trainable parameters by a factor of four compared to the pixel shuffling used in the OSLO framework, while maintaining similar compression performance. Therefore, in total, our proposed method offers significant rate savings with a smaller architecture and can be applied to any spherical convolutional applicatio
Keywords: 360-degree; spherical transposed convolution; end-to-end compression; image compression

Outlier-robust DFT-domain adaptive filtering for bin-wise stepsize controls, and its application to a generalized sidelobe canceller - Herbordt, Wolfgang; Buchner, Herbert; Nakamura, S.; Kellermann, Walter (Conf. Rec. Intl. Workshop on Acoustic Echo and Noise Control (IWAENC), 2005)
LMS::554

Out-of-Distribution Detection for Monocular Depth Estimation - Hornauer, Julia; Holzbock, Adrian; Belagiannis, Vasileios (Proceedings of the IEEE International Conference on Computer Vision, 2023)
In monocular depth estimation, uncertainty estimation approaches mainly target the data uncertainty introduced by image noise. In contrast to prior work, we address the uncertainty due to lack of knowledge, which is relevant for the detection of data not represented by the training distribution, the so-called out-of-distribution (OOD) data. Motivated by anomaly detection, we propose to detect OOD images from an encoder-decoder depth estimation model based on the reconstruction error. Given the features extracted with the fixed depth encoder, we train an image decoder for image reconstruction using only in-distribution data. Consequently, OOD images result in a high reconstruction error, which we use to distinguish between in- and out-of-distribution samples. We built our experiments on the standard NYU Depth V2 and KITTI benchmarks as in-distribution data. Our post hoc method performs astonishingly well on different models and outperforms existing uncertainty estimation approaches without modifying the trained encoder-decoder depth estimation model.
CRIS-Team Scopus Importer:2024-12-16

Overcomplete BSS for convolutive mixtures based on hierarchical clustering - Winter, Stefan; Sawada, H.; Araki, S.; Makino, Shoji (Proc. ICA -Independent Component Analysis and Blind Source Separation, 2004)
LMS::528

Overview of low-complexity video transcoding from H.263 to H.264 - Bialkowski, Jens; Barkowsky, Marcus; Kaup, André (2006 IEEE International Conference on Multimedia and Expo, ICME 2006, 2006)
With the standardization of H.264/AVC by ITU-T and ISO/IEC and the adaptatation into new hardware, the necessity of transcoding between existing standards and H.264 will arise to achieve interoperability between hardware devices. Because of the many new prediction parameters as well as the pixel-based deblocking filter and the new transform of H.264 this is a difficult task to perform. In our work we propose a fast cascaded pixel-domain transcoder from H.263 to H.264 for both intra- and inter-frame coding. The rate-distortion (RD) performance of the encoded bitstreams is compared to an exhaustive full-search approach. Our approach leads to 9% higher data rate in average, but the computational complexity for the prediction can be reduced by 90% and more. It will be shown that the algorithms proposed for H.263 are applicable for transcoding MPEG-2 to H.264, too. © 2006 IEE

Packet Loss Resilient Internet Video Streaming - Girod, Bernd; Stuhlmüller, Klaus; Link, M.; Horn, Uwe (Visual Communications and Image Processing '99, 1999)
LMS::298

Parallel iterative decoding of Transform Domain Wyner-Ziv video using cross bitplane correlation - van Luong, Huynh; Huang, X.; Forchhammer, S. (18th IEEE International Conference on Image Processing, 2011)
In recent years, Transform Domain Wyner-Ziv (TDWZ) video coding has been proposed as an efficient Distributed Video Coding (DVC) solution, which fully or partly exploits the source statistics at the decoder to reduce the computational burden at the encoder. In this paper, a parallel iterative LDPC decoding scheme is proposed to improve the coding efficiency of TDWZ video codecs. The proposed parallel iterative LDPC decoding scheme is able to utilize cross bitplane correlation during decoding, by iteratively refining the soft-input, updating a modeled noise distribution and thereafter enhancing the bitplane decoding performance. Experimental results show that the proposed scheme reduces the bit rate of Wyner-Ziv frames up to 5.6% and improves the rate-distortion (RD) performance of TDW
LMS::1541
Keywords: Wyner-Ziv; Cross-bitplane correlation; noise distribution

Partial differential equation models for continuous multidimensional systems - Rabenstein, Rudolf; Trautmann, Lutz (Proceedings of the IEEE 2000 Internaitonal Symposium on Circuits and Systems, 2000)
The description of the continuous and discrete multidimensional (MD) models in current use has not yet reached the same state of maturity as for one-dimensional systems. To proceed in that direction, we investigate the connections between certain discrete partial differential equation (PDE) models (Finite-Difference Models, Transfer Function Models, MD Wave Digital Filters). The starting points are potential-flux models that are the standard form of physics-based continuous MD systems. It is shown how certain matrix operations lead to various popular PDE models. The properties of the associated matrices provide the link to the discrete PDE models mentioned above. These investigations are presented in general form along with examples for an electrical transmission line and for acoustic wave propagation.

ParticleAugment: Sampling-based data augmentation - Tsaregorodtsev, Alexander; Belagiannis, Vasileios (Computer Vision and Image Understanding, 2023)
We present an automated data augmentation approach for image classification. The problem is formulated as a Monte Carlo sampling problem where the goal is to approximate the optimal augmentation policies using a policy mixture distribution. We propose a particle filter scheme for the policy search where the probability of applying a set of augmentation operations forms the state of the filter. The policy performance is measured based on the loss function difference between a reference model and the actual model. This performance measure is then used to re-weight the particles and finally update the policy distribution. In our experiments, we show that our formulation for automated augmentation reaches promising results on CIFAR-10, CIFAR-100, and ImageNet datasets using the standard network architectures for this problem. By comparing with the related work, our method reaches a balance between the computational cost of policy search and the model performance. The source code of our approach is publicly available.
CRIS-Team Scopus Importer:2023-02-10
Keywords: Automated augmentation; Deep learning optimization; Particle filters; Sequential Monte Carlo

Partitioned block frequency-domain adaptive second-order Volterra filter - Küch, Fabian; Kellermann, Walter (IEEE Transactions on Signal Processing, 2005)
Finite-length Volterra filters are known to be able to model a wide range of real world nonlinear systems. In this paper, we present an algorithm that allows for different memory lengths of the linear and quadratic Volterra kernel while preserving the advantages of fast convolution techniques in the frequency domain for a second-order Volterra filter. This is achieved by extending partitioned block methods to second-order Volterra filters. To obtain corresponding adaptive realizations of the proposed approach, we present generalizations of known frequency-domain algorithms for Volterra filters and partitioned block frequency-domain adaptive filters for linear systems, respectively. To exploit the advantages of adaptive frequency-domain algorithms with respect to convergence speed, we provide a frequency bin-wise normalization of the step-size parameters. To evaluate the performance of the proposed approach, simulation results are given for the application to nonlinear acoustic echo cancellation. The results confirm the improved convergence of a second-order partitioned block frequency-domain adaptive Volterra filter (PBFDAVF) compared with time-domain adaptation of the kernel coefficients. © 2005 IEEE.
Keywords: Acoustic echo cancellation; Frequency-domain adaptive filtering; Nonlinear systems; Volterra filter

Partitioning of Stereoscopic Sequences by Evaluation of Stereo Disparity and Temporal Change Detection - Aach, T.; Kaup, André (Picture Coding Symposium, Tokyo, 1991)
LMS::1006

Pedestrian Counting Using Deep Models Trained on Synthetically Generated Images - Ghosh, Sanjukta; Amon, Peter; Hutter, Andreas; Kaup, André (International Conference on Computer Vision Theory and Applications (VISAPP), 2017)
Counting pedestrians in surveillance applications is a common scenario. However, it is often challenging to obtain sufficient annotated training data, especially so for creating models using deep learning which require a large amount of training data. To address this problem, this paper explores the possibility of training a deep convolutional neural network (CNN) entirely from synthetically generated images for the purpose of counting pedestrians. Nuances of transfer learning are exploited to train models from a base model trained for image classification. A direct approach and a hierarchical approach are used during training to enhance the capability of the model for counting higher number of pedestrians. The trained models are then tested on natural images of completely different scenes captured by different acquisition systems not experienced by the model during training. Furthermore, the effectiveness of the cross entropy cost function and the squared error cost function are evaluated and analyzed for the scenario where a model is trained entirely using synthetic images. The performance of the trained model for the test images from the target site can be improved by fine-tuning using the image of the background of the target sit
Keywords: Pedestrian Counting; Deep Learning; Convolutional Neural Networks; Synthetic Images; Transfer Learning; Cross Entropy Cost Function; Squared Error Cost Function

Pedestrian Environment Model for Automated Driving - Holzbock, Adrian; Tsaregorodtsev, Alexander; Belagiannis, Vasileios (IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, 2023)
Besides interacting correctly with other vehicles, automated vehicles should also be able to react in a safe manner to vulnerable road users like pedestrians or cyclists. For a safe interaction between pedestrians and automated vehicles, the vehicle must be able to interpret the pedestrian's behavior. Common environment models do not contain information like body poses used to understand the pedestrian's intent. In this work, we propose an environment model that includes the position of the pedestrians as well as their pose information. We only use images from a monocular camera and the vehicle's localization data as input to our pedestrian environment model. We extract the skeletal information with a neural network human pose estimator from the image. Furthermore, we track the skeletons with a simple tracking algorithm based on the Hungarian algorithm and an ego-motion compensation. To obtain the 3D information of the position, we aggregate the data from consecutive frames in conjunction with the vehicle position. We demonstrate our pedestrian environment model on data generated with the CARLA simulator and the nuScenes dataset. Overall, we reach a relative position error of around 16% on both datasets.
CRIS-Team Scopus Importer:2024-12-16

Perceptual evaluation of signal enhancement strategies for hearing aids: a multicenter study (presentation) - Luts, H.; Eneman, K.; Wouters, J.; Büchler, M.; Dillier, N.; Dreschler, W.; Froehlich, M.; Grimm, G.; Harlander, N.; Hohmann, V.; Houben, R.; Jansen, S.; et al. (Int. Hearing Aid Research Conference (IHCON), 2008)
LMS::748

Perceptually motivated spatial and temporal integration of pixel based video quality measures - Barkowsky, Marcus; Eskofier, Björn; Bitto, R.; Bialkowski, Jens; Kaup, André (Mobile Content Quality of Experience, MobConQoE '07, 2007)
In the evaluation of video quality often a full reference approach is used, thus calculating some measure of difference between the reference frames and the distorted frames. Often this measure returns one value per pixel, in the simplest case the squared difference. Conventionally, this pixel based measure is averaged over space and time. This paper introduces a psychophysically derived algorithm for this step. It uses the distribution of the cells in the fovea and the assumption that in a subjective test the part with the highest distortion is most important. Additionally, a temporal integration step is proposed which models the recency and forgiveness effect. Different video quality measures are enhanced with these two steps and their performance is evaluated using the results of a subjective test. Copyright 2007 AC
Keywords: Human visual system; Video quality measurement

Performance analysis of the covariance-whitening and the covariance-subtraction methods for estimating the relative transfer function - Markovich-Golan, Shmulik; Gannot, Sharon; Kellermann, Walter (26th European Signal Processing Conference (EUSIPCO), 2018)

Performance analysis of wave field simulation with the functional transformation method - Petrausch, Stefan; Rabenstein, Rudolf (120th Convention Spring Preprints 2006, 2006)
The application of the Functional Transformation Method (FTM) for the simulation of acoustical wave fields has been recently extended to complex room geometries by the usage of so called "block-based" modeling techniques. The complete model is split into several elementary blocks with simple geometry, which are solved separately with the FTM and reconnected in the discrete system with Wave Digital Filter (WDF) principles. Concerning the performance of this algorithm, two questions arise: how much additional error is introduced compared with an all-in-one FTM solution, and how accurate are the FTM simulations compared with classical methods (e.g. digital waveguide meshes). This paper offers an answer for both questions by proving, that the worst-case scenario of the proposed procedure, i.e. minimum size block models, is identical to a waveguide mesh. The complete derivation is first performed for 1D models and then extended to 2D wave fields, demonstrating the equivalence of minimum-size block-based FTM modeling and digital waveguide meshe

Performance and Comparison of Tracer Gas Methods for Measuring Airflows in Two-Zone Buildings - Heidt, F. D.; Rabenstein, Rudolf; Schepers, G. (ASHRAE Transactions, 1991)
LMS::1595

Performance and Comparison of Tracer Gas Methods for Measuring Airflows in Two-Zone Buildings - Rabenstein, Rudolf; et al. (Indoor Air, 1991)
UnivIS-Import:2017-07-10:Pub.1991.tech.IE.LEN.perfor_7

Performance and Complexity Analysis of Rate Constrained Motion Estimation in MPEG-4 - Kaup, André; Mooshofer, H. (Proceedings Multimedia Systems and Applications II, SPIE, 1999)
LMS::459

Performance evaluation of feature-based image alignment techniques for inter-sequence error concealment - Tröger, Tobias; Hirschbeck, Martin; Kaup, André (2011 14th ITG Conference on Electronic Media Technology, CEMT 2011, 2011)
Mobile reception of digital TV signals is typically error-prone. In multi-broadcast scenarios, inter-sequence error concealment replaces lost image blocks of distorted TV signals with error-free ones of low-resolution reference signals. As the latter are typically delayed, scaled and possibly cropped, however, spatio-temporal image alignment is crucial. Up to now, spatial alignment parameters have been estimated with moderate computational complexity by matching of the widely-used scale-invariant features. In this paper, more recent features from literature are evaluated to further reduce the complexity of the alignment step. Also, one novel feature transform is proposed. Simulations show that the well-known speeded-up robust features can reduce the complexity by 12%. However, the reconstruction quality is lowered due to the misalignment of single frames. The novel transform achieves a speed gain of 11%, but does not decrease the reconstruction quality significantly. © 2011 Informatik Centru
Keywords: Digital TV; error concealment; feature description; image alignment; keypoint detection; model fitting

Performance Evaluation of Multimedia Services over Mobile IP - Ben Younes, Khaled; Spitz, S.; Dulz, W.; Girod, Bernd (6th Workshop on High Speed Networks, 1997)
LMS::428

Performance Evaluation of Wireless Indoor Communication Systems Using Three-Dimensional Ray Tracing - Ben Younes, Khaled; Girod, Bernd (PIERS 97- Progress in Electromagnetic Research Symposium, 1997)
LMS::430

Performance Limitations of Subband Adaptive Filters - Weiß, S.; Stewart, R.W.; Stenger, Alex; Rabenstein, Rudolf (Proc. EUSIPCO 98, 1998)
LMS::341

Performance of a Practical Blind Watermarking Scheme - Eggers, Joachim; Su, Jonathan; Girod, Bernd (Proceedings of SPIE: Security and Watermarking of Multimedia Contents III, 2001)
LMS::130

Performance Prediction and Verification for the Synchronization Link of TanDEM-X - Younis, Marwan; Metzig, Robert; Krieger, Gerhard; Bachmann, Markus; Klein, Rainer (IEEE Geoscience and Remote Sensing Symposium (IGARSS), 2007)
The paper describes the synchronization link of the two satellites of the TanDEM-X mission. The synchronization link is established to track the oscillator phase noise difference which -after appropriate processing- can be used to compensate the effect of oscillator phase noise. A signal model based on the synchronization link hardware is presented which is then used to derive an analytical expression for the compensation phase including effects such as instrument drift, Doppler, antennas, and receiver noise. Specifically the influence of the synchronization link RF hardware on the quality of the derived compensation signal is crucial for the performance. Therefore the synchronization link RF hardware of the TerraSAR-X satellite is characterized to obtain realistic data. These measurements will be used for the calibration of the synchronization link, i.e. to remove systematic errors due to temperature drifts of the instrument. © 2007 IEEE.
DLR Import::2019-04-03

P-Frame Coding with Generalized Difference: A Novel Conditional Coding Approach - Brand, Fabian; Seiler, Jürgen; Kaup, André (2022 IEEE International Conference on Image Processing (ICIP), 2022)
Motion compensated inter frame prediction is a common component of all video coders and greatly reduces temporal redundancy. With the rise of deep learning-based image and video compression, this concept has been successfully taken over from traditional coding approaches. These approaches offer a larger flexibility than traditional transform coding and therefore enable efficient conditional coding. In this work, we develop a novel conditional coding approach based on the generalized difference and generalized sum operators. This approach is a special case of a general conditional coder and has a very small complexity overhead. We also propose an extension which enables dynamic content-adaptive switching between conditional and residual coding. We show that the extended generalized difference coding outperforms both residual and conditional coding, saving 27.8% Bjøntegaard delta rate compared to the former.<br
Keywords: Image coding; Redundancy; Transform coding; Focusing; Switches; Video compression; Complexity theory; Video Compression; Conditional Coding; Autoencoder; Deep Learning

Phase-optimized K-SVD for signal extraction from underdetermined multichannel sparse mixtures - Deleforge, Antoine; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2015)
We propose a novel sparse representation for heavily underdetermined multichannel sound mixtures, i.e., with much more sources than microphones. The proposed approach operates in the complex Fourier domain, thus preserving spatial characteristics carried by phase differences. We derive a generalization of K-SVD which jointly estimates a dictionary capturing both spectral and spatial features, a sparse activation matrix, and all instantaneous source phases from a set of signal examples. This dictionary can be used to extract the learned signal from a new input mixture. The method is applied to the challenging problem of ego-noise reduction for robot audition. We demonstrate its superiority relative to conventional dictionary-based techniques using real-room recording

Photometric limits for digital camera systems - Schöberl, Michael; Brückner, A.; Foessel, S.; Kaup, André (Journal of Electronic Imaging, 2012)
Image sensors for digital cameras are built with ever decreasing pixel sizes. The size of the pixels seems to be limited by technology only. However, there is also a hard theoretical limit for classical video camera systems: During a certain exposure time only a certain number of photons will reach the sensor. The resulting shot noise thus limits the signal-to-noise ratio. In this letter we show that current sensors are already surprisingly close to this limit. © 2012 SPIE and IS&

Physical modeling for spatial sound synthesis - Müller, Alexander; Rabenstein, Rudolf (12th International Conference on Digital Audio Effects, DAFx 2009, 2009)
This contribution combines techniques for sound synthesis and spatial reproduction for a joint synthesis of the sound production and sound propagation properties of virtual string instruments. The generated sound field is reproduced on a massive multichannel loudspeaker system using wave field synthesis techniques. From physical descriptions of string vibrations and sound waves by partial differential equations follows an algorithmic procedure for synthesis-driven wave field reproduction. Its processing steps are derived by mathematical analysis and signal processing principles. Three different building blocks are addressed: The simulation of string vibrations, a model for the radiation pattern of the generated acoustical waves, and the determination of the driving signals for the multichannel loudspeaker array. The proposed method allows the spatial reproduction of synthetic spatial sound without the need for pre-recorded or pre-synthesized source track

Physical Modeling in Sound Synthesis: Vibrating Plates - Schäfer, Maximilian; Werner, Manuel; Rabenstein, Rudolf (26th International Congress on Sound and Vibration (ICSV26), 2019)

Physical modeling of drums by transfer function methods - Trautmann, Lutz; Petrausch, Stefan; Rabenstein, Rudolf (2001 IEEE Interntional Conference on Acoustics, Speech, and Signal Processing, 2001)
Multidimensional (MD) physical systems are usually given in terms of partial differential equations (PDEs). Similar to one-dimensional systems, they can also be described by transfer function models (TFMs). In addition to including initial and boundary conditions as well as excitation functions exactly, the TFM can also be discretized in a simple way. This leads to suitable implementations for digital signal processors. Therefore it is possible to implement physics based digital sound synthesis algorithms derived from TFMs in real-time. This paper extends the recently presented solution for vibrating strings with one spatial dimension to two-dimensional drum models.

Physical Modeling using Recurrent Neural Networks with Fast Convolutional Layers - Parker, Julian; Schlecht, Sebastian J.; Rabenstein, Rudolf; Schäfer, Maximilian (25th International Conference on Digital Audio Effects (DaFX22), 2022)

Physical Modelling of Guitar Strings with Realistic Boundary Conditions - Schäfer, Maximilian; Rabenstein, Rudolf (Jahrestagung für Akustik (DAGA), 2018)
The physical modelling of guitar strings consists of two main tasks: First, the modelling of an interior model describing the oscillation behavior of the string itself and, second, the incorporation of realistic boundary conditions connecting the string to a guitar body. For the modelling of the interior string model the Functional Transformation Method can be used, which turns a physical system described by a set of partial differential equations into a synthesis algorithm with real-time capability. However, the incorporation of realistic boundary conditions may largely increase the complexity of the interior model, therefore it is reasonable to initially implement the string model with simple boundary conditions. In a second step realistic boundary conditions can be realized by the design of a feedback control loop around the interior model. This design procedure leads to a modular real-time algorithm with separate components for strings and for the coupling to the guitar body.

Physikalische Modellierung schwingender Platten - Schäfer, Maximilian; Werner, Manuel; Rabenstein, Rudolf (Jahrestagung für Akustik (DAGA), 2019)

Pixel-based averaging predictor for HEVC lossless coding - Wige, Eugen; Yammine, Gilbert; Amon, P.; Hutter, A.; Kaup, André (2013 20th IEEE International Conference on Image Processing, ICIP 2013, 2013)
This paper presents an intra-frame prediction scheme designed for lossless coding using HEVC. The proposed coding method comprises a pixel-wise prediction based on original samples. It is realized as a separate intra prediction mode, which replaces the PLANAR mode. In order to perform the prediction, a four-sample template around the pixel that is to be predicted is compared to the respective template of a four-pixel neighborhood. For each reference template, the sum of absolute differences (SAD) is determined. A table look-up of the SAD value gives the respective weighting factor for each neighborhood pixel. The predictor for the current pixel is calculated as the weighted average of the neighborhood pixels. In comparison to the unmodified HEVC Test Model HM-9.1 configured for lossless coding by disabling/bypassing transformation, quantization, and in-loop filters, the proposed method provides average bitrate savings up to 10.88% for intra-only coding at similar computational complexity. © 2013 IEE
Keywords: HEVC; lossless compression; pixel-based prediction

Planning and Evaluation of Reorienting Osteotomies of the Proximal Femur in Cases of SCFE Using Virtual 3D Models - Richolt, J.; Teschner, Matthias; Everett, P.; Girod, Bernd; Millis, M.; Kikinis, R. (Second Annual North American Program on Computer Assisted Surgery, 1998)
LMS::339

Planning and Evaluation of Reorienting Osteotomies of the Proximal Femur in Cases of SCFE Using Virtual Three-Dimensional Models - Richolt, J.; Teschner, Matthias; Everett, P.; Girod, Bernd; Millis, M.; Kikinis, R. (First International Conference on Medical Image Computing and Computer-Assisted Intervention MICCAI '98, 1998)
LMS::344

PLL FM Demodulator Performance Under Gaussian Modulation - Hasan, Pavel (IEEE Transactions on Communications, 1998)
LMS::352

Plug and Play for Mobile Users - Stein, P.; Kaup, André (Siemens CIO Journal, 2000)

Polygonal Shape Descriptors - An Efficient Solution for Image Retrieval and Object Localization - Kaup, André; Heuer, J. (34th Asilomar Conference on Signals, Systems, and Computers, 2000)
LMS::467

Postfiltering based on the Coherent-to-Diffuse Power Ratio - Günther, Michael; Brendel, Andreas; Kellermann, Walter (ASA Fall 2018 Meeting, 2018)

Post-Processing for BSS Algorithms to Recover Spatial Cues - Wehr, Stefan; Zourub, Meray; Aichner, Robert; Kellermann, Walter (International Workshop on Acoustic Echo and Noise Control, 2006)
LMS::642

Post-processing for convolutive blind source separation - Aichner, Robert; Zourub, Meray; Buchner, Herbert; Kellermann, Walter (2006 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2006, 2006)
Convolutive blind source separation (BSS) aims at separating point sources from mixtures picked up by several sensors. In real-world environments moving speakers, background noise and long reverberation are encountered which often degrade the performance of BSS algorithms. In such cases, the application of a post-filter can improve the output signal quality by suppression of residual crosstalk and of background noise. In this paper we propose a novel technique to estimate the necessary power spectral densities of the cross-talk components and present a robust system which allows to further suppress both, the remaining interference from point sources and the background noise. Experimental results show the benefit of this post-processing method in realistic environments. © 2006 IEE

Power-Efficient Video Streaming on Mobile Devices Using Optimal Spatial Scaling - Herglotz, Christian; Kaup, André; Coulombe, Stéphane; Vakili, Ahmad (IEEE International Conference on Consumer Electronics (ICCE), 2019)
This paper derives optimal parameter settings for power-efficient wireless video streaming on portable devices. Therefore, a mobile video streaming application is studied, which receives a high-resolution and high-quality video stream from a remote server and displays the content to the end user. We show that the resolution of the input video can be chosen such that the quality-power tradeoff is improved. Making use of a power model from the literature and subjective quality evaluation using a perceptual metric, we derive optimal combinations of the scaling factor and the rate-control parameter of the encoder. In particular, 8% of power can be saved at negligible quality losses and 14% of power can be saved at tolerable distortions.<br
Keywords: video coding; resolution; low-power; smartphone; video streaming; VMAF

Power Modeling for Video Streaming Applications on Mobile Devices - Herglotz, Christian; Coulombe, Stéphane; Vazquez, Carlos; Vakili, Ahmad; Kaup, André; Grenier, Jean-Claude (IEEE Access, 2020)
In this paper, we derive an accurate power model for video streaming which we condense to the essential components contributing the most to the overall power consumption. As a use case, we choose mobile devices on the receiver side performing video streaming in broadcasting or end-to-end scenarios. In modeling, we consider the complete video streaming toolchain, which mainly consists of data acquisition, video processing, display, and audio handling.We compose an overall power model with the help of models from the literature and propose a dedicated feature selection approach to reveal the most important factors related to power consumption. The resulting models achieve mean estimation errors below 7:61%. Results from feature selection indicate that the display brightness, the bitrate, and the frame rate have the highest impact on the power consumption.<br
Keywords: Video coding; modeling; power measurement; mobile communication; streaming; H.264/AVC; HEVC.

Power Modeling for Virtual Reality Video Playback Applications - Herglotz, Christian; Coulombe, Stéphane; Vakili, Ahmad; Kaup, André (IEEE 23rd International Symposium on Consumer Technologies (ISCT), 2019)
This paper proposes a method to evaluate and model the power consumption of modern virtual reality playback and streaming applications on smartphones. Due to the high computational complexity of the virtual reality processing toolchain, the corresponding power consumption is very high, hich reduces operating times of battery-powered devices. To tackle this problem, we analyze the power consumption in detail by performing power measurements. Furthermore, we construct a model to estimate the true power consumption with a mean error of less than 3.5%. The model can be used to save power at critical battery levels by changing the streaming video parameters. Particularly, the results show that the power consumption is significantly reduced by decreasing the input video resolution.<br

Power Reduction Opportunities on End-User Devices in Quality-Steady Video Streaming - Herglotz, Christian; Robitza, Werner; Raake, Alexander; Hoßfeld, Tobias; Kaup, André (15th International Conference on Quality of Multimedia Experience, 2023)

Power Reduction Opportunities on End-User Devices in Quality-Steady Video Streaming - Herglotz, Christian; Robitza, Werner; Raake, Alexander; Hoßfeld, Tobias; Kaup, André (15th International Conference on Quality of Multimedia Experience (QoMEX), 2023)
This paper uses a crowdsourced dataset of online video streaming sessions to investigate opportunities to reduce the power consumption while considering QoE. For this, we base our work on prior studies which model both the end-user’s QoE and the end-user device’s power consumption with the help of high-level video features such as the bitrate, the frame rate, and the resolution. On top of existing research, which focused on reducing the power consumption at the same QoE optimizing video parameters, we investigate potential power savings by other means such as using a different playback device, a different codec, or a predefined maximum quality level. We find that based on the power consumption of the streaming sessions from the crowdsourcing dataset, devices could save more than 55% of power if all participants adhere to low-power settings.<br
Keywords: video; communication; energy; crowdsourcing

Power-Spectrum Condition for Energy-Efficient Watermarking - Su, Jonathan; Girod, Bernd (IEEE International Conf. on Image Processing, 1999)
LMS::301

Precise visibility determination of displays in camera images - Lienhart, R.; Kellermann, Walter; Bouguet, J. Y. (Proceedings of IEEE International Conference on Multimedia and Expo, 2005)
In many novel application scenarios such as smart rooms or sensing rooms visual sensors (such as cameras) need to know which visual actuators (such as displays) are visible to them. Often only parts of a display are visible from a camera. Therefore, a novel algorithm for precise visibility determination is presented. The algorithm makes the assumption that the displays are active, i.e., they can be controlled by the application. Under these conditions the algorithm determines precisely where which parts of a display are imaged by a camera. ©2005 IEEE.
Keywords: Cameras; Image sequences; Signal processing algorithms; Pattern analysis; Frequency estimation; Signal analysis; Computer displays; Multimedia communication; Multimedia computing; Signal processing

Predicting Request Success with Objective Features in German Multimodal Speech Assistants - Weber, Mareike; Halimeh, Mhd Modar; Kellermann, Walter; Popp, Birgit (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2022)
We investigate whether objective features, like occurrence of an error and number of turns, can automatically predict success in interactions with multimodal speech assistants. We used interactions from the SmartKom corpus, a data set on multimodal interactions with virtual assistants in German. In a first step, we segmented the interactions into requests and labeled them as successful or unsuccessful. Afterwards, we defined task success as the average of request success rate. Next, we investigated whether subjective features such as emotions expressed by users show a relation to task success. We find no significant correlation. Finally, we exploited objective features, e.g., number of turns to predict request success. We find that objective features suffice to reach F1 scores over 0.9 (prediction of successful requests) and F0 scores above 0.83 (prediction of unsuccessful requests). Finally, we discuss implications of our findings for automatic evaluation of pragmatic aspects of user experience.
CRIS-Team Scopus Importer:2022-06-10
Keywords: German; Logistic regression; Multimodal; Random forest; Request success; Speech assistant; Task success

Predictive Circuit Protection for DC Grids - Strobl, Christian; Schäfer, Maximilian; Rabenstein, Rudolf (28th International Conference on Electrical Contacts (ICEC), 2016)

Probabilistic modeling for learning-based distance estimation - Brendel, Andreas; Regensky, Andy; Kellermann, Walter (23rd International Congress on Acoustics (ICA), 2019)

Probability Distribution Estimation for Autoregressive Pixel-Predictive Image Coding - Weinlich, Andreas; Amon, Peter; Hutter, Andreas; Kaup, André (IEEE Transactions on Image Processing, 2016)
Pixelwise linear prediction using backward-adaptive least-squares or weighted least-squares estimation of prediction coefficients is currently among the state-of-the-art methods for lossless image compression. While current research is focused on mean intensity prediction of the pixel to be transmitted, best compression requires occurrence probability estimates for all possible intensity values. Apart from common heuristic approaches, we show how prediction error variance estimates can be derived from the (weighted) least-squares training region and how a complete probability distribution can be built based on an autoregressive image model. The analysis of image stationarity properties further allows deriving a novel formula for weight computation in weighted least-squares proofing and generalizing ad hoc equations from the literature. For sparse intensity distributions in non-natural images, a modified image model is presented. Evaluations were done in the newly developed C++ framework volumetric, artificial, and natural image lossless coder (Vanilc), which can compress a wide range of images, including 16-bit medical 3D volumes or multichannel data. A comparison with several of the best available lossless image codecs proofs that the method can achieve very competitive compression ratios. In terms of reproducible research, the source code of Vanilc has been made publi
Keywords: Vanilc; weighted least-squares weighting function; parametric distribution model; variance estimation; 16 bit lossless medical 3D image compression; histogram sparsification

Proc. 17th International Conference on Digital Audio Effects (DAFx) - Disch, Sascha; Edler, Bernd; Herre, Jürgen; Müller, Meinard; Rabenstein, Rudolf; Turowski, Stefan (International Audio Laboratories, 2014)

Proceedings of the 8th International Workshop on Multidimensional Systems - Universität Erlangen-Nürnberg, Erlangen, Germany, September 9-11, 2013 - Rabenstein, Rudolf; Annibale, Paolo (VDE Verlag, 2013)

Processing Energy Modeling for Neural Network Based Image Compression - Herglotz, Christian; Brand, Fabian; Regensky, Andy; Rievel, Felix; Kaup, André (accepted for IEEE International Conference on Image Processing (ICIP), 2023)
Nowadays, the compression performance of neural-networkbased image compression algorithms outperforms state-ofthe- art compression approaches such as JPEG or HEIC-based image compression. Unfortunately, most neural-network based compression methods are executed on GPUs and consume a high amount of energy during execution. Therefore, this paper performs an in-depth analysis on the energy consumption of state-of-the-art neural-network based compression methods on a GPU and show that the energy consumption of compression networks can be estimated using the image size with mean estimation errors of less than 7%. Finally, using a correlation analysis, we find that the number of operations per pixel is the main driving force for energy consumption and deduce that the network layers up to the second downsampling step are consuming most energy.<br
Keywords: video; compression; autoencoder; energy

Proportionate NLMS algorithm for second-order Volterra filters and its application to nonlinear echo cancellation - Küch, Fabian; Kellermann, Walter (Conf. Rec. Intl. Workshop on Acoustic Echo and Noise Control (IWAENC), 2003)
LMS::444

Providing surround sound with loudspeakers: A synopsis of current methods - Blauert, Jens; Rabenstein, Rudolf (Archives of Acoustics, 2012)
Available methods for room-related sound presentation are introduced and evaluated. A focus is put on the synthesis side rather than on complete transmission systems. Different methods are compared using common, though quite general criteria. The methods selected for comparison are: INTENSITY STEREOPHONY after Blumlein, vector-base amplitude panning (VBAP), 5.1-SURROUND and its discrete-channel derivatives, synthesis with spherical harmonics (AMBISONICS, HOA), synthesis based on the boundary method, namely, wave-field synthesis (WFS), and binaural-cue selection methods (e.g., DIRAC). While VBAP, 5.1-SURROUND and other discrete-channel-based methods show a number of practical advantages, they do, in the end, not aim at authentic sound-field reproduction. The so-called holophonic methods that do so, particularly, HOA and WFS, have specific advantages and disadvantages which will be discussed. Yet, both methods are under continuous development, and a decision in favor of one of them should be taken from a strictly application-oriented point of view by considering relevant application-specific advantages and disadvantages in detail. Copyright © 2012 by PAN - IPP
Keywords: Ambisonics; Amplitude panning; Holography; Summing localization; Surround sound; Wavefield synthesis

Public Key Watermarking By Eigenvectors of Linear Transforms - Eggers, Joachim; Su, Jonathan; Girod, Bernd (X European Signal Processing Conference (EUSIPCO 2000), 2000)
LMS::193

Qualitätsoptimierte Transcodierung digitaler Videodatenströme mit geringem Berechnungsaufwand - Bialkowski, Jens (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2009)
Digitale Kommunikationsnetze stellen heute eine immer größere zuverlässige Datenrate für immer mehr Menschen weltweit bereit. Damit entsteht eine ständig steigende Vielfalt an Anwendungsmöglichkeiten, vor allem auch im Bereich der Mobilkommunikation über z.B. mobile Telefone oder PDA. Gerade bei der digitalen Übertragung von Videodaten ist hier ein großer Zuwachs von Jahr zu Jahr deutlich erkennbar, wie Intemet-Videoportale oder Video-Telefone. Für einen Anwender ist es dabei wichtig, transparent, d.h. anwendungs- und formatunabhängig, auf die Daten zugreifen zu können. Diese Transparenz ist nur durch den Einsatz von Transcodem realisierbar, die in dieser Arbeit bezüglich der Ausgabequalität und des Berechnungsaufwandes untersucht werden. Die Arbeit unterteilt sich in zwei wesentliche Abschnitt

Quality Assurance of Weld Seams Using Laser Triangulation Imaging and Deep Neural Networks - Spruck, Andreas; Seiler, Jürgen; Roll, Michael; Dudziak, Thomas; Eckstein, Jürgen; Kaup, André (2020 IEEE International Workshop on Metrology for Industry 4.0 & IoT, 2020)

Quality Enhancement in Mobile Terrestrially Received Analog and Digital Video Signals - Friebe, Markus (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2007)

Quality Improvement of Low Data-rate Compressed Video Signals by Pre- and Postprocessing - Kutka, R.; Kaup, André; Hager, M. (Digital Compression Technologies and Systems for Video Communications, Berlin, 1996)
LMS::1027

Quantization effect on digital watermarks - Eggers, Joachim; Girod, Bernd (Signal Processing, 2001)
LMS::137

Quantization Watermarking - Eggers, Joachim; Girod, Bernd (Proceedings of SPIE: Security and Watermarking of Multimedia Contents II, Electronic Imaging 2000, 2000)
LMS::138

Radio Propagation Prediction in Indoor Environments using Ray Tracing - Ben Younes, Khaled; Fleischmann, C.; Girod, Bernd (Proceedings Conference and Exhibition on Microwaves, Radio Communication and Electromagnetic Compatibility 1997, 1997)
LMS::391

Rate-Constrained Multi-Hypothesis Motion-Compensated Prediction for Video Coding - Flierl, Markus; Wiegand, Thomas; Girod, Bernd (Proc. IEEE International Conference on Image Processing, 2000)
LMS::183

Rate-Distortion-Efficient Video Compression Using a 3-D Head Model - Eisert, Peter; Wiegand, Thomas; Girod, Bernd (IEEE Int. Conf. on Image Processing, 1999)

Rate Distortion Opitmization for Hybrid Video Coding - Li, Xiang (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2009)

Rate-Distortion Optimal Contour Prediction for a Region-Based Coder - Stuhlmüller, Klaus; Girod, Bernd (Proc. IEEE International Conference on Image Processing (ICIP 97), 1997)
LMS::433

Rate-Distortion Optimal Transform Coefficient Selection for Unoccupied Regions in Video-Based Point Cloud Compression - Herglotz, Christian; Genser, Nils; Kaup, André (IEEE Transactions on Circuits and Systems For Video Technology, 2022)
This paper presents a novel method to determine rate-distortion optimized transform coefficients for efficient compression of videos generated from point clouds. The method exploits a generalized frequency selective extrapolation approach that iteratively determines rate-distortion-optimized coefficients for all basis functions of two-dimensional discrete cosine and sine transforms. The method is applied to blocks containing both occupied and unoccupied pixels in video based point cloud compression for HEVC encoding. In the proposed algorithm, only the values of the transform coefficients are changed such that resulting bit streams are compliant to the V-PCC standard. For all-intra coded point clouds, bitrate savings of more than 4% for geometry and more than 6% for texture error metrics with respect to standard encoding can be observed. These savings are more than twice as high as savings obtained using competing methods from literature. In the randomaccess case, our proposed method outperforms competing V-PCC methods by more than 0.5%.<br
Keywords: Coding; HEVC; unoccupied pixels; inactive regions; point cloud compression; V-PCC

Rate-Distortion Optimization for Video Compression - Sullivan, Gary J.; Wiegand, Thomas (IEEE Signal Processing Magazine, 1998)
LMS::333

Rate-distortion optimized frame level rate control for H.264/AVC - Li, Xiang; Oertel, Norbert; Hutter, Andreas; Kaup, André (16th European Signal Processing Conference, EUSIPCO 2008, 2008)
In this paper, a rate-distortion optimized frame level rate control algorithm is presented for H.264/AVC. To improve the performance in both distortion and rate, two techniques are developed. First, an adaptive frame layer rate-distortion optimization technique is included into the rate control module so that the average distortion is decreased. Second, the mapping from the bit budget to the quantization step is well considered at frame level by an accurate rate model. Consequently, a more precise control over the rate is achieved. Simulations show that the proposed rate control outperforms both the coding with fixed quantization step size (Fixed-QP) and the recommended rate control JVT-G012 in the H.264/AVC reference software. Compared with Fixed-QP, a gain up to 1.04 dB is observed and the maximal rate mismatch for seven sequences is only 0.61%. For JVT-G012, an improvement up to 0.42 dB is obtained while the average rate mismatch is reduced by 70%. copyright by EURASI

Rate-Distortion Optimized Learning-Based Image Compression using an Adaptive Hierachical Autoencoder with Conditional Hyperprior - Brand, Fabian; Fischer, Kristian; Kaup, André (IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021)
Deep-learning-based compressive autoencoders consist of a single non-linear function mapping the image to a latent space which is quantized and transmitted. Afterwards, a second non-linear function transforms the received latent space back to a reconstructed image. This method achieves superior quality than many traditional image coders, which is due to a non-linear generalization of linear transforms used in traditional coders. However, modern image and video coder achieve large coding gains by applying rate-distortion optimization on dynamic block-partitioning. In this paper, we present RDONet, a novel approach to achieve similar effects in compression with full image autoencoders by using different hierarchical levels, which are transmitted adaptively after performing an external rate-distortion optimization. Using our model, we are able to save up to 20% rate over comparable non-hierarchical models while maintaining the same quality.<br

Raumklangwiedergabe und der MPEG-4 Standard: Das CARROUSO-Projekt - Rabenstein, Rudolf; Teutsch, Heinz; Buchner, Herbert; Herbordt, Wolfgang; Küch, Fabian; Spors, Sascha; Trautmann, Lutz (22. Tonmeistertagung, 2002)
LMS::443

RDONet: Rate-Distortion Optimized Learned Image Compression with Variable Depth - Brand, Fabian; Fischer, Kristian; Kopte, Alexander; Windsheimer, Marc; Kaup, André (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2022)
Rate-distortion optimization (RDO) is responsible for large gains in image and video compression. While RDO is a standard tool in traditional image and video coding, it is not yet widely used in novel end-to-end trained neural methods. The major reason is that the decoding function is trained once and does not have free parameters. In this paper, we present RDONet, a network containing state-of-the-art components, which is perceptually optimized and capable of rate-distortion optimization. With this network, we are able to outperform VVC Intra on MS-SSIM and two different perceptual LPIPS metrics. This paper is part of the CLIC challenge, where we participate under the team name RDONet FAU.
CRIS-Team Scopus Importer:2022-09-30

RDONet: Rate-Distortion Optimized Learned Image Compression With Variable Depth - Brand, Fabian; Fischer, Kristian; Kopte, Alexander; Windsheimer, Marc; Kaup, André (IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022)
Rate-distortion optimization (RDO) is responsible for large gains in image and video compression. While RDO is a standard tool in traditional image and video coding, it is not yet widely used in novel end-to-end trained neural methods. The major reason is that the decoding function is trained once and does not have free parameters. In this paper, we present RDONet, a network containing state-of-the-art components, which is perceptually optimized and capable of rate-distortion optimization. With this network, we are able to outperform VVC Intra on MS-SSIM and two different perceptual LPIPS metrics. This paper is part of the CLIC challenge, where we participate under the team name RDONet_FAU.

Real-Time Convolutive Blind Source Separation based on a Broadband Approach - Aichner, Robert; Buchner, Herbert; Yan, F.; Kellermann, Walter (Fifth Int. Symposium on Independent Component Analysis and Blind Signal Separation (ICA), 2004)
LMS::508

Real-time dereverberation for deep neural network speech recognition - Schwarz, Andreas; Hümmer, Christian; Maas, Roland; Kellermann, Walter (Jahrestagung für Akustik (DAGA), 2015)
We evaluate a real-time multi-channel dereverberation method for the application to speech recognition with deep neural networks (DNN). The dereverberation method is based on modeling the reverberated signal as a mixture of a fully coherent direct path signal and a diffuse reverberation component, and estimating the coherent-to-diffuse power ratio (CDR) from the spatial coherence of the signals. The method can operate in real-time, i.e., without requiring processing of entire utterances. We compare CDR estimators which are “blind”, i.e., do not require information about the direction of arrival (DOA) of the target signal, with estimators which make use of a DOA estimate. The impact of the dereverberation method on speech recognition accuracy with different DNN-based acoustic models is investigated with the REVERB challenge corpus and the Kaldi speech recognition toolkit.

Real-Time Frequency Selective Reconstruction through Register-Based Argmax Calculation - Regensky, Andy; Grosche, Simon; Seiler, Jürgen; Kaup, André (IEEE International Workshop on Multimedia Signal Processing (MMSP), 2020)
Frequency Selective Reconstruction (FSR) is a state-of-the-art algorithm for solving diverse image reconstruction tasks, where a subset of pixel values in the image is missing. However, it entails a high computational complexity due to its iterative, blockwise procedure to reconstruct the missing pixel values. Although the complexity of FSR can be considerably decreased by performing its computations in the frequency domain, the reconstruction procedure still takes multiple seconds up to multiple minutes depending on the parameterization. However, FSR has the potential for a massive parallelization greatly improving its reconstruction time. In this paper, we introduce a novel highly parallelized formulation of FSR adapted to the capabilities of modern GPUs and propose a considerably accelerated calculation of the inherent argmax calculation. Altogether, we achieve a 100-fold speed-up, which enables the usage of FSR for real-time applications.<br

Real time heart ischemia detection in the smart home care system - Pang, L.; Tchoudovski, I.; Braecklein, M.; Egorouchkina, K.; Kellermann, Walter; Bolz, Armin (2005 27th Annual International Conference of the Engineering in Medicine and Biology Society, IEEE-EMBS 2005, 2005)
In this paper, a novel real-time algorithm for detecting ischemia in the ECG signal is proposed. The goal of this research is to meet the requirements of some smart cardiac home care devices, which can automatically diagnose the ECG and detect the heart risks outside the hospital, especially heart ischemia without symptoms in their early stages. The algorithm is developed based on a real time R peak detector, time domain traditional ECG parameters, the advanced morphologic parameters from Karhunen-Loève transform, and the adaptive neurofuzzy logic classification. Besides, in order to improve the reliability of our algorithm, several significant constraints of the ECG signal are considered. As a result, the ischemia episodes can be detected if the ischemic alteration persists longer than one minute in the ECG signal. © 2005 IEEE.

Realtime Laser Beam Steering and Calibration Method for Coherent Biomedical Distance and Motion Sensing - Schmidt, Marius; Sadeghi, Misha; Rahimi, Farnaz; Eskofier, Björn; Buglagil, Arda; Schmauß, Bernhard; Carlowitz, Christian (CLEO: Science and Innovations, CLEO: S and I 2024 in Proceedings CLEO 2024, Part of Conference on Lasers and Electro-Optics, 2024)
This paper presents a depth camera based 3D position tracking and calibration method for realtime millimeter-accuracy galvanometer laser beam steering that enables high sensitivity coherent optical distance and movement sensing for biomedical applications.
CRIS-Team Scopus Importer:2024-10-04

Real-time motion classification of HD video sequences on embedded systems - Springer, Dominic; Herglotz, Christian; Simmet, F.; Niederkorn, D.; Kaup, André (6th European Embedded Design in Education and Research Conference, EDERC 2014, 2014)
Fast local and global motion estimation (ME) is crucial for a wide variety of different video processing applications, but poses significant requirements on CPU resources. Interestingly, application scenarios shift more and more from traditional PC processing to embedded devices, mostly driven by home automation, machine vision and autonomous navigation. In this paper, we demonstrate how to use DSP capabilities on embedded ARM-based OMAP Systems-On-Chip (SoCs) to provide motion analysis up to HD at speeds of 15-30 fps. We show how to acquire reliable classification of motion types like zoom, rotation or translation, and how to calculate and decompose the so-called homography matrix for further refinement. Our focus lies on navigation sequences from automotive test scenarios, but we also demonstrate the accuracy of the approach on natural video sequence

Real-time moving object detection in video sequences using spatio-temporal adaptive Gaussian mixture models - Quast, Katharina; Obermann, Matthias; Kaup, André (5th International Conference on Computer Vision Theory and Applications, VISAPP 2010, 2010)
In this paper we present a background subtraction method for moving object detection based on Gaussian mixture models which performs in real-time. Our method improves the traditional Gaussian mixture model (GMM) technique in several ways. It takes into account spatial and temporal dependencies, as well as a limitation of the standard deviation leading to a faster update of the model and a smoother object mask. A shadow detection method which is able to remove the umbra as well as the penumbra in one single processing step is further used to get a mask that fits the object outline even better. Using the computational power of parallel computing we further speed up the object detection proces
Keywords: Background modeling; Object detection

Receptor Saturation Modeling for Synaptic DMC - Lotter, Sebastian; Schäfer, Maximilian; Zeitler, Johannes; Schober, Robert (ICC 2021 - IEEE International Conference on Communications, 2021)
Synaptic communication is a natural Molecular Communication (MC) system which may serve as a blueprint for the design of synthetic MC systems. In particular, it features highly specialized mechanisms to enable inter-symbol interference (ISI)-free and energy efficient communication. The understanding of synaptic MC is furthermore critical for disruptive innovations in the context of brain-machine interfaces. However, the physical modeling of synaptic MC is complicated by the possible saturation of the molecular receiver arising from the competition of postsynaptic receptors for neurotransmitters. Saturation renders the system behavior nonlinear and is commonly neglected in existing analytical models. In this work, we propose a novel model for receptor saturation in terms of a nonlinear, state-dependent boundary condition for Fick’s diffusion equation. We solve the resulting boundary-value problem using an eigenfunction expansion of the Laplace operator and the incorporation of the receiver memory as feedback system into the corresponding state-space description. The presented solution is numerically stable and computationally efficient. Furthermore, the proposed model is validated with particle-based stochastic computer simulations.
Keywords: Degradation; Neurotransmitters; Computational modeling; Biological system modeling; Stochastic processes; Receivers; Brain modeling

Reconstruction of Images Taken by a Pair of Non-Regular Sampling Sensors Using Correlation Based Matching - Jonscher, Markus; Seiler, Jürgen; Richter, Thomas; Bätz, Michel; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2014)
Multi-view image acquisition systems with two or more cameras can be rather costly due to the number of high resolution image sensors that are required. Recently, it has been shown that by covering a low resolution sensor with a non-regular sampling mask and by using an efficient algorithm for image reconstruction, a high resolution image can be obtained. In this paper, a stereo image reconstruction setup for multi-view scenarios is proposed. A scene is captured by a pair of non-regular sampling sensors and by incorporating information from the adjacent view, the reconstruction quality can be increased. Compared to a state-of-the-art single-view reconstruction algorithm, this leads to a visually noticeable average gain in PSNR of 0.74 d
Keywords: Resolution Enhancement; Non-Regular Sampling; Multi-View; Signal Extrapolation

Reconstruction of multiview images taken with non-regular sampling sensors - Richter, Thomas; Jonscher, Markus; Seiler, Jürgen; Kaup, André (2014 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2014, 2014)
Increasing spatial image resolution is a widely discussed area in the field of image processing. In this paper, we present an efficient reconstruction approach for high-resolution images, taken with irregularly shielded low-resolution sensors in a multiview setup. The approach is based on the sparsity assumption, meaning that natural images can be efficiently represented in a transform-domain using only few coefficients. Utilizing information from adjacent cameras results in a better reconstruction quality for the central high-resolution view. Since neighboring camera perspectives might differ in illumination, the information from adjacent views has to be adapted to the view to be reconstructed. The simulation results show that a proper incorporation of information from neighboring views leads to a PSNR gain of up to 2.20 dB compared to a state-of-the-art singleview reconstruction approach. © 2014 IEE
Keywords: depth-image based rendering; image reconstruction; Multiview; non-regular sampling; sparsity

Reconstruction of Videos Taken by a Non-Regular Sampling Sensor - Jonscher, Markus; Seiler, Jürgen; Bätz, Michel; Richter, Thomas; Schnurrer, Wolfgang; Kaup, André (IEEE International Conference on Visual Communications and Image Processing (VCIP), 2015)
Recently, it has been shown that a high resolution image can be obtained without the usage of a high resolution sensor. The main idea has been that a low resolution sensor is covered with a non-regular sampling mask followed by a reconstruction of the incomplete high resolution image captured this way. In this paper, a multi-frame reconstruction approach is proposed where a video is taken by a non-regular sampling sensor and fully reconstructed afterwards. By utilizing the temporal correlation between neighboring frames, the reconstruction quality can be further enhanced. Compared to a state-of-the-art single-frame reconstruction approach, this leads to a visually noticeable gain in PSNR of up to 1.19 dB on averag
Keywords: video processing; non-regular sampling; temporal inpainting; signal extrapolation; resolution enhancement

Reconstuction, Enhancement and Efficient Processing of Signals using Appropriate Representations / Verwendung geeigneter Repräsentationen von Signalen zu deren Rekonstruktion, Verbesserung und effizienten Verarbeitung - Seiler, Jürgen (2018)

Recursive Frequency Selective Reconstruction of Non-Regularly Sampled Video Data - Jonscher, Markus; Jaskolka, Karina; Seiler, Jürgen; Kaup, André (Picture Coding Symposium (PCS), 2016)
High resolution images can be acquired using a non-regular sampling sensor which consists of an underlying low resolution sensor that is covered with a non-regular sampling mask. The reconstructed high resolution image is then obtained during post-processing. Recently, it has been shown that the temporal correlation between neighboring frames can be exploited in order to enhance the reconstruction quality of non-regularly sampled video data. In this paper, a new recursive multi-frame reconstruction approach is proposed in order to further increase the reconstruction quality. By using a new reference order, previously reconstructed frames can be used for the subsequent motion estimation and a new weighting function allows for the incorporation of multiple pixels projected onto the same position. With the new recursive multi-frame approach, a visually noticeable average gain in PSNR of up to 1.13 dB with respect to a state-of-the-art single-frame reconstruction approach can be achieved. Compared to the existing multi-frame approach, a gain of 0.31 dB is possible. SSIM results show the same behavior as PSNR results. Additionally, the pre-reconstruction step of the existing multi-frame approach can be avoided and the new algorithm is, in general, capable of real-time processin

Recursive Halfband-Filters - Schüßler, Hans Wilhelm; Steffen, Peter (AEÜ - International Journal of Electronics and Communications, 2001)
LMS::361

Reducing Randomness of Non-Regular Sampling Masks for Image Reconstruction - Jonscher, Markus; Seiler, Jürgen; Richter, Thomas; Kaup, André (IEEE International Conference on Visual Communications and Image Processing (VCIP), 2014)
Increasing spatial image resolution is an often required, yet challenging task in image acquisition. Recently, it has been shown that it is possible to obtain a high resolution image by covering a low resolution sensor with a non-regular sampling mask. Due to the masking, however, some pixel information in the resulting high resolution image is not available and has to be reconstructed by an efficient image reconstruction algorithm in order to get a fully reconstructed high resolution image. In this paper, the influence of different sampling masks with a reduced randomness of the non-regularity on the image reconstruction process is evaluated. Simulation results show that it is sufficient to use sampling masks that are non-regular only on a smaller scale. These sampling masks lead to a visually noticeable gain in PSNR compared to arbitrary chosen sampling masks which are non-regular over the whole image sensor size. At the same time, they simplify the manufacturing process and allow for efficient storag
Keywords: Image Reconstruction; Image Sensor; Non-Regular Sampling; Resolution Enhancement; Signal Extrapolation

Reduction of Block Artifacts by Selective Removal and Reconstruction of the Block Borders - Panis, S.; Kutka, R.; Kaup, André (Picture Coding Symposium, ITG-Fachbericht 143, 1997)
LMS::480

Reduction of Ringing Noise in Transform Image Coding Using a Simple Adaptive Filter - Kaup, André (Electronics Letters, 1998)
LMS::473

Re-estimation of Motion and Reconstruction for Distributed Video Coding - van Luong, Huynh; Raket, L. L.; Forchhammer, S. (IEEE Transactions on Image Processing, 2014)
Transform domain Wyner-Ziv (TDWZ) video coding is an efficient approach to distributed video coding (DVC), which provides low complexity encoding by exploiting the source statistics at the decoder side. The DVC coding efficiency depends mainly on side information and noise modeling. This paper proposes a motion re-estimation technique based on optical flow to improve side information and noise residual frames by taking partially decoded information into account. To improve noise modeling, a noise residual motion re-estimation technique is proposed. Residual motion compensation with motion updating is used to estimate a current residue based on previously decoded frames and correlation between estimated side information frames. In addition, a generalized reconstruction algorithm to optimize a multihypothesis reconstruction is proposed. The proposed techniques using motion and reconstruction re-estimation (MORE) are integrated in the SING TDWZ codec, which uses side information and noise learning. For Wyner-Ziv frames using GOP size 2, the MORE codec significantly improves the TDWZ coding efficiency with an average (Bjøntegaard) PSNR improvement of 2.5 dB and up to 6 dB improvement compared with DISCOVE
LMS::1538
Keywords: distributed video coding; optical flow; motion reestimation; residual motion compensation; generalized reconstruction

Refined Fault Detection in LVDC-Grids with Signal Processing, System Identification and Machine Learning Methods - Strobl, Christian; Ott, Leopold; Kaiser, Julian; Gosses, Kilian; Schäfer, Maximilian; Rabenstein, Rudolf (29th International Conference on Electrical Contacts, Together with 64th IEEE Holm Conference on Electrical Contacts, 2018)
This paper considers particle propagation in a cylindrical molecular communication channel, e.g. a simplified model of a blood vessel. Emitted particles are influenced by diffusion, flow, and a vertical force induced e.g. by gravity or magnetism. The dynamics of the diffusion process are modeled by multi-dimensional transfer functions in a spatio-temporal frequency domain. Realistic boundary conditions are incorporated by the design of a feedback loop. The result is a discrete-time semi-analytical model for the particle concentration. The model is validated by comparison to particle-based simulations. These numerical experiments reveal that the particle concentration of the proposed semi-analytical model and the particle-based model are in excellent agreement. The analytical form of the proposed solution provides several benefits over purely numerical models, e.g. high variability, existence of low run-time algorithms, extendability to several kinds of boundary conditions, and analytical connection to parameters from communication theory.<br

Reflection coefficient estimation by pseudospectrum matching - Marković, D.; Hofmann, Christian; Antonacci, F.; Kowalczyk, Konrad; Sarti, A.; Kellermann, Walter (International Workshop on Acoustic Signal Enhancement (IWAENC), 2012)
This paper presents a novel model-based method for in-situ estimation of sound reflection coefficients in acoustic enclosures. The method uses a modeling engine for simulation of acoustic propagation and generation of the spatial pseudospectrum using a beamforming technique. The simulated pseudospectrum is matched with the measured pseudospectrum acquired by a real beamforming microphone array in order to estimate reflection coefficients of the walls of the acoustic enclosure. Experimental results confirm that the proposed method allows for an accurate estimation of the reflection coefficients, especially for reflective walls, typical of everyday environment
Keywords: Beamforming; Microphone array; Reflection coefficient estimation

Region-Based Image Coding Using Functional Approximation - Kaup, André; Aach, T. (Picture Coding Symposium, Lausanne, 1993)
LMS::1018

Region-of-Interest Prioritised Sampling for Constrained Autonomous Exploration Systems - Bhattacharjee, Protim; Burger, Martin; Boerner, Anko; Morgenshtern, Veniamin (IEEE Transactions on Computational Imaging, 2022)
Goal oriented autonomous operation of space rovers has been known to increase scientific output of a mission. In this work we present an algorithm, called the RoI Prioritised Sampling (RPS), that prioritises Region-of-Interests (RoIs) in an exploration scenario in order to utilise the limited resources of the imaging instrument on the rover effectively. This prioritisation is based on an estimator that evaluates the change in information content at consecutive spatial scales of the RoIs without calculating the finer scale reconstruction. The estimator, called the Refinement Indicator (RI), is motivated and derived. Multi-scale acquisition approaches, based on classical and multi-level compressed sensing, with respect to the single pixel camera architecture are discussed. The performance of the algorithm is verified on remote sensing images and compared with the state-of-the-art multi-resolution reconstruction algorithms. At the considered sub-sampling rates the RPS is shown to better utilise the system resources for reconstructing the RoIs.
CRIS-Team Scopus Importer:2022-04-15
Keywords: Autonomous systems; Cameras; Compressed sensing; compressed sensing; Detectors; image acquisition; Image reconstruction; Imaging; space exploration; Spatial resolution; spatial resolution; Transforms

Reinforcement Learning-based Microphone Selection in Wireless Acoustic Sensor Networks considering Network and Acoustic Utilities - Afifi, Haitham; Günther, Michael; Brendel, Andreas; Karl, Holger; Kellermann, Walter (14th ITG Conference on Speech Communication, 2021)

Relation between blind system identification and convolutive blind source separation - Buchner, Herbert; Aichner, Robert; Kellermann, Walter (Conf. Rec. Joint Workshop for Hands-Free Speech Communication and Microphone Arrays (HSCMA), 2005)
LMS::538

Relative impulse response estimation during doubletalk with an artificial neural network-based step size control - Meier, Stefan; Kellermann, Walter (International Workshop on Acoustic Signal Enhancement (IWAENC), 2016)
The Normalized Least-Mean Squares (NLMS) algorithm is a widely used method for linear system identification (e.g., for Acoustic Echo Cancellation (AEC), where the acoustic path between loudspeaker and microphone needs to be estimated). As soon as interferers or background noise are active, step size control becomes a crucial task in order to ensure a fast but stable adaptation. Conventional step size control methods address the case of additive noise contaminating the system output, i.e., microphone signal. When the NLMS algorithm is used for Relative Impulse Response (RIR) estimation, however, both the input signal and the output signal of the unknown system are noisy (since both of them are microphone signals) and a different step size control is required. In this paper, the derivation of a new step size is presented. Since the resulting step size cannot be determined directly, an Artificial Neural Network (ANN)-based estimation of the step size is proposed and its effectiveness is demonstrate
Keywords: System Identification; Step Size Adaptation; Artificial Neural Networks

Reliability-based Mesh-to-Grid Image Reconstruction - Koloda, Jan; Seiler, Jürgen; Kaup, André (IEEE Workshop on Multimedia Signal Processing (MMSP), 2016)
This paper presents a novel method for the reconstruction of images from samples located at non-integer positions, called mesh. This is a common scenario for many image processing applications, such as super-resolution, warping or virtual view generation in multi-camera systems. The proposed method relies on a set of initial estimates that are later refined by a new reliability-based content-adaptive framework that employs denoising in order to reduce the reconstruction error. The reliability of the initial estimate is computed so stronger denoising is applied to less reliable estimates. The proposed technique can improve the reconstruction quality by more than 2 dB (in terms of PSNR) with respect to the initial estimate and it outperforms the state-of-the-art denoising-based refinement by up to 0.7 d

Reliability Scoring for the Recognition of Degraded License Plates - Maier, Anatol; Moussa, Denise; Spruck, Andreas; Seiler, Jürgen; Rieß, Christian (2022 18th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2022)
Criminal investigations oftentimes need the identification of license plates of escape vehicles. The vehicles may be recorded by low-quality cameras in the wild. Their license plates may be unreadable for police officers. Recent efforts aim to use machine learning to forensically decipher license plates from such low-quality images. These methods operate near the information-theoretic limit of recognition and hence show quite high error rates. Unfortunately, it is unclear when such prediction errors occur, which makes it difficult to use these methods in practice. In this work, we propose a Bayesian Neural Network to inherently incorporate a reliability measure into the classifier. We additionally propose to integrate multiple estimations with an entropy weight to further improve the reliability. Our experiments show that this uncertainty metric dramatically reduces the number of false predictions while preserving most of the true predictions.<br

Reliable Pedestrian Detection Using A Deep Neural Network Trained On Pedestrian Counts - Ghosh, Sanjukta; Amon, Peter; Hutter, Andreas; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2017)
Pedestrian detection is an important task for applications like surveillance, driver assistance systems and autonomous driving. We present a novel approach for detecting pedestrians using a deep convolutional neural network (CNN) trained for counting pedestrians. Our method avoids the need for annotation of the position of the pedestrians in the training data via bounding boxes. The deconvolved outputs of the filters of the trained counting model are used to detect the pedestrians. The average miss rate values on the tested datasets were found to be in the same range as other methods in spite of a simpler training using only pedestrian counts. This method is found to be suitable for detecting pedestrians in crowded scenes with occlusion as well as less crowded scene
Keywords: Pedestrian Detection; Deep Learning; CNN; Counting Model; Deconvolution

Representation of deformable motion for compression of dynamic cardiac image data - Weinlich, Andreas; Amon, P.; Hutter, A.; Kaup, André (Proceedings of SPIE - The International Society for Optical Engineering 8314, 2012)
We present a new approach for efficient estimation and storage of tissue deformation in dynamic medical image data like 3-D + t computed tomography reconstructions of human heart acquisitions. Tissue deformation between two points in time can be described by means of a displacement vector field indicating for each voxel of a slice, from which position in the previous slice at a fixed position in the third dimension it has moved to this position. Our deformation model represents the motion in a compact manner using a down-sampled potential function of the displacement vector field. This function is obtained by a Gauss-Newton minimization of the estimation error image, i. e., the difference between the current and the deformed previous slice. For lossless or lossy compression of volume slices, the potential function and the error image can afterwards be coded separately. By assuming deformations instead of translational motion, a subsequent coding algorithm using this method will achieve better compression ratios for medical volume data than with conventional block-based motion compensation known from video coding. Due to the smooth prediction without block artifacts, particularly whole-image transforms like wavelet decomposition as well as intra-slice prediction methods can benefit from this approach. We show that with discrete cosine as well as with Karhunen-Loève transform the method can achieve a better energy compaction of the error image than block-based motion compensation while reaching approximately the same prediction error energy. © 2012 SPI

Representation of two-dimensional wave fields by multidimensional signals - Rabenstein, Rudolf; Steffen, Peter; Spors, Sascha (Signal Processing, 2006)
The representation of information by waves is common to many fields in engineering and physics like electromagnetism, acoustics, and optics. Many different tools for the description of wave-like signals have been developed. Most of them are based on methods and results from mathematical physics with different viewpoints according to the specific application areas. This paper presents a framework for some of the most common of these representations. It is based on well-known transform domain signal descriptions from linear systems theory. © 2005 Elsevier B.V. All rights reserve
Keywords: Acoustics; Functional transformations; Multidimensional systems; Waves