Resampling and Reconstructing Corrupted Images for Image Classification - Heimann, Viktoria; Spruck, Andreas; Kaup, André (IEEE International Workshop on Multimedia Signal Processing, 2022)

Resampling Images to a Regular Grid from a Non-Regular Subset of Pixel Positions Using Frequency Selective Reconstruction - Seiler, Jürgen; Jonscher, Markus; Schöberl, Michael; Kaup, André (IEEE Transactions on Image Processing, 2015)
Even though image signals are typically defined on a regular 2D grid, there also exist many scenarios where this is not the case and the amplitude of the image signal only is available for a non-regular subset of pixel positions. In such a case, a resampling of the image to a regular grid has to be carried out. This is necessary since almost all algorithms and technologies for processing, transmitting or displaying image signals rely on the samples being available on a regular grid. Thus, it is of great importance to reconstruct the image on this regular grid, so that the reconstruction comes closest to the case that the signal has been originally acquired on the regular grid. In this paper, Frequency Selective Reconstruction is introduced for solving this challenging task. This algorithm reconstructs image signals by exploiting the property that small areas of images can be represented sparsely in the Fourier domain. By further considering the basic properties of the optical transfer function of imaging systems, a sparse model of the signal is iteratively generated. In doing so, the proposed algorithm is able to achieve a very high reconstruction quality, in terms of peak signal-to-noise ratio (PSNR) and structural similarity measure as well as in terms of visual quality. The simulation results show that the proposed algorithm is able to outperform state-of-the-art reconstruction algorithms and gains of more than 1 dB PSNR are possibl
UnivIS-Import:2015-10-26:Pub.2015.tech.IE.LEN.resamp
Keywords: Image reconstruction; interpolation; non-regular sampling

RESET: Revisiting Trajectory Sets for Conditional Behavior Prediction - Schmidt, Julian; Huissel, Pascal; Wiederer, Julian; Jordan, Julian; Belagiannis, Vasileios; Dietmayer, Klaus (IEEE Intelligent Vehicles Symposium, Proceedings, 2023)
It is desirable to predict the behavior of traffic participants conditioned on different planned trajectories of the autonomous vehicle. This allows the downstream planner to estimate the impact of its decisions. Recent approaches for conditional behavior prediction rely on a regression decoder, meaning that coordinates or polynomial coefficients are regressed. In this work we revisit set-based trajectory prediction, where the probability of each trajectory in a predefined trajectory set is determined by a classification model, and first-time employ it to the task of conditional behavior prediction. We propose RESET, which combines a new metric-driven algorithm for trajectory set generation with a graph-based encoder. For unconditional prediction, RESET achieves comparable performance to a regression-based approach. Due to the nature of set-based approaches, it has the advantageous property of being able to predict a flexible number of trajectories without influencing runtime or complexity. For conditional prediction, RESET achieves reasonable results with late fusion of the planned trajectory, which was not observed for regression-based approaches before. This means that RESET is computationally lightweight to combine with a planner that proposes multiple future plans of the autonomous vehicle, as large parts of the forward pass can be reused.
CRIS-Team Scopus Importer:2023-08-25
Keywords: AI-Based Methods; Behavior-Based Systems; Motion and Path Planning

Residual Cross-talk and Noise Suppression for Convolutive Blind Source Separation - Aichner, Robert; Zourub, Meray; Buchner, Herbert; Kellermann, Walter (Proc. 32nd Annual German Conf. on Acoustics (DAGA), 2006)
LMS::591

Residual Pattern Learning for Pixel-wise Out-of-Distribution Detection in Semantic Segmentation - Liu, Yuyuan; Ding, Choubo; Tian, Yu; Pang, Guansong; Belagiannis, Vasileios; Reid, Ian; Carneiro, Gustavo (2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023)
Semantic segmentation models classify pixels into a set of known ("in-distribution") visual classes. When deployed in an open world, the reliability of these models depends on their ability to not only classify in-distribution pixels but also to detect out-of-distribution (OoD) pixels. Historically, the poor OoD detection performance of these models has motivated the design of methods based on model re-training using synthetic training images that include OoD visual objects. Although successful, these re-trained methods have two issues: 1) their in-distribution segmentation accuracy may drop during re-training, and 2) their OoD detection accuracy does not generalise well to new contexts outside the training set (e.g., from city to country context). In this paper, we mitigate these issues with: (i) a new residual pattern learning (RPL) module that assists the segmentation model to detect OoD pixels with minimal deterioration to inlier segmentation accuracy; and (ii) a novel context-robust contrastive learning (CoroCL) that enforces RPL to robustly detect OoD pixels in various contexts. Our approach improves by around 10% FPR and 7% AuPRC previous state-of-the-art in Fishyscapes, Segment-Me-If-You-Can, and RoadAnomaly datasets.
CRIS-Team Scopus Importer:2024-12-16

Resonant Electric Arcs in DC Microgrids with Low System Impedance in the VLF-Band - Strobl, Christian; Ott, Leopold; Kaiser, Julian; Streck, Matthias; Nothnagel, Frank; Berger, Frank; Schäfer, Maximilian; Rabenstein, Rudolf (IEEE Int. Conf. on DC Microgrids, 2017)
Various fault scenarios have been analyzed by running a number of differently combined 380 VDC microgrid tests. These tests represented a common grid topology with low system impedance at grid resonance points within the single- or lower double-digit kHz range. At serial arc faults, self-excited resonant modes of the arc plasma column have been observed. They lead to an increased arc column stability compared to non-resonant arcs with colored noise behavior. These characteristics require a special focus on pattern recognition methods for arc fault sensors along with extended suitability tests for mechanical and hybrid switchgear concerning the altered stability of switching arcs. The use of small-signal models for system components such as source and load converters as well as for arcs with regard to large-signal DC operating points and converter control modes is helpful in order to describe the reaction of the system in the event of a malfunction. This is essential for the development of suitable protective components and algorithm
LMS::1718

Resource allocation for distributed blind source separation - Bachmann, Markus; Brendel, Andreas; Kellermann, Walter (ITG Conference on Speech Communication, 2018)
The real-time capability of the TRINICON (TRIple-N- Independent component analysis for CONvolutive mix- tures) framework for Blind Source Separation (BSS) is not guaranteed for devices of low computational power like the ones typically used in sensor networks. In order to ensure real-time capability in this case, as many algorith- mic parts as possible are distributed on the available nodes. This should lead to an optimal resource exploitation for a given network model which accounts for the data rates and latencies of the links between the nodes and for the compu- tational power of the nodes themselves. The simulation re- sults for homogeneous and heterogeneous networks show that TRINICON-BSS is still possible even if the adaptation exploits only parts of the originally intended signal.<br

Retina model inspired image quality assessment - Zhai, Guangtao; Kaup, André; Wang, J.; Yang, X. (2013 IEEE International Conference on Visual Communications and Image Processing, IEEE VCIP 2013, 2013)
We proposed in this paper a retina model based approach for image quality assessment. The retinal model is consisted of an optical modulation transfer module and an adaptive low-pass filtering module. We treat the model as a black box and design the adaptive filer using an information theoretical approach. Since the information rate of visual signals is far beyond the processing power of the human visual system, there must be an effective data reduction stage in human visual brain. Therefore, the underlying assumption for the retina model is that the retina reduces the data amount of the visual scene while retaining as much useful information as possible. For full reference image quality assessment, the original and distorted images pass through the retinal filter before some kind of distance is calculated between the images. Retina filtering can serve as a general preprocessing stage for most existing image quality metrics. We show in this paper that retina model based MSE/PSNR, though being straightforward, has already state of the art performance on several image quality databases. © 2013 IEE
Keywords: Human Visual System; Image Quality assessment; Mean Squared Error; Peak Signal to Noise Ratio; Retinal Model

Retrieval of individualized head-related transfer functions for hearing aid applications - Bürger, Michael; Meier, Stefan; Hofmann, Christian; Fischer, Eghart; Puder, Henning; Kellermann, Walter (25th European Signal Processing Conference (EUSIPCO), 2017)
The capability of modern hearing aids to provide hearing-impaired humans with enhanced signals, which ultimately leads to an increased speech intelligibility, may benefit from fitting the device for each subject individually. This ideally also involves the exploitation of Head-Related Impulse Responses (HRIRs). However, HRIRs vary from person to person and thus require tedious measurements for each individual. In this work, we investigate two approaches which aim at speeding up the HRIR acquisition procedure. These are continuous measurements and interpolation, where Dynamic Time Warping (DTW) as well as linear interpolation of the magnitude and phase responses are considered. In contrast to related publications, the continuous HRIR measurements are not performed in anechoic environments here. The quality of the obtained HRIRs is assessed by means of the system mismatch and the proposed error of relative transfer functions. Both measures reveal that continuous HRIR measurements are on average much more capable than the investigated interpolation approaches, and they furthermore provide a more uniform performance for different source direction

Reusing the H.264/AVC deblocking filter for efficient spatio-temporal prediction in video coding - Seiler, Jürgen; Kaup, André (36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 2011)
The prediction step is a very important part of hybrid video codecs for effectively compressing video sequences. While existing video codecs predict either in temporal or in spatial direction only, the compression efficiency can be increased by a combined spatio-temporal prediction. In this paper we propose an algorithm for reusing the H.264/AVC deblocking filter for spatio-temporal prediction. Reusing this highly optimized filter allows for a very low computational complexity of this prediction mode and an average rate reduction of up to 7.2% can be achieved. © 2011 IEE
Keywords: Deblocking; Prediction; Video coding

Reverberation model-based decoding in the logmelspec domain for robust distant-talking speech recognition - Sehr, Armin; Maas, Roland; Kellermann, Walter (IEEE Transactions on Audio Speech and Language Processing, 2010)
The REMOS (REverberation MOdeling for Speech recognition) concept for reverberation-robust distant-talking speech recognition, introduced in Distant-talking continuous speech recognition based on a novel reverberation model in the feature domain (A. Sehr , in Proc. Interspeech, 2006, pp. 769772) for melspectral features, is extended to logarithmic melspectral (logmelspec) features in this contribution. Thus, the favorable properties of REMOS, including its high flexibility with respect to changing reverberation conditions, become available in the more competitive logmelspec domain. Based on a combined acoustic model consisting of a hidden Markov model (HMM) network and a reverberation model (RM), REMOS determines clean-speech and reverberation estimates during recognition. Therefore, in each iteration of a modified Viterbi algorithm, an inner optimization operation maximizes the joint density of the current HMM output and the RM output subject to the constraint that their combination is equal to the current reverberant observation. Since the combination operation in the logmelspec domain is nonlinear, numerical methods appear necessary for solving the constrained inner optimization problem. A novel reformulation of the constraint, which allows for an efficient solution by nonlinear optimization algorithms, is derived in this paper so that a practicable implementation of REMOS for logmelspec features becomes possible. An in-depth analysis of this REMOS implementation investigates the statistical properties of its reverberation estimates and thus derives possibilities for further improving the performance of REMOS. Connected digit recognition experiments show that the proposed REMOS version in the logmelspec domain significantly outperforms the melspec version. While the proposed RMs with parameters estimated by straightforward training for a given room are robust to a mismatch of the speakermicrophone distance, their performance significantly decreases if they are used in a room with substantially different conditions. However, by training multi-style RMs with data from several rooms, good performance can be achieved across different rooms. © 2010 IEE
Keywords: Acoustic modeling; distant-talking automatic speech recognition (ASR); model-based dereverberation; reverberation model; robust ASR

Reverberation Modeling for Robust Distant-Talking Speech Recognition - Sehr, Armin (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2010)

Reverberation Modeling for Robust Speech Recognition - Maas, Roland; Sehr, Armin; Kellermann, Walter (Deutsche Jahrestagung für Akustik (DAGA), 2011)
The REMOS (REverberation MOdeling for Speech recognition) concept for reverberation-robust distanttalking speech recognition [1] is presented in this paper. REMOS extends a conventional hidden Markov model (HMM) trained on close-talking data with a reverberation model describing the acoustical environment. The combination of both models is performed during recognition to match the reverberant observation. Since varying acoustic conditions only require a reestimation of the reverberation model, REMOS is significantly more flexible than recognition systems trained on reverberant dat

Revisiting Gradient-Based Uncertainty for Monocular Depth Estimation - Hornauer, Julia; El-Ghoussani, Amir; Belagiannis, Vasileios (IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025)
Monocular depth estimation, similar to other image-based tasks, is prone to erroneous predictions due to ambiguities in the image, for example, caused by dynamic objects or shadows. For this reason, pixel-wise uncertainty assessment is required for safety-critical applications to highlight the areas where the prediction is unreliable. We address this in a post hoc manner and introduce gradient-based uncertainty estimation for already trained depth estimation models. To extract gradients without depending on the ground truth depth, we introduce an auxiliary loss function based on the consistency of the predicted depth and a reference depth. The reference depth, which acts as pseudo ground truth, is in fact generated using a simple image or feature augmentation, making our approach simple and effective. To obtain the final uncertainty score, the derivatives w.r.t. the feature maps from single or multiple layers are calculated using back-propagation. We demonstrate that our gradient-based approach is effective in determining the uncertainty without re-training using the two standard depth estimation benchmarks KITTI and NYU. In particular, for models trained with monocular sequences and therefore most prone to uncertainty, our method outperforms related approaches. In addition, we publicly provide our code and models: https://github.com/jhornauer/GrUMoDepth.
CRIS-Team Scopus Importer:2025-02-28
Keywords: Depth estimation; training-free; uncertainty estimation

RFVTM: A Recovery and Filtering Vertex Trichotomy Matching for Remote Sensing Image Registration - Zhao, Ming; An, Bowen; Wu, Yongpeng; van Luong, Huynh; Kaup, André (IEEE Transactions on Geoscience and Remote Sensing, 2017)
Reliable feature point matching is a vital yet challenging process in feature-based image registration. In this paper, a robust feature point matching algorithm, which is called recovery and filtering vertex trichotomy matching, is proposed to remove outliers and retain sufficient inliers for remote sensing images. A novel affine-invariant descriptor, which is called the vertex trichotomy descriptor, is proposed on the basis of that geometrical relations between any of vertices and lines are preserved after affine transformations, which is constructed by mapping each vertex into trichotomy sets. The outlier removals in vertex trichotomy matching (VTM) are implemented by iteratively comparing the disparity of the corresponding vertex trichotomy descriptors. Some inliers mistakenly validated by a large number of outliers are removed in VTM iterations, and several residual outliers that are close to the correct locations cannot be excluded with the same graph structures. Therefore, a recovery and filtering strategy is designed to recover some inliers based on identical vertex trichotomy descriptors and restricted transformation errors. Assisted with the additional recovered inliers, residual outliers can be also filtered out during the process of reaching identical graphs for the expanded vertex sets. Experimental results demonstrate the superior performance on precision and stability of this algorithm under various conditions, such as remote sensing images with large transformations, duplicated patterns, or inconsistent spectral conten
LMS::1688
Keywords: feature point matching; graph-based matching; image registration; remote sensing; vertex trichotomy

RGB-Guided Resolution Enhancement of IR Images - Trammer, Marcel; Genser, Nils; Seiler, Jürgen (International Conference on Systems, Signals and Image Processing (IWSSIP), 2023)

RLS-adapted polynomial for nonlinear acoustic echo cancelling - Stenger, Alexander; Kellermann, Walter (2000 10th European Signal Processing Conference, EUSIPCO 2000, 2000)
Low-cost audio components in hands-free telephone applications call for nonlinear adaptive echo cancellation. It has been demonstrated that a cascade of a polynomial and an FIR filter can cancel echoes due to nonlinearities of typical low-cost applications [1]. Overcoming the slow convergence of an NLMS-based adaptation [5], we derive an RLS-type adaptation for the polynomial preprocessor. Furthermore, we present a robust control of the RLS algorithm. Experiments with double-talk scenarios and real audio hardware show more than 8 dB overall echo reduction gain and fast and robust convergence of the polynomial. In addition, an implementation with reduced computational complexity and similar robustness is presented.

Robust audio localization with phase unwrapping - Chen, Kainan; Geiger, Jürgen; Kellermann, Walter (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017)
Most of multichannel sound source Direction Of Arrival (DOA) estimation algorithms suffer from spatial aliasing problems. The phase differences between a pair of microphones are wrapped beyond the spatial aliasing frequency. A common solution is to adjust the distance between the microphones to obtain a suitable aliasing frequency, and take only the frequency band below the aliasing frequency for localization. With correct phase unwrapping, a broader frequency band can be utilized for localization. In this paper, we investigate a method for phase unwrapping solving the spatial aliasing problem for scenarios with a single source and high-level diffuse background noise (around 0dB SNR). The aliasing frequency is estimated from the signal, and is used to unwrap a phase difference vector. Pre- and post-processing steps are applied to increase the robustness. Our experiments with a large number of simulated and real signals demonstrate the robustness of our method in noise.
Keywords: Sound Source Localization; DOA; Spatial Aliasing; Phase Unwrapping

Robust Binaural Blind Source Separation in Hearing Aids - Wehr, Stefan (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2013)
In this thesis, a special kind of spatial filtering is investigated for the application in binaural hearing aids: Blind Source Separation (BSS). In contrast to conventional beamforming techniques, BSS is capable to deal with both unknown microphone positions and unknown source positions in so-called cocktail party situations, where multiple sources are simultaneously active. Moreover, BSS is able to handle arbitrary obstacles in between the microphones. After introducing the considered BSS algorithms, both the sound propagation from the (external) sources to the ears as well as the voice of the hearing-impaired person are investigated. Moreover, the problem of identifying the target source is addressed. By exploiting a BSS-based source localization algorithm, the target source can be automatically selected by the look direction. Furthermore, a new concept for providing fast and robust source tracking is presented. This concept allows for both source separation and source tracking in rapidly time-varying scenarios. Additionally, a wide spectrum of algorithms for binaural reproduction is discussed. This diversity allows for balancing between the computational complexity and the quality of the binaural reproduction. Finally, the investigations are extended to the voice of the hearing-impaired person, where both the source separation performance and the binaural reproduction are considere

Robust coherence-based spectral enhancement for speech recognition in adverse real-world environments - Barfuß, Hendrik; Hümmer, Christian; Schwarz, Andreas; Kellermann, Walter (Computer Speech and Language, 2017)
Speech recognition in adverse real-world environments is highly affected by reverberation and non-stationary background noise. A well-known strategy to reduce such undesired signal components in multi-microphone scenarios is spatial filtering of the microphone signals. In this article, we demonstrate that an additional coherence-based postfilter, which is applied to the beamformer output signal to remove diffuse interference components from the latter, is an effective means to further improve the recognition accuracy of modern deep learning speech recognition systems. To this end, the 3rd CHiME Speech Separation and Recognition Challenge (CHiME-3) baseline speech enhancement system is extended by a coherence-based postfilter and the postfilter’s impact on the Word Error Rates (WERs) of a state-of-the-art automatic speech recognition system is investigated for the realistic noisy environments provided by CHiME-3. To determine the time- and frequency-dependent postfilter gains, we use Direction-of-Arrival (DOA)-dependent and DOA-independent estimators of the coherent-to-diffuse power ratio as an approximation of the short-time signal-to-noise ratio. Our experiments show that incorporating coherence-based postfiltering into the CHiME-3 baseline speech enhancement system leads to a significant reduction of the WERs, with relative improvements of up to 11.31
Keywords: Robust speech recognition; Postfiltering; Spectral enhancement; Coherence-to-diffuse power ratio; Wiener filter

Robust Deep Neural Object Detection and Segmentation for Automotive Driving Scenario with Compressed Image Data - Fischer, Kristian; Blum, Christian; Herglotz, Christian; Kaup, André (IEEE International Symposium on Circuits and Systems (ISCAS), 2021)
Deep neural object detection or segmentation networks are commonly trained with pristine, uncompressed data. However, in practical applications the input images are usually deteriorated by compression that is applied to efficiently transmit the data. Thus, we propose to add deteriorated images to the training process in order to increase the robustness of the two state-of-the-art networks Faster and Mask R-CNN. Throughout our paper, we investigate an autonomous driving scenario by evaluating the newly trained models on the Cityscapes dataset that has been compressed with the upcoming video coding standard Versatile Video Coding (VVC). When employing the models that have been trained with the proposed method, the weighted average precision of the R-CNNs can be increased by up to 3.68 percentage points for compressed input images, which corresponds to bitrate savings of nearly 48 %. <
Keywords: Video Coding for Machines; Resilient Learning; Faster R-CNN; Mask R-CNN; Versatile Video Coding

Robust Dereverberation with Kronecker Product Based Multichannel Linear Prediction - Yang, Wenxing; Chen, Jingdong; Benesty, Jacob; Cohen, Israel; Kellermann, Walter; Huang, Gongping (IEEE Signal Processing Letters, 2020)
Reverberation impairs not only the speech quality, but also intelligibility. The weighted-prediction-error (WPE) method, which estimates the late reverberation component based on a multichannel linear predictor, is by far one of the most effective algorithms for dereverberation. Generally, the WPE prediction filter in every short-time-Fourier-transform (STFT) subband has to be long enough to estimate accurately the late reverberation component. As a consequence, WPE is computationally expensive, which makes it difficult to implement into real-time embedded or edge computing devices. Moreover, WPE is sensitive to additive noise and its performance may suffer from dramatic degradation even in environments where the signal-to-noise ratio (SNR) is high. To address these drawbacks, this paper proposes to decompose the multichannel linear prediction filter as a Kronecker product of a temporal (interframe) prediction filter and a spatial filter. An iterative algorithm is then developed to optimize the two filters. In comparison with the original WPE algorithm, the presented method not only exhibits better performance in terms of dereverberation and robustness to additive noise, as there are fewer parameters to estimate for a given number of observation signal samples, but is also computationally more efficient, since the dimensions of the covariance matrices after Kronecker product decomposition are smalle
Keywords: Dereverberation; noise robustness; weightedprediction-error; beamforming; Kronecker product filter; speech enhancement

Robust Estimation of Multi-Component Motion in Image Sequences using the Epipolar Constraint - Steinbach, Eckehard; Chaudhuri, S.; Girod, Bernd (Proceedings IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 97), 1997)
LMS::420

Robust extended multidelay filter and double-talk detector for acoustic echo cancellation - Buchner, Herbert; Benesty, Jacob; Gaensler, T.; Kellermann, Walter (IEEE Transactions on Audio Speech and Language Processing, 2006)
We propose an integrated acoustic echo cancellation solution based on a novel class of efficient and robust adaptive algorithms in the frequency domain, the extended multidelay filter (EMDF). The approach is tailored to very long adaptive filters and highly auto-correlated input signals as they arise in wideband full-duplex audio applications. The EMDF algorithm allows an attractive tradeoff between the well-known multidelay filter and the recursive least-squares algorithm. It exhibits fast convergence, superior tracking capabilities of the signal statistics, and very low delay. The low computational complexity of the conventional frequency-domain adaptive algorithms can be maintained thanks to efficient fast realizations. We also show how this approach can be combined efficiently with a suitable double-talk detector (DTD). We consider a corresponding extension of a recently proposed DTD based on a normalized cross-correlation vector whose performance was shown to be superior compared to other DTDs based on the cross-correlation coefficient. Since the resulting DTD also has an EMDF structure it is easy to implement, and the fast realization also carries over to the DTD scheme. Moreover, as the robustness issue during double talk is particularly crucial for fast-converging algorithms, we apply the concept of robust statistics into our extended frequency-domain approach. Due to the robust generalization of the cost function leading to a so-called M-estimator, the algorithms become inherently less sensitive to outliers, i.e., short bursts that may be caused by inevitable detection failures of a DTD. The proposed structure is also well suited for an efficient generalization to the multichannel case. © 2006 IEE
Keywords: Adaptive filtering; Doubletalk; Echo cancellation; Frequency domain; Robust statistics

Robust H.263 Compatible Video Transmission for Mobile Access to Video Servers - Färber, Niko; Girod, Bernd (Proc. IEEE International Conference on Image Processing (ICIP 97), 1997)
LMS::426

Robust localization of multiple sources in reverberant environments using EB-ESPRIT with spherical microphone arrays - Sun, Haohai; Teutsch, Heinz; Mabande, Edwin; Kellermann, Walter (36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 2011)
Spherical microphone array eigenbeam (EB)-ESPRIT gives an elegant closed-form solution for 3D broadband source localization based on the spherical harmonics (eigenbeam) framework. However, in practical implementations, there are still several issues not being rigorously studied, e.g. how to avoid the ill-conditioning of an EB-ESPRIT matrix, solve the ambiguity problem, handle a large number of sources, and localize coherent broadband sources, etc. In this work, we propose to use the condition number of the EB-ESPRIT matrix as a robustness measure, and to use Wigner-D weighting to avoid the ill-conditioning issue and improve the robustness. In addition, power spectrum testing, frequency smoothing, and manifold vector extension techniques are employed to address the ambiguity, coherent source localization, and large source number problems, respectively. Experimental results based on measurements taken with a real spherical microphone array in a room environment show the effectiveness of the proposed methods. © 2011 IEE
Keywords: EB-ESPRIT; eigenbeam; spherical harmonics; Spherical microphone array

Robustness of a Blind Image Watermarking Scheme - Eggers, Joachim; Su, Jonathan; Girod, Bernd (Int. Conf. on Image Processing (ICIP'2000), 2000)
LMS::197

Robustness of Acoustic Multiple-Source Localization in Adverse Environments - Aichner, Robert; Buchner, Herbert; Wehr, Stefan; Kellermann, Walter (Proc. ITG Fachtagung Sprachkommunikation, 2006)
In this paper the robustness of a previously introduced localization algorithm for multiple acoustic sources is investigated and compared to two popular single-source localization algorithms. To show the versatility of the proposed method, experiments are conducted in various environments with different reverberation times and background noises. In addition, moving speakers are used to show the tracking capability. The application to binaural hearing aids shows the applicability of the algorithm to realistic adverse environments such as, e.g., a cafeteria. <br
LMS::601

Robustness of Deep Convolutional Neural Networks for Image Degradations - Ghosh, Sanjukta; Shet, Rohan; Amon, Peter; Hutter, Andreas; Kaup, André (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018)

Robust phase replication method for spatial aliasing problem in multiple sound sources localization - Chen, Kainan; Geiger, Jürgen T.; Jin, Wenyu; Taghizadeh, Mohammad; Kellermann, Walter (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2017)
Most of multichannel sound source Direction Of Arrival (DOA) estimation algorithms suffer from spatial aliasing problems. The Interchannel Phase Differences (IPDs) are wrapped beyond the spatial aliasing frequency. This invalidates the DOA estimation due to significant errors. A real-time algorithm is proposed to address the general IPD wrapping problem for both single-source and multisource scenarios. The algorithm comprises two steps, IPD replication and denoising. In the first step, the obtained IPDs are replicated to all the possible 2π-periodic repetitions, thus performing the correct unwrapping while introducing errors inevitably. However, the replication can also emphasize the correct DOAs of the sources when the replicated matrix is transformed into a histogram. The second step contains a post-processing method supporting our method more robust for a noisy environment. A theoretical analysis and experimental results with simulations and real recordings show the robustness of our method.
Keywords: Spatial aliasing; sound source localization; interchannel phase difference

Robust Rotational Motion Estimation for efficient HEVC compression of 2D and 3D navigation video sequences - Springer, Dominic; Simmet, F.; Niederkorn, D.; Kaup, André (2013 38th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2013, 2013)
In the context of test automation for automobiles, the compressed video recording of infotainment system components like navigation devices is a required practice. These recordings are then analyzed, archived, and forwarded to the responsible engineering teams. In order to compress navigation video sequences efficiently, the dominant rotational motion must be compensated properly. However, the process of Rotational Motion Estimation (RME) is hindered by the presence of static areas like info boxes and overlay graphics. We analyze this problem and show how to build masks for static areas in order to allow high speed feature transforms to be applied. With the acquired fast and accurate RME, we then demonstrate how to significantly reduce the required bitrate during HEVC encoding of navigation sequences. © 2013 IEE
Keywords: Background Subtraction; HEVC; Rotational Motion Estimation; Video Compression

Robust spatial estimation of the signal-to-interference ratio for non-stationary mixtures - Herbordt, Wolfgang; Trini, T.; Kellermann, Walter (Proc. Int. Conf. on Acoustic Echo and Noise Control, 2003)
LMS::447

Robust speaker localization using a microphone array - Strobel, Nina; Rabenstein, Rudolf (2000 10th European Signal Processing Conference, EUSIPCO 2000, 2000)
This paper presents a speaker localization system using a microphone array. The array is operated as a steered filter-and-sum beamformer implemented as a summed correlator. In particular, we emphasize the use of a speech pause detector to improve the robustness of the speaker localization system by avoiding erroneous position estimates when no speech signal is present. Simulation results and measurements show that speech pause detection improves the overall system performance considerably.

Robust Superdirectional Beamforming for Hands-Free Speech Capture in Cars - Mabande, Edwin; Schad, A.; Kellermann, Walter (Proc. NAG/DAGA International Conference on Acoustics, 2009)
The application of multichannel signal processing algo-rithms in front-ends of hands-free speech communication systems facilitates a better extraction of a desired source signal and allows for a suppression of unwanted noise and interference in car interiors. In this paper, a novel superdirective beamforming design method called the Robust Least-Squares Frequency-Invariant Beamformer (RLSFIB) is investigated for microphone arrays in cars. The RLSFIB design offers superior spatial selectivity at low frequencies even for small array apertures and some variability in microphone positioning. To cope with the high sensitivity inherent to superdirectional beamformers, this method directly incorporates a white noise gain (WNG) constraint for controlling the beamformer sensitivity. The resulting problem is formulated as a convex optimization problem which can be solved by conventional iterative solvers and the convexity guarantees a globally optimal solution. As the WNG constraint can be chosen freely, the method can match desired robustness levels, and therefore can be adapted to given prior knowledge on microphone mismatch, positioning errors, and microphone self-noise. Design examples for typical car requirements will be presente

Robust Super-Resolution for Mixed-Resolution Multiview Image plus Depth Data - Richter, Thomas; Seiler, Jürgen; Schnurrer, Wolfgang; Kaup, André (IEEE Transactions on Circuits and Systems For Video Technology, 2016)
Increasing spatial resolution and thus improving the image quality is a key issue in the mixed-resolution multiview image and video processing domain. Given adjacent camera perspectives with various spatial resolutions and their corresponding depth information, the high-frequency part of a high-resolution view can be used for increasing the image quality of a neighboring low-resolution camera perspective. However, a reasonable projection of high-frequency information onto the image plane of a neighboring low-resolution view typically requires pixel-wise error-free depth data for the high-resolution reference image. Starting from this, a novel image super-resolution approach is proposed that is robust against both inaccurate depth acquisition and nonperfect calibration of spatially low-resolution depth sensors. The algorithm is based on displacement-compensated high-frequency synthesis and aims at correcting the projection errors introduced by inaccurate depth information. The proposed approach is further effectively extended by a signal extrapolation technique. For a wide range of proper scenarios, the proposed framework achieves substantial objective and visual gains compared with the considered reference approaches. The improvement of quality is shown for both simulated and self-recorded experimental dat
UnivIS-Import:2015-10-26:Pub.2015.tech.IE.LEN.robust
Keywords: depth calibration; displacement compensation; DC; multiview; signal extrapolation; super-resolution; SR; time-of-flight camera; ToF

Robust super-resolution in a multiview setup based on refined high-frequency synthesis - Richter, T.; Seiler, Jürgen; Schnurrer, Wolfgang; Kaup, André (2012 IEEE 14th International Workshop on Multimedia Signal Processing, MMSP 2012, 2012)
Increasing image sharpness and thus improving the visual quality is an important task in multiview image and video processing. We propose a novel super-resolution approach for multiview images in a mixed-resolution setup that is robust to various depth map distortions. The considered distortion scenarios may be caused by an inaccurate calibration of the depth camera or a limitation of depth range. Our method is based on a refined high-frequency synthesis that relies on a blockwise and depth-dependant low-frequency registration. The refinement step efficiently adapts the high-frequency content from a neighboring high-resolution camera to a low-resolution view and thereby compensates the displacement caused by depth inaccuracies. In case of undistorted depth maps, the results show that our algorithm leads to a PSNR gain of up to 1.33 dB with respect to a comparable unrefined super-resolution approach for a mixed-resolution multiview video plus depth format. Compared to the initial low-resolution view, a PSNR gain of up to 2.61 dB is obtained. In case of distorted depth maps, a PSNR gain of even 4.78 dB is achieved with respect to the reference superresolution algorithm. The PSNR gains get confirmed by the corresponding SSIM values which manifest a similar behaviour. The improvement of visual quality is also convincingly for all considered scenarios. © 2012 IEE

Robust Time-Invariant Broadband Beamforming as a Convex Optimization Problem - Mabande, Edwin (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2014)
Beamformer designs that provide high directional gain with a small array aperture and a small number of sensors are highly desirable for applications such as hands-free communication and telecommunication, and acoustic front-ends for human-machine interfaces. However, their application in practice is greatly limited due to the high sensitivity of these designs to sensor self-noise, mismatch between sensor characteristics, and imprecise sensor positioning, which are typically unavoidable in practice. It is therefore necessary to control the robustness of these beamformer designs. The white noise gain (WNG) is a well-known and widely used robustness measure for beamformers. However, its application in controlling the robustness of broadband beamformer designs has been somewhat limited due to the difficulty of incorporating it directly into the design as a constraint. Beamformer designs that control the robustness by constraining the WNG directly are highly desirable. This thesis provides a generic framework for the design of robust time-invariant broadband beamformers as a constrained optimization problem, where robustness is achieved by constraining the WNG directly. In the constrained problem we seek to minimize a beamformer cost function that is convex subject to constraints on WNG and on the response in the desired look direction. Six special cases of the generic framework were derived. The constrained problems are shown to be convex and therefore well-known methods for convex optimization can be used to solve these problems resulting in globally optimal solutions for the chosen design parameters. Simulations confirmed the ability of these designs to constrain the WNG effectively, thus ensuring robust beamformer designs. Thus the generic framework allows for flexible robustness control via constraining the WNG directly. Furthermore, this thesis provides a method for three-dimensional room geometry inference based on robust and high-resolution beamforming techniques that are special cases of the generic framework. Uncontrolled broadband acoustic sources such as speech are used to infer the room geometry. The high accuracy of the proposed room geometry inference technique is confirmed by experimental evaluations based on both simulated and measured data for moderately reverberant room
Keywords: convex optimization; robust broadband beamforming

Room acoustic analysis: Complementing 2D measurements by 3D simulations - Kuntz, Achim; Rabenstein, Rudolf (2008 3rd International Symposium on Communications, Control, and Signal Processing, ISCCSP2008, 2008)
This paper demonstrates the analysis of complex acoustic environments using high-resolution circular measurements and complementary 3D simulations. The possibilities and limitations of room acoustic analysis using either measurements or simulations are discussed. An exemplary study is presented on the sound propagation in a complex shaped room: The analysis based on multichannel room impulse responses measured on a large circular aperture is introduced as a starting point of the room acoustic investigation. It is shown how the extrapolation of the measured data provides some insight into the sound propagation but fails to explain sound field characteristics caused by the complex 3D shape of the acoustic environment. In particular the individual 3D acoustic paths of higher order reflections cannot be derived from measurements alone. This paper demonstrates how they can be identified by complementary 3D simulations. It is elaborated that in the proposed joint application of 2D measurements and 3D simulations both techniques complement each other such that their limitations can be removed. ©2008 IEE

Room Acoustics Simulation Using 3-D Compact Explicit FDTD Schemes - Kowalczyk, Konrad; van Walstijn, M. (IEEE Transactions on Audio Speech and Language Processing, 2011)
This paper presents methods for simulating room acoustics using the finite-difference time-domain (FDTD) technique, focusing on boundary and medium modeling. A family of nonstaggered 3-D compact explicit FDTD schemes is analyzed in terms of stability, accuracy, and computational efficiency, and the most accurate and isotropic schemes based on a rectilinear grid are identified. A frequency-dependent boundary model that is consistent with locally reacting surface theory is also presented, in which the wall impedance is represented with a digital filter. For boundaries, accuracy in numerical reflection is analyzed and a stability proof is provided. The results indicate that the proposed 3-D interpolated wideband and isotropic schemes outperform directly related techniques based on Yee's staggered grid and standard digital waveguide mesh, and that the boundary formulations generally have properties that are similar to that of the basic scheme use
LMS::937
Keywords: Acoustic propagation; acoustic reflection; acoustic signal processing; architectural acoustics; finite-difference time-domain (FDTD) methods; numerical analysis

Room geometry inference based on spherical microphone array eigenbeam processing - Mabande, Edwin; Kowalczyk, Konrad; Sun, Haohai; Kellermann, Walter (Journal of the Acoustical Society of America, 2013)
The knowledge of parameters characterizing an acoustic environment, such as the geometric information about a room, can be used to enhance the performance of several audio applications. In this paper, a novel method for three-dimensional room geometry inference based on robust and high-resolution beamforming techniques for spherical microphone arrays is presented. Unlike other approaches that are based on the measurement and processing of multiple room impulse responses, here, microphone array signal processing techniques for uncontrolled broadband acoustic signals are applied. First, the directions of arrival (DOAs) and time differences of arrival (TDOAs) of the direct signal and room reflections are estimated using high-resolution robust broadband beamforming techniques and cross-correlation analysis. In this context, the main challenges include the low reflected-signal to background-noise power ratio, the low energy of reflected signals relative to the direct signal, and their strong correlation with the direct signal and among each other. Second, the DOA and TDOA information is combined to infer the room geometry using geometric relations. The high accuracy of the proposed room geometry inference technique is confirmed by experimental evaluations based on both simulated and measured data for moderately reverberant rooms. © 2013 Acoustical Society of Americ

Room-related Sound Representation using Loudspeakers - Blauert, Jens; Rabenstein, Rudolf (Architectural Acoustics Handbook, 2017)
Available methods for room-related sound presentation are introduced and evaluated. A focus is put on the synthesis side rather than on complete transmission systems. Different methods are compared using common, though quite general criteria. The methods selected for comparison are: intensity stereophony after Blumlein (1931), vector base amplitude panning (VBAP), 5.1-Surround and its discrete-channel derivatives, synthesis with spherical harmonics [Ambisonics and higher-order Ambisonics (HOA)], and synthesis based on the boundary method, namely, wave-field synthesis (WFS), and binaural-cue selection methods (e.g., DIRAC). While VBAP, 5.1-Surround and other discrete-channel-based methods show a number of practical advantages, they do not, in the end, aim at authentic sound-field reproduction. The holophonic methods that do so, particularly, HOA and WFS, have specific advantages and disadvantages that will be discussed. Yet, both methods are under continuous development, and a decision in favor of one of them should be taken from a strictly application-oriented point of view by considering relevant application-specific advantages and disadvantages in detail.
UnivIS-Import:2017-12-18:Pub.2017.tech.IE.LEN.roomre
Keywords: spatial reproduction; intensity stereophony; vector base amplitude panning; surround sound; Ambisonics; wave-field synthesis; binaural-cue; spherical harmonics

Safety Concepts and Circuit Protection for LVDC-Grids in Datacenters and in Telecommunications - Strobl, Christian; Köpf, Hendrik; Mehl, Richard; Ott, Leopold; Kaiser, Julian; Gosses, Kilian; Schäfer, Maximilian; Rabenstein, Rudolf (International Telecommunications Energy Conference (Intelec), 2018)
Specific system protection is needed for direct current microgrids in the low voltage range. Model-based analysis of transient events applying system identification and machine learning procedures provides additional knowledge on system topology and components. Using this information, refined model-based fault detection and discrimination methodologies can be set up in addition to the basic functionality of conventional protective devices. Consequential damage and grid sector breakdown can be avoided by fast and selective tripping of suitable switching devices. <br

Saliency-Driven Hierarchical Learned Image Coding for Machines - Fischer, Kristian; Brand, Fabian; Blum, Christian; Kaup, André (ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023)
We propose to employ a saliency-driven hierarchical neural image compression network for a machine-to-machine communication scenario following the compress-then-analyze paradigm. By that, different areas of the image are coded at different qualities depending on whether salient objects are located in the corresponding area. Areas without saliency are transmitted in latent spaces of lower spatial resolution in order to reduce the bitrate. The saliency information is explicitly derived from the detections of an object detection network. Furthermore, we propose to add saliency information to the training process in order to further specialize the different latent spaces. All in all, our hierarchical model with all proposed optimizations achieves 77.1 % bitrate savings over the latest video coding standard VVC on the Cityscapes dataset and with Mask R-CNN as analysis network at the decoder side. Thereby, it also outperforms traditional, non-hierarchical compression networks.
CRIS-Team Scopus Importer:2024-12-16
Keywords: Coding for machines; Hierarchical coding; Machine-to-machine communication; Neural image compression

Saliency-Driven Versatile Video Coding for Neural Object Detection - Fischer, Kristian; Fleckenstein, Felix; Herglotz, Christian; Kaup, André (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021)
Saliency-driven image and video coding for humans has gained importance in the recent past. In this paper, we propose such a saliency-driven coding framework for the video coding for machines task using the latest video coding standard Versatile Video Coding (VVC). To determine the salient regions before encoding, we employ the real-time-capable object detection network You Only Look Once (YOLO) in combination with a novel decision criterion. To measure the coding quality for a machine, the state-of-the-art object segmentation network Mask R-CNN was applied to the decoded frame. From extensive simulations we find that, compared to the reference VVC with a constant quality, up to 29 % of bitrate can be saved with the same detection accuracy at the decoder side by applying the proposed saliency-driven framework. Besides, we compare YOLO against other, more traditional saliency detection methods.<br
Keywords: Video Coding for Machines; Saliency Coding; Versatile Video Coding; Mask R-CNN; YOLO

Sample-based Weighted Prediction for Lossless Enhancement Layer Coding in HEVC - Heindel, Andreas; Wige, Eugen; Kaup, André (Grand Challenge at Picture Coding Symposium (PCS), 2013)
LMS::1408

Sample-based Weighted Prediction for Lossless Enhancement Layer Coding in SHVC - Heindel, Andreas; Wige, Eugen; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2014)
Many professional applications require video content to be compressed in a lossless manner. However, often it is also advantageous to have access to a lossy coded version of the same video content, for example for preview purposes. In this paper the Sample-based Weighted Prediction (SWP) algorithm is applied for two-layer lossy to lossless scalable video compression with SHVC. This means that lossy compression is used for the first layer, the so-called base layer, and lossless compression is used for the second layer, also referred to as enhancement layer. In order to increase the coding efficiency of the enhancement layer the SWP algorithm is used for sample-wise prediction. Depending on the quantization parameter for the base layer, significant bitrate savings considering both layers of up to 8.38% (up to 6.20% on average) compared to the unmodified SHVC reference software can be achieve
Keywords: Lossless coding; sample-based prediction; scalable coding; SHVC

Sample-based weighted prediction with directional template matching for HEVC lossless coding - Wige, Eugen; Yammine, Gilbert; Amon, P.; Hutter, A.; Kaup, André (2013 Picture Coding Symposium, PCS 2013, 2013)
The recently introduced High Efficiency Video Coding (HEVC) standard is currently further investigated for potential use in professional applications. The considered Range Extensions should on the one hand introduce higher bit depths and additional color formats, and on the other hand the coding efficiency of HEVC for high fidelity compression as well as lossless compression is to be improved. In this paper we investigate and improve the recently introduced Sample-based Weighted Prediction (SWP) for HEVC lossless coding. Although being very efficient for natural video content, the SWP algorithm can be further improved for screen content by using a directional template predictor in cases where the SWP algorithm yields worse prediction. The mainly introduced predictor improves the lossless coding results by up to 9.9% compared to the unmodified HEVC reference software for lossless compression. © 2013 IEE

Saturating Receiver and Receptor Competition in Synaptic DMC: Deterministic and Statistical Signal Models - Lotter, Sebastian; Schäfer, Maximilian; Zeitler, Johannes; Schober, Robert (IEEE Transactions on Nanobioscience, 2021)
Synaptic communication is based on a biological Molecular Communication (MC) system which may serve as a blueprint for the design of synthetic MC systems. However, the physical modeling of synaptic MC is complicated by the possible saturation of the molecular receiver caused by the competition of neurotransmitters (NTs) for postsynaptic receptors. Receiver saturation renders the system behavior nonlinear in the number of released NTs and is commonly neglected in existing analytical models. Furthermore, due to the ligands’ competition for receptors (and vice versa), the individual binding events at the molecular receiver are in general not statistically independent and the commonly used binomial model for the statistics of the received signal does not apply. Hence, in this work, we propose a novel deterministic model for receptor saturation in terms of a state-space description based on an eigenfunction expansion of Fick’s diffusion equation. The presented solution is numerically stable and computationally efficient. Employing the proposed deterministic model, we show that saturation at the molecular receiver effectively reduces the peak-value of the expected received signal and accelerates the clearance of NTs as compared to the case when receptor occupancy is neglected. We further derive a statistical model for the received signal in terms of the hypergeometric distribution which accounts for the competition of NTs for receptors and the competition of receptors for NTs. The proposed statistical model reveals how the signal statistics are shaped by the number of released NTs, the number of receptors, and the binding kinetics of the receptors, respectively, in the presence of competition. In particular, we show that the impact of these parameters on the signal variance is qualitatively different depending on the relative numbers of NTs and receptors. Finally, the accuracy of the proposed deterministic and statistical models is verified by particle-based computer simulations.
Keywords: Receivers; Mathematical model; Analytical models; Computational modeling; Numerical models; Monte Carlo methods; Degradation

Scalable Codec Architctures for Internet Video-on-Demand - Girod, Bernd; Färber, Niko; Horn, Uwe (Proc. 1997 Asilomar Conference on Signals and Systems, 1997)
LMS::427

Scalable Internet Video Streaming With Unequal Error Protection - Stuhlmüller, Klaus; Link, M.; Girod, Bernd; Horn, Uwe (Packet Video Workshop, 1999)
LMS::286

Scalable Kernel-Based Minimum Mean Square Error Estimator for Accelerated Image Error Concealment - Koloda, Jan; Seiler, Jürgen; Peinado, A.M.; Kaup, André (IEEE Transactions on Broadcasting, 2017)
Error concealment (EC) is of great importance for block-based video systems, such as digital video broadcasting or video streaming services. In this paper, we propose a novel scalable spatial EC algorithm that aims at obtaining high quality reconstructions with reduced computational burden. The proposed technique exploits the excellent reconstructing abilities of the kernel-based minimum mean square error (K-MMSE) estimator. We propose to decompose this approach into a set of hierarchically stacked layers. The first layer performs the basic reconstruction that the subsequent layers can eventually refine. In addition, we design a layer management mechanism, based on profiles, that dynamically adapts the use of higher layers to the visual complexity of the area being reconstructed. The proposed technique outperforms other state-of-the-art algorithms and produces high quality reconstructions, equivalent to K-MMSE, while requiring around one tenth of its computational tim
LMS::1689
Keywords: scalable error concealment; kernel-based MMSE; image reconstruction

Scalable Lossless Coding of Dynamic Medical CT Data Using Motion Compensated Wavelet Lifting with Denoised Prediction and Update - Lanz, Daniela; Schilling, Franz; Kaup, André (Picture Coding Symposium (PCS), 2019)

Scalable Near-Lossless Video Compression Based on HEVC - Heindel, Andreas; Kaup, André (IEEE Visual Communications and Image Processing (VCIP), 2017)
Lossless or near-lossless video compression with a certain maximum absolute reconstruction error is required in many professional applications like archiving or medical imaging. However, lossless coding resulting in rather high bit rates and near-lossless coding being traditionally used and especially optimized for still image coding prevents these techniques from practical application. Consequently, this paper proposes a system for scalable lossy to near-lossless video compression, where the inter picture correlations are exploited by the base layer codec and near-lossless enhancement layer compression delivers the required final reconstruction fidelity in terms of maximum absolute reconstruction error. For the proposed near-lossless enhancement layer codec we extended the Sample-based weighted prediction for Enhancement Layer Coding (SELC) scheme, and the HEVC reference software is used for base layer coding. The experimental results show significant average savings compared to both JPEG-LS single layer coding (27%) and also JPEG-LS for enhancement layer coding (about 14 percentage points
Keywords: Near-lossless coding; scalable coding; video coding; HEVC; SELC-NL; SWP

Scalable Video Transmission for the Internet - Horn, Uwe; Girod, Bernd (Computer Networks and ISDN Systems, 1997)
LMS::425

Scalable Wavelet-Based Multiview Coding - Garbas, Jens-Uwe (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2010)

Scale and shape adaptive mean shift object tracking in video sequences - Quast, Katharina; Kaup, André (17th European Signal Processing Conference, EUSIPCO 2009, 2009)
A new technique for object tracking based on the mean shift method is presented. Instead of using a symmetric kernel like in traditional mean shift tracking, the proposed tracking algorithm uses an asymmetric kernel which is retrieved from an object mask. During the mean shift iterations not only the new object position is located but also the kernel scale is altered according to the object scale, providing an initial adaption of the object shape. The final shape of the kernel is then obtained by segmenting the area inside and around the adapted kernel and distinguishing the object segments from the non-object segments. Thus, the object shape is tracked very well even if the object is performing out-of-plane rotations. © EURASIP, 200

Scaled Fixed-Point Frequency Selective Extrapolation for Fast Image Error Concealment - Genser, Nils; Seiler, Jürgen; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2017)
Image and video signal processing demands in many areas for error concealment algorithms, whereby the execution time plays an important role. Within this paper, we introduce the scaled fixed-point Frequency Selective Extrapolation for fast image error concealment. This algorithm is based on the complex-valued Frequency Selective Extrapolation, but reduces computational complexity. It is shown that a fixed-point realization of the state-of-the-art algorithm requires an impracticable word-length, which makes it necessary to design a novel, scaled Frequency Selective Extrapolation that reduces the required word-length. Regarding the new approach, execution time can be reduced on average by 22.84 % and at best by up to 40.45 % compared to the state-of-the-art floating point Frequency Selective Extrapolation at the same reconstruction quality. Moreover, platforms can be exploited, which support fixed-point arithmetic onl
Keywords: Error Concealment; Inpainting; Signal Extrapolation

Scattering Transform Based Image Clustering using Projection onto Orthogonal Complement - Villar-Corrales, Angel; Morgenshtern, Veniamin (ICDAR '21: PROCEEDINGS OF THE 2021 WORKSHOP ON INTELLIGENT CROSS-DATA ANALYSIS AND RETRIEVAL, 2021)
CRIS-Team WoS Importer:2022-04-01

Schallfeldsynthese mit Lautsprechern I - Beschreibung und Bewertung - Blauert, Jens; Rabenstein, Rudolf (9. ITG Fachtagung Sprachkommunikation, 2010)
Verfügbare Methoden raumbezogener Lautsprecherwiedergabe werden vorgestellt und evaluiert, wobei Anwendungen zur Synthese räumlicher Schallfelder und weniger solche zur Übertragung im Vordergrund der Betrachtung stehen. Folgende Methoden werden beschrieben und vergleichend diskutiert: Amplitudendifferenz-Stereophonie (Intensitätsstereophonie), Amplitudenbasierte 3-D-Richtungsmischung (z. B. VBAP), Rundum-Klang (5.1-Surround), Synthese mittels Kugelflächenfunktionen (Ambisonics), Synthese nach der Hüllflächenmethode (WFS) sowie Binaurale Merkmalselektion (z. B. DirAC

Schallfeldsynthese mit Lautsprechern II - Signalverarbeitung - Rabenstein, Rudolf; Blauert, Jens (9. ITG Fachtagung Sprachkommunikation, 2010)

Schnelle Bestimmung von Kamerabewegungen in blockbasiert codierten Videosequenzen - Heuer, J.; Kaup, André; Burkhardt, H. (Proceedings 20. DAGM-Symposium Mustererkennung, 1998)
LMS::472

Schutzorgane und -konzepte in Niederspannungs-Gleichstromnetzen – Modellierung von Komponenten, Fehlersituationen und Gesamtsystem - Strobl, Christian; Schäfer, Maximilian; Rabenstein, Rudolf; Ott, Leopold; Kaiser, Julian (Workshop der ASIM/GI - Fachgruppen STS und GMMS, 2017)
LMS::1716

Segment-Oriented Coding of Texture Images Based on Successive Approximation - Kaup, André; Aach, T. (IEEE International Symposium on Speech, Image Processing and Neural Networks, Hong Kong, 1994)
LMS::1023

Selbstabbildung: Grundlagen und Anwendungsmöglichkeiten - Lohmann, Adolf; Jahns, J.; Knuppertz, H. (104. Jahrestagung der DGaO, Münster, 2003)
LMS::656

Selective Signal Extrapolation and its Application in Image and Video Communications, PhD thesis - Meisinger, Katrin (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2008)

SelectNAdapt: Support Set Selection for Few-Shot Domain Adaptation - Dawoud, Youssef; Carneiro, Gustavo; Belagiannis, Vasileios (Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023, 2023)
Generalisation of deep neural networks becomes vulnerable when distribution shifts are encountered between train (source) and test (target) domain data. Few-shot domain adaptation mitigates this issue by adapting deep neural networks pre-trained on the source domain to the target domain using a randomly selected and annotated support set from the target domain. This paper argues that randomly selecting the support set can be further improved for effectively adapting the pre-trained source models to the target domain. Alternatively, we propose SelectNAdapt, an algorithm to curate the selection of the target domain samples, which are then annotated and included in the support set. In particular, for the K-shot adaptation problem, we first leverage self-supervision to learn features of the target domain data. Then, we propose a per-class clustering scheme of the learned target domain features and select K representative target samples using a distance-based scoring function. Finally, we bring our selection setup towards a practical ground by relying on pseudo-labels for clustering semantically similar target domain samples. Our experiments show promising results on three few-shot domain adaptation benchmarks for image recognition compared to related approaches and the standard random selection.
CRIS-Team Scopus Importer:2024-12-16

Self-Configuring System Identification via Evolutionary Frequency-Domain Adaptive Filters - Zeller, Marcus; Kellermann, Walter (Int. Workshop on Acoustic Echo and Noise Control (IWAENC), 2010)
This paper proposes a novel strategy for adaptive frequencydomain filtering in arbitrary system identification tasks. Employing a convex combination of two competing filters with different size, the lengths of the components are dynamically adjusted, following a performance-based soft decision. Thus, a doubly-adaptive, evolutionary filter structure is realized that can find and track the optimum filter length so as to automatically match the adaptive model size to that of the unknown system. Regarding the proposed DFT-domain algorithm, a focus is put on the various implications on coefficient updates and memory organization for different choices of the length increment. Results for the effectiveness of the estimation and identification are shown for noise and speech signals and several systems in an AEC scenari

Separating convolutive mixtures with trinicon - Kellermann, Walter; Buchner, Herbert; Aichner, Robert (2006 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2006, 2006)
Blind source separation (BSS) algorithms are often categorized as either narrowband or broadband algorithms depending on whether their respective cost functions aim at individual DFT bins or the entire broadband signal. In this contribution, we present comparable general natural gradient-based formulations of both concepts based on the TRINICON framework. As a distinctive feature, narrowband algorithms imply an internal permutation and scaling problem. We show that the common DOA estimation-based methods for aligning the permutations effectively rely on geometric a-priori knowledge, and we explain why they need to be complemented by additional repair mechanisms for robust BSS. The latter can already be viewed as approximations of the generic TRINICON broadband algorithm. As a conclusion, we propose to always use a generic broadband algorithm as a starting point for the design of new BSS algorithms. © 2006 IEE

Sequential Error Concealment for Video/Images by Sparse Linear Prediction - Koloda, Jan; Østergaard, J.; Jensen, S.H.; Peinado, A.M.; Sánchez, V. (IEEE Transactions on Multimedia, 2013)
In this paper, we propose a novel sequential error concealment algorithm for video and images based on sparse linear prediction. Block-based coding schemes in packet loss environments are considered. Images are modelled by means of linear prediction, and missing macroblocks are sequentially reconstructed using the available groups of pixels. The optimal predictor coefficients are computed by applying a missing data regression imputation procedure with a sparsity constraint. Moreover, an efficient procedure for the computation of these coefficients based on an exponential approximation is also proposed. Both techniques provide high-quality reconstructions and outperform the state-of-the-art algorithms both in terms of PSNR and MS-SSI
LMS::1532

Sequential Error Concealment for Video/Images by Weighted Template Matching - Koloda, Jan; Østergaard, J.; Jensen, S.H.; Peinado, A.M.; Sánchez, V. (Data Compression Conference (DCC), 2012)
In this paper we propose a novel spatial error concealment algorithm for video and images based on convex optimization. Block-based coding schemes in packet loss environment are considered. Missing macro blocks are sequentially reconstructed by filling them with a weighted set of templates extracted from the available neighbourhood. Moreover, a fast approximation of the optimization method is proposed. The technique produces high quality reconstructions that outperforms the state-of-the-art algorithms both in terms of PSNR and MS-SSI
LMS::1535

Sequential Polarization Imaging Using Multi-View Fusion - Richter, Thomas; Saloman, Christopher; Genser, Nils; Kasnakli, Koray; Seiler, Jürgen; Nowak, Arne; Kaup, André; Schöberl, Michael (IEEE International Workshop on Signal Processing Systems (SiPS), 2017)
Multi-channel imaging like RGB is common today, however, a custom configuration of other modalities, such as wavelengths or polarization, is expensive and often unavailable. We propose a novel configurable multi-modal imaging system that solves this problem by using off-the-shelf components and proper fusion algorithms. The paper shows the application of polarization imaging for machine vision inspection. Sequential images acquired with different cameras and polarization angles are first registered and then fused together. With standard components, we demonstrate a superior image quality to existing specialized cameras while being significantly cheaper, available, and easily configurabl

Serially Connected Channels: Capacity and Video Streaming Application Scenario for Separate and Joint Channel Coding - Petrausch, Stefan; Sörgel, Wolfgang; Kaup, André (5th International ITG Conference on Source and Channel Coding (SCC), 2004)
The serial connection of transmission channels is examined. The main focus lies on the comparison of the achievable capacities resp. data rates, using one channel code for both channels on one hand, and using separate channel codes for each channel on the other hand. First some principal properties are shown on memoryless channel models. Afterward the innuence of memory is examined on the example of serially connected Gilbert-Elliott channels. To proof the practical relevance of this result, a closer look is taken on the example application scenario of video streaming to mobile clients. A suitable and manageable channel model for this wired-wireless combination is introduced, which allows to estimate the maximal relative capacity gain by separate channel coding. It is shown to be approximately equal to the error probability of the wired channel. For verification a simple video transmission system is used, Peak Signal to Noise Ratio (PSNR) measurements of a layer based video stream, corrupted by a serial connection of two block oriented channels, are confirming the expected gains in data rate.

‘Shadow BSS’ for Blind Source Separation in Rapidly Time-Varying Acoustic Scenes - Wehr, Stefan; Lombard, Anthony; Buchner, Herbert; Kellermann, Walter (7th International Conference on Independent Component Analysis and Signal Separation, 2007)
This paper addresses the tracking capability of blind source separation algorithms for rapidly time-varying sensor or source positions. Based on a known algorithm for blind source separation, which also allows for simultaneous localization of multiple active sources in reverberant environments, the source separation performance will be investigated for abrupt microphone array rotations representing the worst case. After illustrating the deficiencies in source-tracking with the given efficient implementation of the BSS algorithm, a method to ensure robust source separation even with abrupt microphone array rotations is proposed. Experimental results illustrate the efficiency of the proposed concept.©2007 Springer<br
LMS::710

Shape Adaptive Mean Shift Object Tracking Using Gaussian Mixture Models - Quast, Katharina; Kaup, André (11th IEEE International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), 2010)
GMM-SAMT, a new object tracking algorithm based on a combination of the mean shift principal and Gaussian mixture models is presented. GMM-SAMT uses an asymmetric shape adapted kernel, instead of a symmetrical one like in traditional mean shift tracking. During the mean shift iterations the kernel scale is altered according to the object scale, providing an initial adaption of the object shape. The final shape of the kernel is then obtained by segmenting the area inside and around the adapted kernel into object and non-object segments using Gaussian mixture model
LMS::843

Shape Adaptive Mean Shift Object Tracking Using Gaussian Mixture Models - Quast, Katharina; Kaup, André (Analysis, Retrieval and Delivery of Multimedia Content, 2012)
GMM-SAMT, a new object tracking algorithm based on a combination of the mean shift principal and Gaussian mixture models is presented. GMM-SAMT uses an asymmetric shape adapted kernel, instead of a symmetrical one like in traditional mean shift tracking. During the mean shift iterations the kernel scale is altered according to the object scale, providing an initial adaption of the object shape. The final shape of the kernel is then obtained by segmenting the area inside and around the adapted kernel into object and non-object segments using Gaussian mixture model

Shape Initialisation of 3-D objects in Videoconference Scenes - Riegel, T.; Kaup, André (Stereoscopic Displays and Virtual Reality Systems IV, SPIE, 1997)
LMS::477

Side Information and Noise Learning for Distributed Video Coding Using Optical Flow and Clustering - van Luong, Huynh; Raket, L. L.; Huang, X.; Forchhammer, S. (IEEE Transactions on Image Processing, 2012)
Distributed video coding (DVC) is a coding paradigm that exploits the source statistics at the decoder side to reduce the complexity at the encoder. The coding efficiency of DVC critically depends on the quality of side information generation and accuracy of noise modeling. This paper considers transform domain Wyner-Ziv (TDWZ) coding and proposes using optical flow to improve side information generation and clustering to improve the noise modeling. The optical flow technique is exploited at the decoder side to compensate for weaknesses of block-based methods, when using motion-compensation to generate side information frames. Clustering is introduced to capture cross band correlation and increase local adaptivity in the noise modeling. This paper also proposes techniques to learn from previously decoded WZ frames. Different techniques are combined by calculating a number of candidate soft side information for low density parity check accumulate decoding. The proposed decoder side techniques for side information and noise learning (SING) are integrated in a TDWZ scheme. On test sequences, the proposed SING codec robustly improves the coding efficiency of TDWZ DVC. For WZ frames using a GOP size of 2, up to 4-dB improvement or an average (Bjøntegaard) bit-rate savings of 37% is achieved compared with DISCOVE
LMS::1539
Keywords: adaptive noise; distributed video coding; multihypothesis; noise residual learning; optical flow

Signal and Loss Geometry Aware Frequency Selective Extrapolation for Error Concealment - Genser, Nils; Seiler, Jürgen; Schilling, Franz; Kaup, André (Picture Coding Symposium (PCS), 2018)

Signale und Systeme - Kellermann, Walter (Informationstechnik kompakt, 1999)

Signal Extrapolation Using Sparse Representations and its Applications in Video Communication - Seiler, Jürgen (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2011)
Signal extrapolation is the process of extending a signal from regions where it is known into regions where no knowledge of the original signal exists. This topic is especially important in the field of image and video communication in case that the available signal is not known completely but only in subareas. This may happen for example if the desired signal becomes distorted. For estimating the original signal, as much information as possible has to be extracted from the known samples in order to obtain a reasonable extrapolation. In the scope of this dissertation, several extrapolation algorithms are introduced and it is shown that a high extrapolation quality is possible, even in the case that only little of the original information is availabl

Signalnachverarbeitung zur Qualitätsverbesserung von mobil empfangenen analogen Videodaten - Friebe, Markus; Kaup, André (Elektronische Medien: 11. Dortmunder Fernsehseminar, ITG-Fachbericht 188, 2005)
LMS::568

Signal processing and machine learning for speech and audio in acoustic sensor networks - Kellermann, Walter; Martin, Rainer; Ono, Nobutaka (EURASIP Journal on Audio, Speech, and Music Processing, 2023)
CRIS-Team Scopus Importer:2023-12-29

Signal Processing for Sound Synthesis: Computer-Generated Sounds and Music for All - Välimäki, V.; Rabenstein, Rudolf; Rocchesso, D.; Serra, X.; Smith, J. O. (IEEE Signal Processing Magazine, 2007)
LMS::697

Signal Processing for Spatial Sound Reproduction with Wave Field Synthesis - Rabenstein, Rudolf; Spors, Sascha (Digital Signal Processing and Applications (DSPA 2006), 2006)
Wave Field Synthesis is a technique for spatial sound reproduction that overcomes the limitations of classical stereophonic techniques. A thorough analysis of acoustic wave propagation prinicples delivers the technical description in terms of loudspeaker array technology. Suitable signal processing methods convert the source signals into driving signals for each loudspeaker. The suitability of this concept has been demonstrated by a number of implemented systems with some ten to a few hundred of loudspeaker
LMS::624

Signal processing in hearing aids: results of the HEARCOM project - Wouters, J.; Luts, Heleen; Eneman, Koen; Spriet, Ann; Moonen, Marc; Büchler, Michael; Dillier, Norbert; Dreschler, Wouter A.; Froehlich, Matthias; Grimm, Giso; Hohmann, Volker; Houben, Rolph; et al. (The Journal of the Acoustical Society of America, 2008)
LMS::747

Signal processing techniques for assisted listening (editorial) - Nordholm, S.; Kellermann, Walter; Doclo, S.; Välimäki, V.; Makino, S.; Hershey, J. (IEEE Signal Processing Magazine, 2015)
The articles in this special section focus on the signal processing technologies and applications that support assisted listenin

Signals and Systems - Girod, Bernd; Rabenstein, Rudolf; Stenger, Alexander (J. Wiley & Sons, 2001)
This book provides a comprehensive, modern approach to signals and systems, concentrating on those aspects that are most relevant for applications such as communication systems and signal processing. Emphasis is placed on building the reader's intuition and problem-solving ability, rather than formal theorems and proofs. "The coverage of the book is comprehensive, providing a broad overview, using a whole host of exercises. The wealth of the worked examples and problems complemented by solutions is particularly attractive. The level of mathematics is not too daunting for the good average student and the authors do their utmost to mitigate the difficulties, skilfully using worked examples." Prof. Lajos Hanzo, University of Southampton
LMS::167

Signalverarbeitung für akustische Mensch-Maschine-Schnittstellen - Kellermann, Walter; Herbordt, Wolfgang; Buchner, Herbert (Tagungsband zur 13. Konf. elektronische Sprachsignalverarbeitung (ESSV), 2002)
LMS::363

Significance-aware filtering for nonlinear acoustic echo cancellation - Hofmann, Christian; Hümmer, Christian; Günther, Michael; Kellermann, Walter (EURASIP Journal on Advances in Signal Processing, 2016)
This article summarizes and extends the recently proposed concept of Significance-Aware (SA) filtering for nonlinear acoustic echo cancellation. The core idea of SA filtering is to decompose the estimation of the nonlinear echo path into beneficially interacting subsystems, each of which can be adapted with high computational efficiency. The previously proposed SA Hammerstein Group Models (SA-HGMs) decompose the nonlinear acoustic echo path into a direct-path part, modeled by a Hammerstein Group Model (HGM) and a complementary part, modeled by a very efficient Hammerstein model. In this article, we furthermore propose a novel Equalization-based SA (ESA) structure, where the echo path is equalized by a linear filter to allow for an estimation of the loudspeaker nonlinearities by very small and efficient models. Additionally, we provide a novel in-depth analysis of the computational complexity of the previously proposed SA and the novel ESA filters and compare both SA filtering approaches to each other, to adaptive HGMs, and to linear filters, where fast partitioned-block frequency-domain realizations of the competing filter structures are considered. Finally, the echo reduction performance of the proposed SA filtering approaches is verified using real recordings from a commercially available smartphone. Beyond the scope of previous publications on SA-HGMs, the ability of the SA filters to generalize for double-talk situations is explicitly considered as well. The low complexity as well as the good echo reduction performance of both SA filters illustrate the potential of SA filtering in practic
Keywords: Significance-aware; Equalization-based significance-aware; Efficiency; Nonlinear acoustic echo cancellation; Hammerstein group model; Hammerstein model

Significance-aware Hammerstein group models for nonlinear acoustic echo cancellation - Hofmann, Christian; Hümmer, Christian; Kellermann, Walter (2014 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2014)
In this work, a novel approach for nonlinear acoustic echo cancellation is proposed. The main innovative idea of the proposed method is to model only the small region of the echo path around the direct path by a group of parallel Hammerstein models, to estimate a nonlinear preprocessor by correlations between the linear kernels of the Hammerstein submodels, and to describe the remaining echo path by a simple Hammerstein model with the preprocessor determined in the aforementioned way. While the computational complexity of such a system increases only slightly in comparison to a linear echo canceller, experiments with speech recordings from a smartphone in different environments confirm a significantly increased echo cancellation performance. © 2014 IEE
Keywords: HGM; NLAEC; nonlinear AEC; SA-HGM

Simulation and visualization of room compensation for wave field synthesis with the functional transformation method - Petrausch, Stefan; Spors, Sascha; Rabenstein, Rudolf (119th Convention Fall Preprints 2005, 2005)
Active room compensation based on wave field synthesis (WFS) has been recently introduced. So far, the verification of the compensation algorithms is only possible through elaborate acoustical measurements. Therefore, a new simulation method is applied that is based on the functional transformation method (FTM). Compared with other simulation techniques, the FTM provides several advantages that facilitate the correct simulation of the complete wave field particularly in the interesting frequency ranges for WFS. The entire procedure, starting from the virtual "measurements" of the acoustical properties of the simulated room, via the correct excitation for the simulated wave field, towards the resulting animations is presented in this paper.

Simulation linearer dissipativer Wellenausbreitungsvorgänge mit mehrdimensionalen digitalen Filtern - Krauss, Herbert (Friedrich-Alexander-Universität Erlangen-Nürnberg, 1997)

Simulation mehrdimensionaler Systeme mit Hilfe von Funktionaltransformationen - Rabenstein, Rudolf (2. ITG-Diskussionssitzung "Neue Anwendungen theoretischer Konzepte in der Elektrotechnik", 1996)

Simulation of Distributed Parameter Systems by Transfer Function Models - Schäfer, Maximilian (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2020)
Most real-world systems are distributed parameter systems. This means that their dynamics depend not only on their temporal, but also on their spatial behaviour. Particularly, their parameters are not concentrated, but distributed over their spatial volume. Well-investigated examples include the sound of a guitar string, which depends on its spatial oscillation on the guitar, or an electrical transmission line, whose resistance is dependent on its length. Distributed parameter systems also occur naturally in the human body, where the transport of particles through a blood vessel is influenced by its spatial and temporal properties. An abstraction of the real world delivers a comprehensive mathematical description of distributed parameter systems in terms of initial-boundary value problems, which are derived by the first principles of physics. Initial-boundary value problems describe the dynamics of a distributed parameter system by partial differential equations, where its temporal initial state and its spatial boundary behaviour is modelled by suitable initial and boundary conditions. The analysis of natural existing distributed parameter systems as well as the design of synthetic distributed parameter systems are very important tools that may be employed to analyse the dynamics of particle transport in the human body from a communications point of view, or to create a digital guitar synthesizer. Therefore, it is indispensable to obtain suitable models to simulate the spatio-temporal dynamics of distributed parameter systems. The main challenge is to choose a suitable modelling technique which leads to a model that meets the predefined requirements. In the literature, a considerable number of different modelling techniques is found, each with its own advantages and disadvantages. These modelling techniques may be roughly divided into two different categories: numerical methods and analytical methods. Most numerical methods apply a suitable discretization rule to a set of partial differential equations. They lead to powerful simulation algorithms which are capable of simulating spatially complex physical problems in a very accurate way. However, these methods often have a very high computational complexity and provide only little insight into the influence of parameters on the output signal. In contrast to that, analytical methods try to find an explicit solution of an initial-boundary value problem before a discrete algorithm is established. Most analytical methods are based on well-investigated techniques from mathematics and systems theory. The derived models allow to establish a relationship between input and output variables of a system in terms of its parameters. Furthermore, they can lead to low-complexity algorithms with real-time capability by the application of a convenient discretization method. However, the elegance of analytical methods decreases for non-linear systems. Therefore, they are mostly applied to distributed parameter systems which can be described mathematically by linear initial-boundary value problems. The choice of a suitable modelling technique thus depends strongly on the distributed parameter system to be modelled and the requirements on the simulation model. If exact numerical results of a spatially complex distributed parameter system are required, numerical methods are preferable. However, if an exact closed-form description is necessary – for the analysis of a distributed parameter system, for example – analytical methods are advantageous. The modelling procedure used in this thesis is the Functional Transformation Method. This procedure contains diverse functional transformations, i.e., a Laplace and a Sturm-Liouville transformation. Finally, a model is formulated in terms of multidimensional transfer functions. The functional transformation method belongs to the class of analytical modelling techniques. But as most other analytical methods, the functional transformation method is not suitable for complex spatial shapes, non-linear distributed parameter systems and for complex boundary behaviour. Then the method loses its elegance and no explicit solution is obtained as numerical evaluations have to be involved. Nevertheless, analytical methods are a desired approach for the modelling of distributed parameter systems. Therefore, this thesis marks a starting point in overcoming some of the previously mentioned problems of analytical modelling techniques, i.e., of the functional transformation method. By developing suitable extensions it is possible to derive an explicit model of a distributed parameter system which includes the influence of complex boundary behaviour. Although the procedure of the functional transformation method is already formalised, the first goal of this dissertation is to improve its formulation. As an extension, an operator-based version of the involved Sturm-Liouville transformation is incorporated into the functional transformation method. Applying this extension variant to an initial-boundary value problem, a multidimensional state space description is obtained as a solution, which constitutes the model of the underlying distributed parameter system. This formulation as a state space description exhibits several advantages: it constitutes a unified solution of the functional transformation method and allows its analysis and modification by concepts from control and systems theory. The formulation of the simulation model in terms of a state space description is the basis for the second goal of this dissertation. The functional transformation method is extended by adapting concepts from control theory to incorporate the influence of complex boundary behaviour by the design of feedback loops. First, the complex boundary behaviour is separated from the system and it is modelled with a generic simple boundary behaviour which defines the open loop system. The complex boundary behaviour is used to design a feedback matrix which is attached to the simple model to form the closed loop system that fulfils the desired complex boundary behaviour. In particular, the feedback matrix shifts the eigenvalues of the open loop system into a position where they fulfil the complex boundary behaviour. With the developed concept it is possible to model distributed parameter systems with complex boundary behaviour in an explicit form. The same concept can be used to incorporate other physical effects into the model of a distributed parameter system. Furthermore, the concept allows to model interconnected systems, which builds the basis for a block-based modelling approach of interconnected distributed parameter systems. Applying the developed techniques to specific problems from different fields of application, their validity is confirmed in the third part of this dissertation. Specifically, the techniques are employed to model musical systems, electrical transmission lines and biological systems in the context of molecular communications. Within these applications, the developed methods are used to incorporate complex boundary behaviour and physical effects. In addition, general system modifications to change the timbre of a musical system are shown. Furthermore, two biological systems are interconnected by the design of a connection matrix. Where possible, the modelling results are compared to numerical simulations or measurements. All considered problems show that the developed concepts are suitable for the modelling of distributed parameter systems and constitute a meaningful extension to the functional transformation method.<br

Simulation of Highway Traffic for Mobile Communication Systems Design - Ben Younes, Khaled; Rabenstein, Rudolf; Girod, Bernd (Proc. 11th European Simulation Multiconference (ESM 97), 1997)
LMS::431

Simulation of linear continuous systems with distributed parameters - Rabenstein, Rudolf (Simulation Modelling Practice and Theory, 1993)
A new method for the simulation of distributed linear systems is presented. It is based on linear systems theory and digital signal processing rather than numerical mathematics. The essence of the method is the replacement of the Green's function for the continuous problem with the discretized Green's function of a discrete system. Its transfer function can be obtained by functional transformations and leads directly to a realizing structure which also incorporates initial and boundary conditions. The application of the method and its numerical efficiency over existing methods is demonstrated by an example. © 1993.
UnivIS-Import:2015-10-26:Pub.1993.tech.IE.LEN.simula
Keywords: Continuous systems simulation; digital signal processing; distributed parameters; numerical methods; partial differential equations

Simulation of Nonlinear Multidimensional Systems Using Functional Transformations - Rabenstein, Rudolf (European Simulation Symposium, 1995)

Simulation of room acoustics via block-based physical modeling with the functional transformation method - Petrausch, Stefan; Rabenstein, Rudolf (2005 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, 2005)
An application of block-based physical modeling for the simulation of room acoustics with the Functional Transformation Method (FTM) is presented in this paper. In doing so, an analytic solution of the two-dimensional wave equation for rectangular regions achieved with the FTM is employed as a simple block element of the entire model. Due to a recently introduced approach to block-based modeling, it is possible to apply arbitrary boundary conditions for this block, including the connection to other blocks and the simulation of openings. By the connection of several block elements, it is possible to construct and simulate complex regions with complex boundaries. In result a new wave field simulation technique is achieved, that avoids the disadvantages of the FTM, the restriction to simple regions, while preserving its advantages like dispersion-free simulation and exact source and receiver positioning. © 2005 IEEE.

Simulation of three-dimensional sound propagation with multidimensional wave digital filters - Schetelig, Th.; Rabenstein, Rudolf (IEEE Int. Conf. on Acoustics, Speech & Signal Processing (ICASSP), 1998)
This paper describes the derivation of a discrete model for three-dimensional sound propagation according to multidimensional wave digital filtering principles. The correct treatment of boundary conditions for various wall impedances is shown. A numerical example for the sound propagation in three interconnected rooms of a building demonstrates the capabilities of the method.

Simulation of wave propagation by multidimensional digital filters - Krauss, Herbert; Rabenstein, Rudolf; Gerken, Max (Simulation Modelling Practice and Theory, 1996)
The application of multidimensional digital filtering theory to the simulation of continuous systems with distributed parameters has been treated in various recent contributions. Here, two different concepts will be compared, using a lossy transmission line of finite extent as an example. One approach is based on multidimensional wave digital filters, the other on the derivation of a simulating structure by functional transformations for each independent variable. The comparison includes a review of the underlying theoretical concepts as well as the presentation of the design process for the digital system and simulation results for various examples.
Keywords: Computer simulation; Digital filters; Digital signal processing; Wave digital filters; Wave propagation

Simulations of string vibrations with boundary conditions of third kind using the functional transformation method - Trautmann, Lutz; Petrausch, Stefan; Bauer, M. (Journal of the Acoustical Society of America, 2005)
LMS::570

Simultaneous localization of multiple sound sources using blind adaptive MIMO filtering - Buchner, Herbert; Aichner, Robert; Stenglein, J.; Teutsch, Heinz; Kellermann, Walter (2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05, 2005)
Blind adaptive filtering for time delay of arrival (TDOA) estimation is a very powerful method for acoustic source localization in reverberant environments with broadband signals like speech. Based on a recently presented generic framework for blind signal processing for convolutive mixtures, called TRINICON, we present a TDOA estimation method for simultaneous multidimensional localization of multiple sources. Moreover, an interesting link to the known single-input multiple-output (SEVIO)-based adaptive eigenvalue decomposition (AED) method is shown. We evaluate the novel multiple-input multiple-output (MIMO)-based approach and compare it with the known SEVIO-based method in a reverberant acoustic environment using reference data of the positions obtained from infrared sensors. The results show that the new approach is very robust against reverberation and background noise. ©2005 IEEE.

Single-Channel Maximum-Likelihood T60 Estimation Exploiting Subband Information - Löllmann, Heinrich; Brendel, Andreas; Vary, Peter; Kellermann, Walter (ACE Challenge Workshop, 2015)
This contribution presents four algorithms developed by the authors for single-channel fullband and subband T60 estimation within the ACE challenge. The blind estimation of the fullband reverberation time (RT) by maximum-likelihood (ML) estimation based on [15] is considered as baseline approach. An improvement of this algorithm is devised where an energy-weighted averaging of the upper subband RT estimates is performed using either a DCT or 1/3-octave filter-bank. The evaluation results show that this approach leads to a lower variance for the estimation error in comparison to the baseline approach at the price of an increased computational complexity. Moreover, a new algorithm to estimate the subband RT is devised, where the RT estimates for the lower octave subbands are extrapolated from the RT estimates of the upper subbands by means of a simple model for the frequency-dependency of the subband RT. The evaluation results of the ACE challenge reveal that this approach allows to estimate the subband RT with an estimation error which is in a similar range as for the presented fullband RT estimators.
Keywords: blind estimation; fullband reverberation time; subband reverberation time; ACE challenge; ML estimation

Single-channel Signal Features for Estimating Microphone Utility for Coherent Signal Processing - Günther, Michael; Brendel, Andreas; Kellermann, Walter (International Congress on Acoustics (ICA), 2019)
Many microphone array signal processing techniques, e. g., for beamforming or localization, rely on coherent input signals. However, low inter-channel coherence may result from the occlusion of microphones, reverberation, or the presence of undesired signal components, so that the according signals contribute little to the overall algorithmic performance. Thus, ranking the microphone channels by their utility for subsequent coherent signal processing schemes is of considerable interest. Direct estimation of the pair-wise coherence is often straight-forward in compact microphone arrays when all microphones share a common sampling clock, while wireless acoustic sensor networks require a potentially costly time synchronization of the microphone signals. In this case, estimating the channel utility ranking from simpler, single-channel features, e. g., statistical moments of the time-domain signal waveform or of the corresponding magnitude spectrum instead of the signal coherence, facilitates the clustering of useful sensor nodes for a particular task. Thereby, it is possible to determine whether it is worth the effort to synchronize the sensor signals in a sensor network, and thus save computational power and data rate if this is not the case. In this contribution, we investigate the efficacy of different single-channel signal features for determining candidate sets of sensors signals for synchronization in sensor networks.<br
Keywords: Microphone utility; Channel selection; Sensor network

Skalierbare Videocodierung mit einer bewegungskompensierten örtlich-zeitlichen Auflösungspyramide in Verbindung mit E8-Gitter-Quantisierung - Horn, Uwe (Friedrich-Alexander-Universität Erlangen-Nürnberg, 1998)

Soliton sonification-experiments with the Korteweg-de Vries equation - Rabenstein, Rudolf (15th International Conference on Digital Audio Effects, DAFx 2012, 2012)
Solitons are special solutions of certain nonlinear partial differential equations of mathematical physics. They exhibit properties that are partly similar to the solutions of the linear wave equation and partly similar to the behaviour of colliding particles. Their characteristic features are well-known in the mathematical literature but few closed-form solutions are available. This contribution derives algorithmic structures for the computation of solitons in a dimensionless space-time domain which can be scaled to the audio frequency range. The investigations are confined to first and second order solutions of the Korteweg-de Vries equation. Sound examples show that the effects of wave propagation and soliton interaction can be represented by audible events.

Solution of general vector partial differential equations by multidimensional transform methods - Dymkou, Vitali; Rabenstein, Rudolf; Steffen, Peter (4th International Workshop on Multidimensional Systems, NDS 2005, 2005)
The multi-functional transformation model for the description of physical systems has recently been introduced to the field of discrete simulation of continuous systems. The multi-functional transformation is applied to physical systems which are described by partial differential equations, typically for the time and space variables. It allows to expand the solution of a initial-boundary-value problem in terms of its eigen-functions and associated functions. This expansion serves as the starting point for the development of efficient discrete algorithms that are suitable for computer implementation. The purpose of this article is to extend the presented physical model to singular initial-boundary-value problems.

Solution of vector partial differential equations by transfer function models - Rabenstein, Rudolf; Trautmann, Lutz (Proceedings of the 1999 IEEE International Symposium on Circuits and Systems, ISCAS '99, 1999)
Transfer function models for the description of physical systems have recently been introduced to the field of multidimensional digital signal processing. They provide an alternative to the conventional representation by partial differential equations (PDE) and are suitable for computer implementation. This paper extends the transfer function approach to vector PDEs. They arise from the physical analysis of multidimensional systems in terms of potential and flux quantities. Expressing the resulting coupled PDEs in vector form facilitates the direct formulation of boundary and interface conditions in their physical context. It is shown how a carefully constructed transformation for the space variable leads to transfer function models for vector PDEs. They are the starting point for the derivation of discrete models by standard methods for one-dimensional systems. The presented functional transformation approach is suitable for a number of technical applications, like electromagnetics, optics, acoustics and heat and mass transfer.

Some Contributions to Adaptive Filtering for Acoustic Multiple-Input/Multiple-Output Systems in the Wave Domain - Schneider, Martin (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2016)
Recently emerging techniques like wave field synthesis (WFS) or Higher-Order Ambisonics (HOA) allow for high-quality spatial audio reproduction, which makes them candidates for the audio reproduction in future telepresence systems or interactive gaming environments with acoustic human-machine interfaces. In such scenarios, acoustic echo cancellation (AEC) will generally be necessary to remove the loudspeaker echoes in the recorded microphone signals before further processing. Moreover, the reproduction quality of WFS or HOA can be improved by adaptive pre-equalization of the loudspeaker signals, as facilitated by listening room equalization (LRE). However, AEC and LRE require adaptive filters, where the large number of reproduction channels of WFS and HOA imply major computational and algorithmic challenges for the implementation of adaptive filters. A technique called wave-domain adaptive filtering (WDAF) promises to master these challenges. However, known literature is still far away from providing sufficient insight to allow for a successful implementation of real-world systems. This thesis is concerned with the further development of WDAF-based generic signal processing algorithms and acoustic models aiming at real-time, real-world implementations of AEC and LRE. As prototypical scenario, an exemplary loudspeaker and microphone setup is considered for which the necessary wave-domain transforms are explicitly derived and analyzed. Thereby, the origins of the desirable wave-domain properties of the loudspeaker-enclosure-microphone system (LEMS) are explained. For both, AEC and LRE, it is necessary to identify an LEMS, while the computational demands of this task render a real-time implementation unrealistic, if a large number of reproduction channels should be considered without approximative models. The originally proposed approximative wave-domain LEMS model is generalized such that the number of degrees of freedom can be chosen to provide the maximum model accuracy given the applicable computational restrictions. Typical reproduction signals will often not allow to find a unique solution to the system identification problem for multichannel reproduction. A novel, rigorous and in-depth analysis of this so-called nonuniqueness problem is conducted in this thesis. Furthermore, a wave-domain technique to improve the system identification when nonuniqueness occurs is presented. This technique does not influence the reproduced signals, as it would be the case for other known state-of-the-art solutions. For an implementation of adaptive filters in the wave domain, modified versions of well-known adaptation algorithms are derived, considering approximative models and improving system identification. The modified algorithms are based on the least mean squares (LMS) algorithm, the affine projection algorithm (APA), the recursive least squares (RLS) algorithm, or the generalized frequency-domain adaptive filtering (GFDAF) algorithm, which is identified as an approximation of the RLS algorithm. Additionally, a novel iterative algorithm for the determination of equalizers is presented. Experimental results support the claim of applicability of the considered approach. Moreover, a real-time wave-domain AEC demonstrator has been developed which facilitates AEC for 48 loudspeaker channels on a conventional personal computer. On the other hand, the equalizer determination necessary for LRE is an inverse system identification problem. It is shown that the nonuniqueness problem occurs also for this task while the properties of the LEMS can additionally cause nonuniqueness. In this thesis, a generalized wave-domain approximative equalizer structure is successfully applied, which potentially allows for a real-time implementation of LRE. The achievable LRE performance with an approximative wave-domain system is assessed by simulations. Decisive problems for the real-word implementation could be solved or mitigated using results presented in this thesis. Nevertheless, other challenging research questions remain unanswered and will fuel future research in this are
Keywords: adaptive filtering; wave domain; acoustic echo cancellation; listening room equalization; listening room compensation; multichannel; mimo systems; WDAF

Some current challenges in multichannel acoustic signal processing - Kellermann, Walter (4th Joint Meeting of Acoustical Society of America and Acoustical Society of Japan,JASA, 2006)
Hands‐free human/machine communication involves major challenges for today’s state of the art signal processing if the humans should not be forced to carry or wear any technical gear and should be free to move. The spatial domain sampled by multiple loudspeakers for reproduction and multiple microphones for signal acquisition allows for spatialized listening experiences and for spatially selective acoustic scene analysis, respectively. Along with the possibility to serve several human communication partners simultaneously, such systems also introduce the need for localization algorithms. Three main types of impairment have to be addressed by signal processing algorithms to fulfill the promises of the general concept: first, acoustic feedback from the loudspeakers into the microphones; second, ambient noiseand undesired pointlike interferers; and, finally, echoes and reverberation as picked up by the human ear when listening to the loudspeaker signals or as recorded by the microphonesthat actually should record only the direct‐path component of the source signal. Accordingly, some of the current challenges in multichannel acoustic echo cancellation, in spatially selective noise and interference suppression, in source separation and localization, and in dereverberation are investigated. © 2019 AIP Publishing LLC <br
LMS::677

Some results on the objective assessment of speech quality for speech enhancement algorithms - Bürger, Michael; Reindl, Klaus; Nordholm, S.; Kellermann, Walter (AIA-DAGA Conference on Acoustics, 2013)

Sound Capture for Human/Machine Interfaces - Practical Aspects of Microphone Array Signal Processing - Herbordt, Wolfgang (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2005)

Sound Field Reproduction - Rabenstein, Rudolf; Spors, Sascha (Springer Handbook of Speech Processing, 2008)
Multichannel sound field reproduction aims at the physically correct synthesis of acoustical wave fields with a large number of loudspeakers. It goes beyond stereophony by extending or eliminating the so-called sweet spot. This chapter presents the physical and mathematical description of various methods for this purpose and presents the physical and mathematical background of several methods for multichannel sound field reproduction. ©Spring

Sound field simulation by computational acoustics. Part I: simulation algorithm - Rabenstein, Rudolf; Zayati, A. (International Journal of Adaptive Control and Signal Processing, 2000)
We present a computational acoustics approach to the computer simulation of sound fields. It is based on physical modelling of sound wave propagation in terms of the partial differential equations for sound pressure and velocity. A step-by-step procedure is given to convert the partial differential equation description into a suitable form for numerical integration. The resulting discrete model is derived with methods from numerical mathematics, linear algebra, and multidimensional system theory and formulated in terms of a state-space representation. The treatment of boundary conditions and examples are presented in a companion paper.

Sound Field Synthesis - Spors, Sascha; Teutsch, Heinz; Kuntz, Achim; Rabenstein, Rudolf (Audio Signal Processing for Next-Generation Multimedia, 2004)

Sound Field Synthesis, Chapter 32 - Rabenstein, Rudolf; Spors, Sascha; Ahrens, Jens (Academic Press Library in Signal Processing, 2014)

Sound Field Synthesis for Audio Presentation - Ahrens, Jens; Rabenstein, Rudolf; Spors, Sascha (Acoustics Today, 2014)

Sound speed estimation from Time of Arrivals: Derivation and comparison with TDOA-based estimation - Annibale, Paolo; Rabenstein, Rudolf (20th European Signal Processing Conference, EUSIPCO 2012, 2012)
In this paper we propose a method for the estimation of the propagation speed of sound waves from Time Of Arrivals (TOAs) observed at a sensor array. In our previous work we presented a speed estimation method based on Time Differences of Arrival (TDOAs), here we show that the same approach may be applied successfully to the TOA-based problem. The exploitation of TOA measurements is driven by applications where synchronization between source and receiver is available. The proposed method provides an estimate of the average propagation speed of the wave along the source-receiver paths. Such a speed estimate is evaluated and compared with the TDOA-based estimate by means of Cramer-Rao Bound analysis and simulations. © 2012 EURASI
Keywords: Propagation speed; source localization; speed of sound; time of arrival; TOA

Sound Synthesis and Spatial Reproduction by Physical Modeling - Müller, A.; Rabenstein, Rudolf (Proc. NAG/DAGA International Conference on Acoustics, 2009)

Sound Synthesis by Physical Modeling using the Functional Transformation Method: Efficient Implementation with Polyphase Filterbanks - Petrausch, Stefan; Rabenstein, Rudolf (Int. Conf. on Digital Audio Effects, DAFx 03, 2003)
The Functional Transformation Method (FTM) is a recently introduced method for sound synthesis by physical modeling. Based on integral transformations, it provides a parallel system description for any linear physical model, usually described by a set of partial differential equations. Such parallel descriptions can be directly implemented by a set of recursive systems in full rate. In this paper we present a new and very efficient method for this implementation which benefits from the spectral decomposition of the system. All recursive systems are working at a subsampled rate and are summed up by the application of a polyphase filterbank. Performance measurements on a real time implementation show, that a flexible and efficient realization is achieved. Compared to the direct implementation it is over nine times faster at the cost of nine milliseconds of delay and even faster with more delay.

Sound synthesis with tension modulated nonlinearities based on functional transformations - Trautmann, L; Rabenstein, Rudolf (Acoustics and Music: Theory and Applications (AMTA), 2000)
Transfer functions for digital sound synthesis based on physical models have recently been presented. The method transforms a continuous model for the vibrating body, given by a partial differential equation (PDE), into a multidimensional (MD) transfer function model (TFM). The TFM takes initial and boundary conditions, as well as excitation functions into account. It also treats the physical effects modeled by the PDE exactly. The algorithms obtained after discretization of the TFM preserve the inherent physical stability and are suitable for real-time implementations on digital signal processors. A recently presented example of a linear transversal oscillating tightened string with frequency dependent loss terms is extended here to a nonlinear vibration model. The nonlinearity of the string vibration is caused by an output dependent tension modulation. Because of the nonlinearity we only obtain an implicit equation instead of a TFM derived in the linear case. The nonlinear string model can easily be implemented after inverse transformations and discretization. This model reproduces the vibrations of a real string better than the linear model. An example of the nonlinear string model and a comparison between the results of the linear and the nonlinear model is given.
Keywords: Functional Transformations; Nonlinear; Sound Synthesis; Vibrating string

Source-specific system identification - Hofmann, Christian; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2016)
Many applications in audio communication require the identification of Loudspeaker-Enclosure-Microphone Systems (LEMS) with multiple inputs and outputs. The according computational complexity typically grows at least proportionally along the number of acoustic paths, which is the product of the number of loudspeakers and the number of microphones. Furthermore, the typical, highly correlated loudspeaker signals preclude an exact identification of the LEMS. To this end, a novel system identification scheme employing prior information from an object-based rendering system, e.g., Ambisonics [1,2] or Wave Field Synthesis (WFS) [3,4], is proposed. In this scheme, only a source-specific system from each virtual source to each microphone is identified adaptively and uniquely. This estimate for a source-specific system can then be transformed into a statistically optimal estimate of the LEMS, which could have been found by a computationally expensive direct LEMS estimation as well. The basic concept is extended to time-varying acoustic scenes and simulations of a WFS application confirm the validity of this novel approach for system identification.
Keywords: AEC; system identification; reduced-complexity; subspace system identification; side information

Sparse Adaptation of Distributed Blind Source Separation in Acoustic Sensor Networks - Günther, Michael; Afifi, Haitham; Brendel, Andreas; Karl, Holger; Kellermann, Walter (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2019)
By distributing the computational load over the nodes of a Wireless Acoustic Sensor Network (WASN), the real-time capability of the TRINICON (TRIple-N-Independent component analysis for CON- volutive mixtures) framework for Blind Source Separation (BSS) can be ensured, even if the individual network nodes are not powerful enough to run TRINICON in real-time by themselves. To optimally utilize the limited computing power and data rate in WASNs, the MARVELO (Multicast-Aware Routing for Virtual network Embedding with Loops in Overlays) framework is expanded for use with TRINICON, while a feature-based selection scheme is proposed to exploit the most beneficial parts of the input signal for adapting the demixing system. The simulation results of realistic scenarios show only a minor degradation of the separation performance even in heavily resource-limited situations. <br
Keywords: Acoustic Sensor Network; Blind Source Separation; Distribution

Sparse Hartley Modeling for Fast Image Extrapolation - Genser, Nils; Grosche, Simon; Seiler, Jürgen; Kaup, André (IEEE 20th International Workshop on Multimedia Signal Processing (MMSP), 2018)

Sparse representation of dense motion vector fields for lossless compression of 4-D medical CT data - Weinlich, Andreas; Amon, P.; Hutter, A.; Kaup, André (19th European Signal Processing Conference, EUSIPCO 2011, 2011)
We present a new method for data-adaptive compression of dense vector fields in dynamic medical volume data. Conventional block-based motion compensation used for temporal prediction in video compression cannot conveniently cope with deformable motion typically found in medical image sequences encoded over time. Based on an approximation of physiologic tissue motion between two succeeding slices in time direction computed by optical flow methods, we find the most significant motion vectors with respect to their prediction capability for a second 2-D slice out of the first one. By coding the components of these vectors, we are able to reconstruct a high quality dense motion vector field at the decoder using only minimal side-information. We show that our approach can achieve a smoother prediction than blockbased motion compensation for such data, reducing storage demands in spatially predictive lossless compression. We also show that such a predictive approach can yield better compression ratios than JPEG 2000 intra coding. © 2011 EURASI

Sparse Signal Reconstruction with Multiple Side Information using Adaptive Weights for Multiview Sources - van Luong, Huynh; Seiler, Jürgen; Kaup, André; Forchhammer, Soren (IEEE Int. Conf. on Image Processing (ICIP), 2016)
This work considers reconstructing a target signal in a context of distributed sparse sources. We propose an efficient reconstruction algorithm with the aid of other given sources as multiple side information (SI). The proposed algorithm takes advantage of compressive sensing (CS) with SI and adaptive weights by solving a proposed weighted n-ℓ 1 minimization. The proposed algorithm computes the adaptive weights in two levels, first each individual intra-SI and then inter-SI weights are iteratively updated at every reconstructed iteration. This two-level optimization leads the proposed reconstruction algorithm with multiple SI using adaptive weights (RAMSIA) to robustly exploit the multiple SIs with different qualities. We experimentally perform our algorithm on generated sparse signals and also correlated feature histograms as multiview sparse sources from a multiview image database. The results show that RAMSIA significantly outperforms both classical CS and CS with single SI, and RAMSIA with higher number of SIs gained more than the one with smaller number of SI
Keywords: side information; compressive sensing; sparse signal; adaptive weights

Sparse Signal Recovery with Multiple Prior Information: Algorithm and Measurement Bounds - van Luong, Huynh; Deligiannis, Nikos; Seiler, Jürgen; Forchhammer, Soren; Kaup, André (Signal Processing, 2018)
We address the problem of reconstructing a sparse signal from compressive measurements with the aid of multiple known correlated signals. We propose a reconstruction algorithm with multiple side information signals (RAMSI), which solves an n − l1 minimization problem by weighting adaptively the multiple side information signals at every iteration. In addition, we establish theoretical bounds on the number of measurements required to guarantee successful reconstruction of the sparse signal via weighted n − l1 minimization. The analysis of the derived bounds reveals that weighted n − l1 minimization can achieve sharper bounds and significant performance improvements compared to classical compressed sensing (CS). We evaluate experimentally the proposed RAMSI algorithm and the established bounds using numerical sparse signals. The results show that the proposed algorithm outperforms state-of-the-art algorithms—including classical CS, l1-l1 minimization, Modified-CS, regularized Modified-CS, and weighted l1 minimization—in terms of both the theoretical bounds and the practical performance.<br

Sparsity-based defect pixel compensation for arbitrary camera raw images - Schöberl, Michael; Seiler, Jürgen; Kasper, B.; Foessel, S.; Kaup, André (36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 2011)
In high quality imaging even tiny distortions as small as a single pixel are visible and can not be accepted. Although the production quality of CMOS image sensors is very high, for reasonable yields we still need to accept some defect pixels and clusters of defects in large image sensors. In this paper we will compare compensation algorithms for raw image sensor data. We propose a new approach based on the sparsity assumption that outperforms existing defect compensation algorithms. Furthermore, our proposed interpolation algorithm is universal and not at all adapted to Bayer pattern images. It can directly be applied to any regular color filter pattern or gray scale image. Our examples show, that image sensors with large clusters of defects can still be used for the generation of high quality images. © 2011 IEE
Keywords: defect pixels; Digital cameras; error concealment; interpolation

Spatial aliasing artifacts produced by linear and circular loudspeaker arrays used for wave field synthesis - Spors, Sascha; Rabenstein, Rudolf (120th Convention Spring Preprints 2006, 2006)
Wave field synthesis allows the exact reproduction of sound fields if the requirements of its physical foundation are met. However, the practical realization imposes certain technical constraints. One of these is the application of loudspeaker arrays as an approximation to a spatially continuous source distribution. The effect of a finite spacing of the loudspeakers can be described as spatial sampling artifacts. This contribution derives a description of the spatial sampling process for planar linear and circular arrays, analyzes the sampling artifacts and discusses the conditions for preventing spatial aliasing. It furthermore introduces the reproduced aliasing-to-signal ratio as a measure for the energy of aliasing contribution

Spatial aliasing artifacts produced by linear loudspeaker arrays used for wave field synthesis - Spors, Sascha (Second IEEE-EURASIP International Symposium on Control, Communication, and Signal Processing, 2006)
Spatial sound reproduction systems with a large number of loudspeakers are increasingly being used. Wave field synthesis is a reproduction system using a large number of densely placed loudspeakers (loudspeaker array). This implies a spatial sampling process that may lead to aliasing artifacts. This paper derives the spatial aliasing artifacts of linear loudspeaker arrays used for wave field synthesis and an antialiasing conditio
LMS::682

Spatial diffuseness features for DNN-based speech recognition in noisy and reverberant environments - Schwarz, Andreas; Hümmer, Christian; Maas, Roland; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2015)
We propose a spatial diffuseness feature for deep neural network (DNN)-based automatic speech recognition to improve recognition accuracy in reverberant and noisy environments. The feature is computed in real-time from multiple microphone signals without requiring knowledge or estimation of the direction of arrival, and represents the relative amount of diffuse noise in each time and frequency bin. It is shown that using the diffuseness feature as an additional input to a DNN-based acoustic model leads to a reduced word error rate for the REVERB challenge corpus, both compared to logmelspec features extracted from noisy signals, and features enhanced by spectral subtractio
Keywords: Deep Neural Networks; Diffuse Noise; Reverberation; Speech Recognition

Spatial Error Concealment Based on Edge Visual Clearness for Image/Video Communication - Koloda, Jan; Sánchez, V.; Peinado, A.M. (Circuits Systems and Signal Processing, 2013)
In this paper, we propose a technique for concealing missing image/video blocks based on the concept of visual clearness of an edge. A scanning procedure based on the Hough transform allows us to find the relevant edges, and the visually clearest ones are employed in an interpolation based reconstruction. Specifically, several interpolations are combined according to a set of weights which allows the reconstruction of more complex textures. These weights are derived from the visual clearness associated to an edge and are unique for every pixel within the missing macroblock. The resulting algorithm is quite efficient, simple, and competitive in comparison with other state-of-the-art technique
LMS::1534
Keywords: error concealment; directional interpolation; block-coded image; block-coded video; hough transform

Spatial error concealment of corrupted image data using frequency selective extrapolation - Meisinger, Katrin; Kaup, André (IEEE International Conference on Acoustics, Speech, and Signal Processing, 2004)
This paper introduces a method for spatial error concealment of lost image data in erroneous image transmission. The image content of the correctly received surrounding blocks is successively approximated by a weighted linear combination of basis functions and the missing block is obtained by extrapolation. An implementation in the frequency domain allows an efficient realization. Investigations show that particularly 2D DFT basis functions are suited for the signal extrapolation in order to be able to reconstruct both monotone areas and edge

Spatial filtering - Markovich-Golan, Shmulik; Kellermann, Walter; Gannot, Sharon (Audio Source Separation and Speech Enhancement, 2018)

Spatially-Adaptive Learning-Based Image Compression with Hierarchical Multi-Scale Latent Spaces - Brand, Fabian; Kopte, Alexander; Fischer, Kristian; Kaup, André (2023 IEEE International Conference on Image Processing (ICIP), 2023)
Adaptive block partitioning is responsible for large gains in current image and video compression systems. This method is able to compress large stationary image areas with only a few symbols, while maintaining a high level of quality in more detailed areas. Current state-of-the-art neural-network-based image compression systems however use only one scale to transmit the latent space. In previous publications, we proposed RDONet, a scheme to transmit the latent space in multiple spatial resolutions. Following this principle, we extend a state-of-the-art compression network by a second hierarchical latent-space level to enable multi-scale processing. We extend the existing rate variability capabilities of RDONet by a gain unit. With that we are able to outperform an equivalent traditional autoencoder by 7% rate savings. Furthermore, we show that even though we add an additional latent space, the complexity only increases marginally and the decoding time can potentially even be decreased.<br
Keywords: Image coding; Runtime; Symbols; Video compression; Distortion; Complexity theory; Decoding; Image Compression; Rate Distortion Optimization; Autoencoder; Multi Scale

Spatially Constrained vs. Unconstrained Filtering in Neural Spatiospectral Filters for Multichannel Speech Enhancement - Briegleb, Annika; Kellermann, Walter (2024 32nd European Signal Processing Conference (EUSIPCO), 2024)
When using artificial neural networks for multichannel speech enhancement, filtering is often achieved by estimating a complex-valued mask that is applied to all or one reference channel of the input signal. The estimation of this mask is based on the noisy multichannel signal and, hence, can exploit spatial and spectral cues simultaneously. While it has been shown that exploiting spatial and spectral cues jointly is beneficial for the speech enhancement result, the mechanics of the interplay of the two inside the neural network are still largely unknown. In this contribution, we investigate how two conceptually different neural spatiospectral filters (NSSFs) exploit spatial cues depending on the training target signal and show that, while one NSSF always performs spatial filtering, the other one is selective in leveraging spatial information depending on the task at hand. These insights provide better understanding of the information the NSSFs use to make their prediction and, thus, allow to make informed decisions regarding their design and deploymen
Keywords: spatial filtering; multichannel speech enhancement; DNN interpretability

Spatially Guided Independent Vector Analysis - Brendel, Andreas; Haubner, Thomas; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020)

Spatially Informed Independent vector analysis for Source Extraction based on the convolutive Transfer Function Model - Wang, Xianrui; Brendel, Andreas; Huang, Gongping; Yang, Yichen; Kellermann, Walter; Chen, Jingdong (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2023)
Spatial information can help improve source separation performance. Numerous spatially informed source extraction methods based on the independent vector analysis (IVA) have been developed, which can achieve reasonably good performance in non- or weakly reverberant environments. However, the performance of those methods degrades quickly as the reverberation increases. The underlying reason is that those methods are derived based on the multiplicative transfer function model with a rank-1 assumption, which does not hold true if reverberation is strong. To circumvent this issue, this paper proposes to use the convolutive transfer function (CTF) model to improve the source extraction performance and develop a spatially informed IVA algorithm. Simulations demonstrate the efficacy of the developed method even in highly reverberant environments.
CRIS-Team Scopus Importer:2024-12-16
Keywords: convolutive transfer function; Independent vector analysis; spatially informed source extraction

Spatially refined inter-sequence error concealment for a multi-broadcast receiver using frequency selective approximation - Tröger, Tobias; Seiler, Jürgen; Kaup, André (18th ACM International Conference on Multimedia ACM Multimedia 2010, MM'10, 2010)
Mobile reception of digital TV often suffers from severe signal degradations. Inter-sequence error concealment reconstructs lost image blocks of a distorted high-resolution TV signal by inserting corresponding error-free blocks from a low-resolution reference TV signal. It is well-suited for application in future automotive multi-broadcast receivers and can outperform state-of-the-art methods by up to 15 dB PSNR depending on the quality of the reference signal. In this contribution, we show that inter-sequence error concealment can be improved by approximating inserted blocks jointly with neighboring pixels according to a well-known frequency selective method. The reconstruction quality can be significantly increased especially in case of low-bitrate reference signals. Inserted blocks can be further refined also for high bitrates. On average, a gain of 1.7 dB PSNR can be achieved. The peak gain which is evaluated on frame basis even reaches 5.6 dB PSNR. © 2010 AC
Keywords: error concealment; image registration; signal approximation

Spatial noise-field control with online secondary path modeling: A wave-domain approach - Zhang, Wen; Hofmann, Christian; Bürger, Michael; Abhayapala, Thushara D.; Kellermann, Walter (IEEE/ACM Transactions on Audio, Speech and Language Processing, 2018)
Due to strong interchannel interference in multichannel active noise control (ANC), there are fundamental problems associated with the filter adaptation and online secondary path modeling remains a major challenge. This paper proposes a wave-domain adaptation algorithm for multichannel ANC with online secondary path modelling to cancel tonal noise over an extended region of two-dimensional plane in a reverberant room. The design is based on exploiting the diagonal-dominance property of the secondary path in the wave domain. The proposed wave-domain secondary path model is applicable to both concentric and nonconcentric circular loudspeakers and microphone array placement, and is also robust against array positioning errors. Normalized least mean squares-type algorithms are adopted for adaptive feedback control. Computational complexity is analyzed and compared with the conventional time-domain and frequency-domain multichannel ANCs. Through simulation-based verification in comparison with existing methods, the proposed algorithm demonstrates more efficient adaptation with low-level auxiliary nois
Keywords: Active noise control; secondary path modeling; wave domain; adaptive processing; feedback system

Spatial scalable JPEG2000 transcoding and tracking of regions of interest for video surveillance - Quast, Katharina; Kaup, André (13th International Fall Workshop Vision, Modeling, and Visualization 2008, VMV 2008, 2008)
This paper presents a method for spatial scalable region of interest transcoding of JPEG2000 coded video frames for video surveillance systems. Based on a user defined ROI the method transcodes HD frames into images in moderate resolution with a ROI in HD resolution. The transcoder extracts all packets belonging to the ROI or the lower resolution levels of the background from the JPEG2000 bitstream. Non-ROI packets of higher resolution levels are replaced by empty packets. The ROI is tracked using a mean shift algorithm to guarantee that always the correct image details are extracted in high resolution. Since the transcoding is performed by extracting and replacing packets of the codestream, an expensive re-encoding of the code-stream is not required. Thus, the transcoding technique is of low complexity and shows a short processing time. Combining the transcoding technique with mean shift tracking leads to a powerful video transcoding techniqu

Spatial scalable region of interest transcoding of JPEG2000 for video surveillance - Quast, Katharina; Kaup, André (IEEE 5th International Conference on Advanced Video and Signal Based Surveillance, AVSS 2008, 2008)
A new method for spatial scalable region of interest transcoding of JPEG2000 images for video surveillance applications is proposed in this paper. The method transcodes HD images into quarter HD images with a ROI in HD resolution. The transcoding method is based on the precinct feature as well as on the empty packet feature of the JPEG2000 standard. Based on a user defined ROI the transcoder extracts all packets which belong to the ROI or to the lower resolution of the background. All non-ROI packets of higher resolution levels are deleted and replaced by empty-packets. Thus, the extracted code stream contains no high frequency information for image regions outside the ROI. Because the transcoder operates on packets and not on codeblocks an expensive re-encoding of the codestream is not required. This leads to a short processing time and a low complexity of the transcoding algorithm. Therefore, our method is appropriate for real-time applications like video surveillance. © 2008 IEEE.
Keywords: JPEG 2000; Transcoding; Image Coding; Region of Interest

Spatial Sound Reproduction with Wave Field Synthesis - Rabenstein, Rudolf; Spors, Sascha (Atti del Congresso 2005, 2005)
LMS::585

Spatial sound synthesis for circular membranes - Rabenstein, Rudolf (13th International Conference on Digital Audio Effects, DAFx 2010, 2010)
Physical models of real or virtual instruments are usually only exploited for the generation of wave forms. However, models of twoand three-dimensional vibrating structures contain also information about the sound radiation into the free field. This contribution presents a model for a membrane from which the required driving functions for a multichannel loudspeaker array are derived. The resulting sound field reproduces not only the musical timbre of the sounding body but also its spatial radiation characteristics. It is suitable for real-time synthesis without pre-recorded or presynthesized source track

Spatio-bi-temporal error concealment in block-based video decoding systems - Friebe, Markus; Kaup, André (2006 IEEE 8th Workshop on Multimedia Signal Processing, MMSP 2006, 2006)
In this paper we present a spatio-bi-temporal fading scheme for block loss recovery in block-based video decoding systems. In the first part of the algorithm, based on two different boundary error criterions obtained from bi-temporal error concealment, either the previous, the future, or fading between both temporal methods is used for bi-temporal macroblock estimation. A weighted absolute difference between motion compensated image samples and macroblock boundary samples of the current frame represents one boundary error. In the second part of the algorithm, based on a boundary error criterion obtained from bi-temporal concealment, spatial, bi-temporal, or fading between both methods is used for recovering a lost macroblock. The advantage of this method is that one lost macroblock can be recovered pelwise spatially from the current or bi-temporally from the previous and the future frame by weighted averaging both error concealment results. The simulation results have shown that for recovering a lost macroblock this method outperforms the reference methods both in subjective and objective video quality. © 2006 IEE

Spatio-spectral Image Reconstruction Using Non-local Filtering - Sippel, Frank; Seiler, Jürgen; Kaup, André (2021 IEEE International Conference on Visual Communications and Image Processing (VCIP), 2021)

Spatio-Temporal Adaptive Inverse Filtering in the Wave Domain - Spors, Sascha; Buchner, Herbert; Rabenstein, Rudolf (Speech and Audio Processing in Adverse Environments, 2008)

Spatio-temporal concealment in H.264/AVC video coding by 3-D selective extrapolation - Meisinger, Katrin; Martin, S.; Kaup, André (2006 IEEE 8th Workshop on Multimedia Signal Processing, MMSP 2006, 2006)
In this contribution, we derive a spatio-temporal extrapolation method for 3-D signals. Lost areas in video signals caused by transmission errors in video communications are concealed by extrapolating the surrounding video signal into the missing area. The method is integrated into the H.264/AVC coder as concealment feature. By exploiting spatial and temporal correlations at the same time, it is possible to inherently compensate motion. Deviations in luminance occurring from frame to frame can be compensated, too. Since the spatio-temporal selective extrapolation method does not rely on motion vectors, the algorithm can be also easily applied to Intra concealment by additionally exploiting temporal correlations. Further, the frequency selective extrapolation shows especially convincing results in connection with Flexible Macroblock Ordering (FMO) causing a chequerboard like loss pattern in case of transmission errors. © 2006 IEE

Spatio-temporal defect pixel interpolation using 3-D frequency selective extrapolation - Meisinger, Katrin; Aach, T.; Kaup, André (14th IEEE International Conference on Image Processing, ICIP 2007, 2007)
Flat panel X-ray detectors allow the immediate availability of the acquired images for display. However, they provide images with defective areas amongst others due to manufacturing problems. In this contribution, we present a frequency selective extrapolation method in order to restore these defects by extending the surrounding signal into the defective area. In case of static radiographs, the spatial surrounding is evaluated by a 2-D approach. Defects in sequences acquired by cine-angiography or fluoroscopy are processed by 3-D extrapolation. The defects are replaced by extrapolating the signal from the spatial and at the same time temporal surrounding, taking previous and subsequent frames into account. Hence, inherent motion compensation is accomplished. The application of the spatial and where applicable spatio-temporal extrapolation approach allows to restore smooth areas, edges, patterns as well as noise. The ability to restore noise is especially important for medical images because it leads to a natural appearance of the concealed defective areas. ©2007 IEE
Keywords: Defect interpolation; Medical imaging; Signal extrapolation

Spatio-temporal error concealment in video by denoised temporal extrapolation refinement - Seiler, Jürgen; Schöberl, Michael; Kaup, André (2013 20th IEEE International Conference on Image Processing, ICIP 2013, 2013)
In video communication, the concealment of distortions caused by transmission errors is important for allowing for a pleasant visual quality and for reducing error propagation. In this article, Denoised Temporal Extrapolation Refinement is introduced as a novel spatiotemporal error concealment algorithm. The algorithm operates in two steps. First, temporal error concealment is used for obtaining an initial estimate. Afterwards, a spatial denoising algorithm is used for reducing the imperfectness of the temporal extrapolation. For this, Non-Local Means denoising is used which is extended by a spiral scan processing order and is improved by an adaptation step for taking the preliminary temporal extrapolation into account. In doing so, a spatio-temporal error concealment results. By making use of the refinement, a visually noticeable average gain of 1 dB over pure temporal error concealment is possible. With this, the algorithm also is able to clearly outperform other spatio-temporal error concealment algorithms. © 2013 IEE
Keywords: Error Concealment; Signal Extrapolation; Video Communication

Spatio-temporal fading scheme for error concealment in block-based video decoding systems - Friebe, Markus; Kaup, André (2006 IEEE International Conference on Image Processing, ICIP 2006, 2006)
In this contribution we propose a new spatio-temporal fading scheme for block loss recovery in block-based video decoding systems. Based on a boundary error criterion obtained from temporal error concealment, either spatial, temporal, or fading of both methods is used for recovering lost image samples in one macroblock. The weights for fading are interpolated from the boundary error. A weighted absolute difference between well received macroblock boundary samples from the current frame and motion compensated macroblock boundary samples from the previous frame represents the boundary error. It is shown, that in case of transmission errors this method can successfully be used for block loss recovery. ©2006 IEE
Keywords: Error concealment; Video coding and transmission; Video restoration

Spatio-temporal prediction in video coding by best approximation - Seiler, Jürgen; Lakshman, Haricharan; Kaup, André (2009 Picture Coding Symposium, PCS 2009, 2009)
Within the scope of this contribution we propose a novel efficient spatio-temporal prediction algorithm for video coding. The algorithm operates in two stages. First, motion compensation is performed on the block to be predicted in order to exploit temporal correlations. Afterwards, in order to exploit spatial correlations, this preliminary estimate is spatially refined by forming a joint model of the motion compensated block and spatially adjacent already decoded blocks. Compared to an earlier refinement algorithm, the novel one only needs very few iterations, leading to a speedup of factor 17. The implementation of this new algorithm into the H.264/AVC leads to a maximum reduction in data rate of up to nearly 13% for the considered sequence
Keywords: Extrapolation; Prediction; Video coding

Spatio-temporal prediction in video coding by non-local means refined motion compensation - Seiler, Jürgen; Richter, Thomas; Kaup, André (28th Picture Coding Symposium, PCS 2010, 2010)
The prediction step is a very important part of hybrid video codecs. In this contribution, a novel spatio-temporal prediction algorithm is introduced. For this, the prediction is carried out in two steps. Firstly, a preliminary temporal prediction is conducted by motion compensation. Afterwards, spatial refinement is carried out for incorporating spatial redundancies from already decoded neighboring blocks. Thereby, the spatial refinement is achieved by applying Non-Local Means de-noising to the union of the motion compensated block and the already decoded blocks. Including the spatial refinement into H.264/AVC, a rate reduction of up to 14 % or respectively a gain of up to 0.7 dB PSNR compared to unrefined motion compensated prediction can be achieved. © 2010 IEE
Keywords: Prediction; Signal Extrapolation; Video coding

Spatio-temporal prediction in video coding by spatially refined motion compensation - Seiler, Jürgen; Kaup, André (2008 IEEE International Conference on Image Processing, ICIP 2008, 2008)
The purpose of this contribution is to introduce a new method of signal prediction in video coding. Unlike most existent prediction methods that either use temporal or use spatial correlations to generate the prediction signal, the proposed method uses spatial and temporal correlations at the same time. The spatio-temporal prediction is obtained by first performing motion compensation for a macroblock, followed by a refinement step that pays attention to the correlations between the macroblock and its surroundings. At the decoder, the refinement step can be performed in the same manner, thus no additional side information has to be transmitted. Implementation of the spatial refinement step into the H.264/AVC video codec leads to reduction in data rate of up to nearly 15% and increase in PSNR of up to 0.75 dB, compared to pure motion compensated prediction. © 2008 IEE
Keywords: Spatio-temporal prediction; Video coding; Video communication

Spatiotemporal selective extrapolation for 3-D signals and its applications in video communications - Meisinger, Katrin; Kaup, André (IEEE Transactions on Image Processing, 2007)
In this paper, we derive a spatiotemporal extrapolation method for 3-D discrete signals. Extending a discrete signal beyond a limited number of known samples is commonly referred to as discrete signal extrapolation. Extrapolation problems arise in many applications in video communications. Transmission errors in video communications may cause data losses which are concealed by extrapolating the surrounding video signal into the missing area. The same principle is applied for TV logo removal. Prediction in hybrid video coding is also interpreted as an extrapolation problem. Conventionally, the unknown areas in the video sequence are estimated from either the spatial or temporal surrounding. Our approach considers the spatiotemporal signal including the missing area in a volume and replaces the unknown samples by extrapolating the surrounding signal from spatial, as well as temporal direction. By exploiting spatial and temporal correlations at the same time, it is possible to inherently compensate motion. Deviations in luminance occurring from frame to frame can be compensated, too. © 2007 IEE
Keywords: Concealment; Extrapolation; Prediction; TV logo removal; Video communications

Spatio-temporal selective extrapolation for 3-D signals applied to concealment in video communications - Meisinger, Katrin; Martin, S.; Kaup, André (14th European Signal Processing Conference, EUSIPCO 2006, 2006)
In this paper we derive a frequency selective extrapolation method for three-dimensional signals. Extending a signal beyond a limited number of known samples is commonly referred to as signal extrapolation. We provide an extrapolation technique which enables to estimate image areas by exploiting simultaneously spatial and temporal correlations of the video signal. Lost areas caused by transmission errors are concealed by extrapolation from the surrounding. The missing areas in the video sequence are estimated conventionally from either the spatial or temporal surrounding. Our approach approximates the known signal by a weighted linear combination of 3-D basis functions from spatial as well as temporal direction and extrapolates it into the missing area. The algorithm is able to extrapolate smooth and structured areas and to inherently compensate motion and changes in luminance from frame to frame.

Spatiotemporal Structuring of Compressed Videos Based on Motion Information - Treetasanatavorn, Siripong (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2007)

Speaker Localization Using a Steered Filter-And-Sum Beamformer - Strobel, Norbert; Meier, Th.; Rabenstein, Rudolf (Proc. Workshop on Vision, Modeling, and Visualization, 1999)
LMS::311

Special issue on multidimensional signal processing applications - Rabenstein, Rudolf; Velten, Joerg (Multidimensional Systems and Signal Processing, 2014)
This special issue on multidimensional signal processing applications covers a broad range of methodologies and algorithms for spatially extended configurations of sensors and emitters for various purposes and applications. The spectrum of the typical wavelengths for these applications ranges from very short ones in electron transmission microsopy to visibile light, up to radio frequencies and further to sound waves in the audio range suitable for human perception. In spite of the diversity of the underlying physical principles there are similarities in the arising problems. Consequently also some of the approaches to the processing of the resulting sensor or emitter signals are shared between these applications. One common problem is the detection of drift and translation between images acquired by electron transmission microsopy and conventional cameras. Also the detection of motion in video sequences falls into this category. Another set of problems is concerned with the detection of the direction of a source of either electromagnetic or sound waves using suitable receiver arrays. But the situation may also be turned around by forming directed wave fronts from loudspeaker arrays to produce sound fields with certain spatial requirements. This set of seven papers shows that modern multidimensional signal processing overcomes the traditional separation of image processing and audio processing. The presented methods and algorithms prescind from plain processing of available sensor or emitter signals. Instead the focus lies on physical processes and mathematical foundations underlying all applications. The paper A comparison between minimum variance control and other online compensation methods for specimen drift in transmission electron microscopy by A. Tejada and A.J. den Dekker presents an online method to reduce image blurring by specimen drift during exposure. They achieve this reduction by employing an adaptive control algorithm for horizontal and vertical drift. A related problem for an application in image processing is discussed in Estimation of horizontal and vertical translations of large images based on columns and rows mean energy matching by E. Skubalska-Rafajlowicz. Here the 2-D translation problem is broken down to one dimension, resulting in computational savings for large images. The solution is reached by optimzation of a suitably defined energy measure. Detecting translations between subsequent frames is also the topic of the paper A multidimensional wave digital filter bank for video-based motion analysis by T. Schwertfeger, J. Velten, and A. Kummert. They solve the problem by space-time processing methods, i.e. by implementing fan-shaped transfer functions by multidimensional wave digital filters. Space-time processing is also employed for shaping the directional characteristics of antennas in the radio frequency domain in Electronically scanned RF-to-bits beam aperture arrays using 2-D IIR spatially bandpass digital filters by A. Madanayake et al. The authors show a fixed-point implementation of a two-dimensional digital filter for processing radio frequencies in the GHz range. Consideration of directional characteristics paves the way to localization of sources in the acoustic domain as well. The fundamental problems are similar as for radio frequencies, although frequency range and receiver technologies are quite different. A unifying perspective on acoustic source localization is offered by P. Bestagini et al. in their paper TDOA-based acoustic source localization in the space-range reference frame. They start with time-differences-of-arrival detected from recorded source signals between different microphones and adopt a new reference frame by combining spatial coordinates and range differences. The conversion from time differences to range differences requires knowledge of the propagation speed which depends on the medium in which waves travel. The paper Closed-form estimation of the speed of propagating waves from time measurments by P. Annibale and R. Rabenstein shows how the propagation speed can be estimated along with the source position from the same set of microphone signals. Finally K. Helwani, S. Spors, and H. Buchner consider directional information for emitters of sound waves rather than sensors. In The synthesis of sound figures they show how to drive arrays of loudspeakers to create sound fields with spatially fixed zones of high sound pressure level along with other zones of low levels. The presented approach is based on a Green’s function description of sound propagatio

Spectral Constrained Frequency Selective Extrapolation for Rapid Image Error Concealment - Genser, Nils; Seiler, Jürgen; Kaup, André (25th International Conference on Systems, Signals and Image Processing (IWSSIP), 2018)
Keywords: Error Concealment; Inpainting; Signal Extrapolation

Spectral Description of Multidimensional Systems with the Sturm-Liouville Transformation - Rabenstein, Rudolf; Trautmann, Lutz (Proc. Int. Workshop on Spectral Methods and Multirate Signal Processing (SMMSP 01), 2001)
LMS::148

Spectral feature-based nonlinear residual echo suppression - Schwarz, Andreas; Hofmann, Christian; Kellermann, Walter (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2013)
We propose a method for nonlinear residual echo suppression that consists of extracting spectral features from the far-end signal, and using an artificial neural network to model the residual echo magnitude spectrum from these features. We compare the modeling accuracy achieved by realizations with different features and network topologies, evaluating the mean squared error of the estimated residual echo magnitude spectrum. We also present a low complexity real-time implementation combining an offline-trained network with online adaptation, and investigate its performance in terms of echo suppression and speech distortion for real mobile phone recordings. © 2013 IEE
Keywords: AES; Nonlinear acoustic echo suppression; RES; residual echo suppression

Spectrogram analysis of selected tremor signals using short-time Fourier transform and continuous wavelet transform - Bartosch, Thorsten; Seidl, D. (Annals of Geophysics, 1999)
LMS::686

Speech Driven Synthesis of Talking Head Sequences - Eisert, Peter; Chaudhuri, S.; Girod, Bernd (3D Image Analysis and Synthesis, 1997)
LMS::417

Speech enhancement for binaural hearing aids based on blind source separation - Reindl, Klaus; Zheng, Yuanhang; Kellermann, Walter (4th International Symposium on Communications, Control, and Signal Processing, ISCCSP-2010, 2010)
The availability of wireless technologies leads from monaural or bilateral hearing aids to binaural processing strategies. In this paper, we investigate a class of blind source separation (BSS)-based speech enhancement algorithms for binaural hearing aids. The blind binaural processing strategies are analyzed and evaluated for different scenarios, i.e., determined scenarios, where the number of sources does not exceed the number of available sensors and underdetermined scenarios, where there are more active source signals than microphones which is typical for hearing aid applications. These blind algorithms are an attractive alternative to beamforming as no a-priori knowledge on the sensor positions is required. Moreover, BSS algorithms have the advantage that their optimization criteria are solely based on the fundamental assumption of mutual statistical independence of the different source signals. ©2010 IEE

Speech Reconstruction by Sparse Linear Prediction - Koloda, Jan; Peinado, A.M.; Sánchez, V. (Advances in Speech and Language Technologies for Iberian Languages (IberSPEECH), 2012)
This paper proposes a new variant of the least square autoregressive (LSAR) method for speech reconstruction, which can estimate via least squares a segment of missing samples by applying the linear prediction (LP) model of speech. First, we show that the use of a single high-order linear predictor can provide better results than the classic LSAR techniques based on short- and long-term predictors without the need of a pitch detector. However, this high-order predictor may reduce the reconstruction performance due to estimation errors, especially in the case of short pitch periods, and non-stationarity. In order to overcome these problems, we propose the use of a sparse linear predictor which resembles the classical speech model, based on short- and long-term correlations, where many LP coefficients are zero. The experimental results show the superiority of the proposed approach in both signal to noise ratio and perceptual performanc
LMS::1536
Keywords: speech reconstruction; error concealment; sparse linear prediction; least squares; autoregressive model

Speed of sound and air temperature estimation using the TDOA-based localization framework - Annibale, Paolo; Rabenstein, Rudolf (2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012, 2012)
Spatially distributed acoustic sensors find increasingly new applications in speech-based human-machine interfaces. One well researched topic is the localisation of an emitting source from Time-Difference-Of-Arrival (TDOA) measurements. This manuscript shows how to exploit the source localization framework and its state of the art techniques to accurately estimate the actual sound speed and the air temperature by measuring TDOAs of an unknown located acoustic source. Simulations and experiments show the validity of the proposed method. © 2012 IEE

Spherical Diffusion Model with Semi-Permeable Boundary: A Transfer Function Approach - Schäfer, Maximilian; Wicke, Wayan; Haselmayr, Werner; Rabenstein, Rudolf; Schober, Robert (IEEE International Conference on Communications (ICC), 2020)
The derivation of suitable analytical models is an important step for the design and analysis of molecular commu- nication systems. However, many existing models have limited applicability in practical scenarios due to various simplifications (e.g., assumption of an unbounded environment). In this paper, we develop a realistic model for particle diffusion in a bounded sphere and particle transport through a semi-permeable bound- ary. This model can be used for various applications, such as modeling of inter-/intra-cell communication or the release process of drug carriers. The proposed analytical model is based on a transfer function approach, which allows for fast numerical evaluation and provides insights into the impact of the relevant molecular communication system parameters. The proposed solution of the bounded spherical diffusion problem is formulated in terms of a state-space description and the semi-permeable boundary is accounted for by a feedback loop. Particle-based simulations verify the proposed modeling approach.

Spiral search based fast rotation estimation for efficient HEVC compression of navigation video sequences - Springer, Dominic; Frank, M.; Simmet, F.; Niederkorn, D.; Kaup, André (2013 Picture Coding Symposium, PCS 2013, 2013)
During the development and testing of navigation systems for modern cars, the video streams of the systems-under-test are carefully observed for errors of any kind. While this observation is typically carried out manually by software engineers monitoring the video output, the complexity and variety of the latest navigation systems demand an automated analysis of system functionality. Recently published work on automated error detection for navigation systems [1] can only enfold its full functionality if video sequences can be recorded and compressed in real-time, so that the full test program can be run under lab conditions after recording. However, the rotational motion of these sequences makes efficient compression a difficult task. In this paper, we present a fast and efficient method for rotation estimation, so that rotational motion compensation and thus efficient encoding can take place. Compared to existing state of the art approaches for SIFT- or SURF-based global motion estimation, our scheme requires 1/16th of the original processing time while providing almost identical quality gains of up to 2.5dB. © 2013 IEE

Stability analysis for implicit second order finite difference schemes - Rabenstein, Rudolf; Steffen, Peter (2011 7th International Workshop on Multidimensional (nD) Systems, nDS 2011, 2011)
Recent applications of iterative learning control and repetitive processes lead to implicit second order finite difference schemes which require practical stability testing. A von Neumann type stability analysis is employed to reduce the problem to a second order polynomial. The conditions under which its zeros lie within the unit circle can be recast by application of the bilinear transformation. Then the problem is reduced to a test for a Hurwitz polynomial. Its coefficients depend not only on the spatial frequency but also on parameters of the initial problem like step sizes in time and space. The admissible ranges of these parameters follow finally from simple inequalities. The method is demonstrated by examples. © 2011 IEE

Stability of Image Sequence Processing Systems - Rabenstein, Rudolf; Steffen, Peter (AEU-Archiv für Elektronik und Uberträgungstechnik, 1983)

Stability of recursive time-varying digital filters by state vector transformation - Rabenstein, Rudolf; Czarnach, Roland (Signal Processing, 1985)
The stability of recursive time-variant digital filters depends on the variable filter coefficients as well as on the chosen state-space structure. A condition for the behaviour of the state vector is given, that guarantees asymptotic stability in the sense of Lyapunov. Some structures do always yield this behaviour. However, all other structures produce the same output signal with a suitable transformation of the state vector with every coefficient change. This transformation can be implemented without increase of the computational complexity when the filter coefficients are altered. This will be demonstrated by an example for a structure with the minimum number of multipliers. © 1985.
Keywords: Stability; time-varying filters

Stability of two-step repetitive processes based on a matrix formulation - Steffen, Peter; Rabenstein, Rudolf; Galkowski, Krzysztof (5th International Workshop on Multidimensional (nD) Systems, nDS2007, 2007)
The stability of two-step repetitive processes will be considered. We assume signals depending on two spatial variables and the pass-number. In all cases of interest the pass lengths will be finite. Since all known approaches to the stability problem do not take this fact into account, the resulting statements concerning stability are of little help. To overcome this dilemma the process is reformulated into a vector-difference-equation of order 2 with the dimension LM as the product of the pass lengths. The stability of systems described by such equations has been investigated earlier. These results can be applied directly to the problem. Simple inequalities for the Fourier-transforms of the mask-coefficients are obtained. In the case of 3 × 3 masks, explicit stability conditions in the form of expressions for the mask-elements can be derive

Stabilizing Training with Soft Dynamic Time Warping: A Case Study for Pitch Class Estimation with Weakly Aligned Targets - Zeitler, Johannes; Deniffel, Simon; Krause, Michael; Müller, Meinard (Proceedings of the 24th International Society for Music Information Retrieval Conference, 2023)
Soft dynamic time warping (SDTW) is a differentiable loss function that allows for training neural networks from weakly aligned data. Typically, SDTW is used to iteratively compute and refine soft alignments that compensate for temporal deviations between the training data and its weakly annotated targets. One major problem is that a mismatch between the estimated soft alignments and the reference alignments in the early training stage leads to incorrect parameter updates, making the overall training procedure unstable. In this paper, we investigate such stability issues by considering the task of pitch class estimation from music recordings as an illustrative case study. In particular, we introduce and discuss three conceptually different strategies (a hyperparameter scheduling, a diagonal prior, and a sequence unfolding strategy) with the objective of stabilizing intermediate soft alignment results. Finally, we report on experiments that demonstrate the effectiveness of the strategies and discuss efficiency and implementation issues.<br

Stable systems for nonlinear discrete sound synthesis with the functional transformation method - Trautmann, Lutz; Rabenstein, Rudolf (2002 IEEE International Conference on Acoustic, Speech and Signal Processing, 2002)
Discrete systems for digital sound synthesis derived with the functional transformation method (FTM) from physical models have recently been presented. For linear systems, the FTM solves the partial differential equation (PDE) describing the vibrating structure analytically. The algorithms obtained after discretization of the analytical solution preserve the inherent physical stability of the continuous system. For nonlinear physical models, as they occur in real musical instruments, the direct application of the FTM leads to an implicit continuous equation. This paper shows that discretization results in an explicit solution. Furthermore, the stability problems occurring after discretization of the nonlinear system are fixed by instantaneous energy considerations. For the example of a slapped transversal oscillating tightened string with frequency dependent loss terms an efficient algorithm is derived. It can also be applied to other types of interaction between a resonating body and its environment.

Standardized Air Exchange Measurements by Computer.Controlled Instrumentation - Rabenstein, Rudolf; Heidt, F. D. (ASHRAE Transactions, 1991)
LMS::1594

Statistical analysis and improvement of coherent-to-diffuse power ratio estimators for dereverberation - Zheng, Chengshi; Schwarz, Andreas; Li, X.; Kellermann, Walter (International Workshop on Acoustic Echo and Noise Control (IWAENC), 2016)
Dereverberation methods based on coherent-to-diffuse power ratio (CDR) estimates exploit the spatial properties of signals to suppress late reverberation components. Naturally, the quality of the CDR estimate has a major effect on the quality of the dereverberated signals. This paper presents a statistical study of the performance of state-of-the-art CDR estimators under a Gaussian signal model. The study reveals that the variance of the magnitude of the spatial coherence estimate is the major cause for the degradation of known CDR estimators, while the variance of the phase has only limited impact. Moreover, the variance has more impact on direction-of-arrival (DOA)-independent CDR estimators than on DOA-dependent estimators. Accordingly, we propose a cepstrum thresholding technique to reduce the variance of the spatial coherence estimate. Experimental results show that better performance can be achieved for dereverberation when the proposed spatial coherence method is applied to CDR estimators, especially for the DOA-independent estimator
Keywords: Speech dereverberation; statistical analysis; cepstrum thresholding

Statistical Analysis of Multi-Reference Block Matching for Dynamic Light Field Coding - Fecker, Ulrich; Kaup, André (10th International Fall Workshop - Vision, Modeling, and Visualization (VMV), 2005)
LMS::583

Statistical Analysis of Randomness in Training of Small-Scale Neural Networks for Speech Enhancement - Briegleb, Annika; Kellermann, Walter (2022 International Workshop on Acoustic Signal Enhancement (IWAENC), 2022)
For the training of neural networks, randomness due to the initialization is a widely acknowledged factor for training success. In speech enhancement, randomness is also introduced via the creation of the training dataset due to the synthetic mixing of speech and noise, and other data augmentation techniques. This aspect of randomness is often neglected in literature. In this contribution, we quantify the effect of randomness in the generation of the training dataset in comparison to the randomness introduced by the network initialization and find that for small networks and very small datasets, the assembly of the training dataset has a significant influence on the network’s performanc
Keywords: speech enhancement; training data; randomness; neural networks

Statistical analysis of the multichannel Wiener filter using a bivariate normal distribution for sample covariance matrices - Zheng, Chengshi; Deleforge, Antoine; Li, Xiaodong; Kellermann, Walter (IEEE/ACM Transactions on Audio, Speech and Language Processing, 2018)
This paper studies the statistical performance of the multichannel Wiener filter (MWF) when the weights are computed using estimates of the sample covariance matrices of the noisy and the noise signals. It is well known that the optimal weights of the minimum variance distortionless response beamformer are only determined by the noisy sample covariance matrix or the noise sample covariance matrix, while those of the MWF are determined by both of them. Therefore, the difficulty increases dramatically in statistically analyzing the MWF when compared to analyzing the MVDR, where the main reason is that expressing the general joint probability density function (p.d.f.) of the two sample covariance matrices presented a Hitherto unsolved problem, to the best of our knowledge. For a deeper insight into the statistical performance of the MWF, this paper first introduces a bivariate normal distribution to approximately model the joint p.d.f. of the noisy and the noise sample covariance matrices. Each sample covariance matrix is approximately modeled by a random scalar multiplied by its true covariance matrix. This approximation is designed to preserve both the bias and the mean squared error of the matrix with respect to a natural distance on covariance matrices. The correlation of the bivariate normal distribution, referred to as the sample covariance matrices intrinsic correlation coefficient, captures all second-order dependencies of the noisy and the noise sample covariance matrices. By using the proposed bivariate normal distribution, the performance of the MWF can be predicted from the derived analytical expressions and many interesting results are revealed. As an example, the theoretical analysis demonstrates that the MWF performance may degrade in terms of noise reduction and signal-to-noise-ratio improvement when using more sensors in some noise scenario
Keywords: Bivariate normal distribution; multichannel Wiener filter; sample covariance matrix; statistical analysis

Statistical Model-Based Change Detection in Moving Video - Aach, T.; Kaup, André; Mester, R. (Signal Processing, 1993)
A major issue with change detection in video sequences is to guarantee robust detection results in the presence of noise. In this contribution, we first compare different test statistics in this respect. The distributions of these statistics for the null hypothesis are given, so that significance tests can be carried out. An objective comparison between the different statistics can thus be based on identical false alarm rates. However, it will also be pointed out that the global thresholding methods resulting from the significance approach exhibit certain weaknesses. Their shortcomings can be overcome by the Markov random field based refining method derived in the second part of this paper. This method serves three purposes: it accurately locates boundaries between changed and unchanged areas, it brings to bear a regularizing effect on these boundaries in order to smooth them, and it eliminates small regions if the original data permits this. © 1993.
UnivIS-Import:2015-03-05:Pub.1993.tech.IE.LEN.statis
Keywords: change detection; deterministic relaxation; Image analysis; image coding; Markov random fields; regularization; significance tests

Steady-state performance limitations of subband adaptive filters - Weiß, S.; Stenger, Alexander; Stewart, R. W.; Rabenstein, Rudolf (IEEE Transactions on Signal Processing, 2001)
Nonperfect filterbanks used for subband adaptive filtering (SAF) are known to impose limitations on the steady-state performance of such systems. In this paper, we quantify the minimum mean-square error (MMSE) and the accuracy with which the overall SAF system can model an unknown system that it is set to identify. First, in case of MMSE limits, the error is evaluated based on a power spectral density description of aliased signal components, which is accessible via a source model for the subband signals that we derive. Approximations of the MMSE can be embedded in a signal-to-alias ratio (SAR), which is a factor by which the error power can be reduced by adaptive filtering. With simplifications, SAR only depends on the filterbanks. Second, in case of modeling, we link the accuracy of the SAF system to the filterbank mismatch in perfect reconstruction. When using modulated filterbanks, both error limits-MMSE and inaccuracy-can be linked to the prototype. We explicitly derive this for generalized DFT modu lated filterbanks and demonstrate the validity of the analytical error limits and their approximations for a number of examples, whereby the analytically predicted limits of error quantities compare favorably with simulations.
Keywords: Acoustic echo cancellation; Adaptive filters; Aliasing; Error approximation; Error variance; Filterbanks

Steganographic System Based on Higher-Order Statistics - Tzschoppe, Roman; Bäuml, R.; Huber, Johannes; Kaup, André (Security and Watermarking of Multimedia Contents V, 2003)
Universal blind steganalysis attempts to detect steganographic data without knowledge about the applied steganographic system. Farid proposed such a detection algorithm based on higher-order statistics for separating original images from stego images. His method shows an astonishing performance on current steganographic schemes. Starting from the statistical approach in Farid's algorithm, we investigate the well known steganographic tool Jsteg as well as a newer approach proposed by Eggers et al., which relies on histogram-preserving data mapping. Both schemes show weaknesses leading to a certain detectability. Further analysis shows which statistic characteristics make both schemes vulnerable. Based on these results, the histogram preserving approach is enhanced such that it achieves perfect security with respect to Farid's algorithm.
Keywords: Higher-order statistics; Histogram-preserving data mapping; Jsteg; Steganalysis; Steganography

STFT bin selection for localization algorithms based on the sparsity of speech signal spectra - Brendel, Andreas; Huang, Chengyu; Kellermann, Walter (EURONOISE 2018, 2018)
Many algorithms for localizing, tracking or Direction of Arrival (DOA) estimation of speech sources, rely on the so-called W-disjoint orthogonality, i.e., only one speaker is assumed to be active at a certain time-frequency bin. Based on this assumption, bin-wise DOA estimates can be computed from pairwise phase differences of each time-frequency bin and clustered afterwards. Averaging the estimates of each cluster, i.e., computing the cluster centroids, increases the robustness of the localization estimate. However, clustering can be computationally demanding due to the large amount of DOA estimates, and at the same time highly sensitive to errors as potentially many of them may not be reliable due to noise and reverberation. Therefore, an efficient selection algorithm for reliable Short-Time Fourier Transform (STFT) bins is desirable that aims at increasing the accuracy of the estimate while simultaneously reducing the computational complexity. In this contribution, we investigate different selection methods for STFT bins as suitable for localization algorithms for speech sources, which are based on the W-disjoint orthogonality, and exploit bin-wise speech signal power, Coherent-to-Diffuse Power Ratio (CDR), and Speech Presence Probability (SPP). The effectiveness of the selection processes is studied for different localization algorithms.

Stochastic Model-Based Image Segmentation Using Functional Approximation - Kaup, André; Aach, T. (IEEE International Workshop on Intelligent Signal Processing and Communication Systems, Sendai, 1993)
LMS::1021

Stochastic Model-Based Image Segmentation Using Functional Approximation - Kaup, André; Aach, T. (IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences, 1994)
LMS::1052

Stochastic Motion Coherency Analysis for Motion Vector Field Segmentation on Compressed Video Sequences - Treetasanatavorn, Siripong; Rauschenbach, U.; Heuer, J.; Kaup, André (6th Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Montreux, Switzerland, 2005)
LMS::548

Strategies for modeling reverberant speech in the feature domain - Sehr, Armin; Kellermann, Walter (2009 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2009, 2009)
The length of the room impulse response characterizing the acoustic path between speaker and microphone is significantly larger than the length of the analysis window used for feature extraction in automatic speech recognition (ASR) systems. Therefore, reverberation caused by multi-path propagation of sound waves from the speaker to distant-talking microphones has a dispersive effect on speech feature sequences. This dispersive effect causes a mismatch between the input speech and the acoustic models of the recognizer, usually trained on clean speech, and leads to a significant reduction of recognition performance. In this contribution, different strategies for obtaining acoustic models capturing the dispersive effect of reverberation are investigated in terms of modeling accuracy, flexibility with respect to changing reverberation conditions, effort for obtaining the reverberation representation and decoding complexity. ©2009 IEE
Keywords: Acoustic modeling; Distant-talking ASR; Reverberation; Robust ASR

Structure-preserving spectral reflectance estimation using guided filtering - Sippel, Frank; Seiler, Jürgen; Genser, Nils; Kaup, André (Journal of the Optical Society of America A-Optics Image Science and Vision, 2020)
Light spectra are a very important source of information for diverse classification problems, e.g., for discrimination of materials. To lower the cost of acquiring this information, multispectral cameras are used. Several techniques exist for estimating light spectra out of multispectral images by exploiting properties about the spectrum. Unfortunately, especially when capturing multispectral videos, the images are heavily affected by noise due to the nature of limited exposure times in videos. Therefore, models that explicitly try to lower the influence of noise on the reconstructed spectrum are highly desirable. Hence, a novel reconstruction algorithm is presented. This novel estimation method is based on the guided filtering technique that preserves basic structures, while using spatial information to reduce the influence of noise. The evaluation based on spectra of natural images reveals that this new technique yields better quantitative and subjective results in noisy scenarios than other state-of-the-art spatial reconstruction methods. Specifically, the proposed algorithm lowers the mean squared error and the spectral angle up to 46% and 35% in noisy scenarios, respectively. Furthermore, it is shown that the proposed reconstruction technique works out of the box and does not need any calibration or training by reconstructing spectra from a real-world multispectral camera with nine channel
Keywords: Multispectral Imaging; Spectral Reconstruction; Light Spectra

Subband-based Blind Separation for Convolutive Mixtures of Speech - Araki, S.; Makino, Shoji; Aichner, Robert; Nishikawa, T.; Saruwatari, H. (IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences, 2005)
LMS::588

Subband based Blind Source Separation for convolutive mixtures of speech - Araki, S.; Makino, Shoji; Aichner, Robert; Nishikawa, T.; Saruwatari, H. (Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), 2003)
LMS::375

Subband Based Blind Source Separation with Appropriate Processing for Each Frequency Band - Araki, S.; Makino, Shoji; Aichner, Robert; Nishikawa, T.; Saruwatari, H. (Proc. Int. Symposium on Independent Component Analysis and Blind Signal Separation (ICA), 2003)
LMS::371

Subjektiver Videobetrachtungstest für niederratige Multimedia-Szenarien - Barkowsky, Marcus; Bialkowski, Jens; Kaup, André (Elektronische Medien: 11. Dortmunder Fernsehseminar, ITG-Fachbericht,, 2005)
LMS::593

Super-resolution for differently exposed mixed-resolution multi-view images adapted by a histogram matching method - Richter, Thomas; Kaup, André (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017)
Super-resolution is an important task in the image and video processing domain. In mixed-resolution multi-view scenarios, neighboring high-resolution reference perspectives can be used to increase the image quality of a given low-resolution target view. By using corresponding depth information, the required high-frequency part can be projected from a reference view onto the image plane of the target perspective. However, the contrast and thus the amount of high-frequency information in a reference view varies with the cameras exposure settings. As a consequence, the resulting super-resolution quality drops in case of exposure time variations between the different views. By incorporating a histogram matching method, the required high-frequency part can be efficiently adapted to the exposure settings of the target view. The simulation results show that the proposed adaption leads to an average PSNR gain of 0.63 dB for differently exposed mixed-resolution multi-view setup
LMS::1707
Keywords: multi-view; mixed-resolution; super-resolution; histogram matching; DIBR

Super-Resolution for Mixed-Resolution Multiview Image plus Depth Data using a Novel Two-Stage High-Frequency Extrapolation Method for Occluded Areas - Richter, Thomas; Seiler, Jürgen; Schnurrer, Wolfgang; Kaup, André (2015 IEEE International Conference on Image Processing (ICIP), 2015)
Mixed-resolution multiview setups offer great savings in both, costs regarding the equipment acquisition and complexity refering to data transmission and storage. However, for applications like free viewpoint television, high-resolution images are desired from all available camera perspectives. Therefore, high-frequency information of adjacent high-resolution cameras can be used to increase the visual quality of a low-resolution camera perspective. However, due to occlusions, some parts of the scene are invisible in the high-resolution views and cannot be synthesized from the reference images. In this paper, a novel high-frequency extrapolation method is proposed, utilizing additional sampling points pre-estimated by a low-resolution block-matching approach. Averaged over all considered test sets, the proposed method achieves a PSNR gain of 0.94 dB with respect to the unrefined signal extrapolation algorith
Keywords: Super-Resolution; Mixed-Resolution; Stereo Imaging; Signal Extrapolation

Super-Resolution for Mixed-Resolution Multiview Images Using a Relative Frequency Response Estimation Method - Richter, Thomas; Habermann, Annelie; Kaup, André (IEEE International Conference on Visual Communications and Image Processing (VCIP), 2015)
In mixed-resolution multiview setups, spatially neighboring high-resolution views can be used to increase the image quality of an intermediate low-resolution camera perspective. By utilizing corresponding depth data, high-frequency information can be projected from the reference perspectives onto the image plane of the low-resolution view, creating the desired super-resolved image. In this context, the high-frequency part is extracted by subtracting an estimated low-resolution image from a high-resolution reference view. Starting from this, an approach is proposed that aims at first estimating the frequency responses of the involved cameras. As a consequence, the low-frequency part of a neighboring view can be adapted to the image characteristics of the intermediate low-resolution camera perspective. Exploiting this knowledge, the approach finally leads to a high-frequency part that better fits the target low-resolution view and thus to a better super-resolution quality. For a downsampling factor of 4, the proposed estimation leads to an average gain of 0.53 d
Keywords: super-resolution; multiview; mixed-resolution; frequency response; image warping

Super-Resolution of Positive Sources on an Arbitrarily Fine Grid - Morgenshtern, Veniamin (Journal of Fourier Analysis and Applications, 2021)
In super-resolution it is necessary to locate with high precision point sources from noisy observations of the spectrum of the signal at low frequencies capped by f(lo). In the case when the point sources are positive and are located on a grid, it has been recently established that the super-resolution problem can be solved via linear programming in a stable manner and that the method is nearly optimal in the minimax sense. The quality of the reconstruction critically depends on the Rayleigh regularity of the support of the signal; that is, on the maximum number of sources that can occur within an interval of side length about 1/f(lo). This work extends the earlier result and shows that the conclusion continues to hold when the locations of the point sources are arbitrary, i.e., the grid is arbitrarily fine. The proof relies on new interpolation constructions in Fourier analysis.
CRIS-Team WoS Importer:2021-12-10

Super-Resolution Techniques for Mixed-Resolution Multi-View Images - Richter, Thomas (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2016)
Super-resolution approaches aim at increasing the spatial resolution of a given low-resolution input image. Basically, respective methods can be grouped into single-image and multi-image approaches, which can be further subdivided, depending on whether temporally or spatially neighboring images are used for super-resolution. Within the scope of this thesis, super-resolution is conducted for the special case of mixed-resolution multi-view images, where a scene is captured from different perspectives with cameras providing various spatial resolutions. Using the concept of high-frequency synthesis, neighboring high-resolution views can be efficiently exploited. Under ideal conditions, remarkable gains can be achieved, by projecting the missing high-frequency part from a reference perspective onto the image plane of the target view. However, these ideal conditions are typically not fullfilled for practical scenarios, negatively influencing the achievable super-resolution quality. This thesis deals with different challenges, arising in the context of super-resolution based on high-frequency synthesis. First, an estimation method is discussed, which aims at extracting a proper high-frequency part that, after projection, well fits the desired target view. Since the required high-frequency projection is based on depth information, the influence of inaccurate depth acquisition is conducted. As a consequence, a robustly extended high-frequency synthesis method is proposed. The simulation results show that, using this extension, high-quality super-resolution results can be obtained, even in the case of strong depth distortions or inaccurate multi-sensor calibration. In addition, dealing with remaining occluded areas is intensively investigated. Depending on the camera setup and the scene geometry, some image parts might not be visible in any of the reference views, leading to holes in the synthesized high-frequency image. To properly handle these unknown high-frequency parts, different strategies are proposed, clearly outperforming state-of-the-art reconstruction methods. However, even in the case of mixed-resolution arrays, high-resolution reference cameras are required. In order to avoid the need of high-resolution sensors, the concept of non-regular sampling is introduced. A reconstruction scheme for irregularly sampled multi-view images is discussed at the end of this thesis.
Keywords: Mehrkamerasystem; Bildsignal; Signalaufbereitung

Survey on approaches to speech recognition in reverberant environments - Yoshioka, Takuya; Sehr, Armin; Delcroix, Marc; Kinoshita, Keisuke; Maas, Roland; Nakatani, Tomohiro; Kellermann, Walter (Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2012)
This paper overviews the state of the art in reverberant speech processing from the speech recognition viewpoint. First, it points out that the key to successful reverberant speech recognition is to account for long-term dependencies between reverberant observations obtained from consecutive time frames. Then, a diversity of approaches that exploit the long-term dependencies in various ways is described, ranging from signal and feature dereverberation to acoustic model compensation tailored to reverberation. A framework for classifying those approaches is presented to highlight similarities and differences between them. © 2012 APSIP

SVT-AV1 Encoding Bitrate Estimation Using Motion Search Information - Eichermüller, Lena; Chaudhari, Gaurang; Katsavounidis, Ioannis; Lei, Zhijun; Tmar, Hassene; Herglotz, Christian; Kaup, André (32nd European Signal Processing Conference, 2024)