Extended Kalman filter with probabilistic data association for multiple non-concurrent speaker localization in reverberant environments - Chakrabarty, Soumitro; Kowalczyk, Konrad; Taseska, Maja; Habets, Emanuël (Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2014)
Acoustic source localization and tracking (ASLT) in reverberant environments is a challenging task due to the multi-path propagation of acoustic waves. ASLT is often based on the use of a Kalman filter or a particle filter, with time-difference-of-arrival (TDOA) estimates used as measurements. In this work, we aim to track non-concurrent speakers by applying an extended Kalman filter (EKF) with probabilistic data association (PDA) that takes into account multiple measurements simultaneously. By using PDA, the inaccuracy of the measurements caused by room reflections and noise is explicitly taken into account. Unlike in typical approaches where the measurements consist of broadband TDOA estimates, the measurements in the proposed approach consist of multiple narrowband direction-of-arrival (DOA) estimates obtained from distributed microphone arrays. Experimental results demonstrate that incorporating PDA and using properly selected narrowband DOA estimates leads to a better tracking performance, as compared to the standard EKF with a single narrowband or broadband measuremen
Keywords: extended Kalman filter; acoustic source localization; source tracking; recursive Bayesian filter

Extended lagrange multiplier selection for hybrid video coding using interframe correlation - Li, Xiang; Örtel, Anne; Hutter, A.; Kaup, André (26th Picture Coding Symposium, PCS 2007, 2007)
The Lagrange multiplier based rate-distortion optimization has been proved an effective technique in hybrid video codec design. To improve the coding efficiency, an efficient La-grange multiplier selection method is presented in this paper. As an extension to our previous work, the Laplace distribution based rate model is further refined by taking correlations among successive frames into account, so that the proposed method shows a superior performance over previous algorithms. Compared with the most recent reference software of H. 264/AVC, a gain of 0.26dB on average, 1.3dB at maximum, was achieve
Keywords: Hybrid video coding; Lagrange multiplier selection; Rate-distortion optimization

Extended Signaling Methods for Reduced Video Decoder Power Consumption Using Green Metadata - Herglotz, Christian; Kränzler, Matthias; Chu, Xixue; Francois, Edouard; He, Yong; Kaup, André (IEEE Transactions on Circuits and Systems II: Express Briefs, 2023)

Extending Vaidyanathan's procedure to improve the performance of unitary filter banks with a fixed lowpass by using additional elementary building blocks - Steffen, Peter; Brandhuber, Wolfgang (Proceedings of SPIE, Optical Engineering + Applications, 2007)
LMS::812

Extending Video Decoding Energy Models for 360° and HDR Video Formats in HEVC - Kränzler, Matthias; Herglotz, Christian; Kaup, André (Picture Coding Symposium (PCS), 2019)
Research has shown that decoder energy models are helpful tools for improving the energy efficiency in video playback applications. For example, an accurate feature-based bit stream model can reduce the energy consumption of the decoding process. However, until now only sequences of the SDR video format were investigated. Therefore, this paper shows that the decoding energy of HEVC-coded bit streams can be estimated precisely for different video formats and coding bit depths. Therefore, we compare a state-of-the-art model from the literature with a proposed model. We show that bit streams of the 360°, HDR, and fisheye video format can be estimated with a mean estimation error lower than 3.88% if the setups have the same coding bit depth. Furthermore, it is shown that on average, the energy demand for the decoding of bit streams with a bit depth of 10-bit is 55% higher than with 8-bit.<br
Keywords: Decoder; Energy Modeling; Estimation error; Power Measurement; 360°; HDR; Fisheye; HEVC

Extension of the remos concept to frequency-filtering-based features for reverberation-robust speech recognition - Maas, Roland; Wolf, Martin; Sehr, Armin; Nadeu, Climent; Kellermann, Walter (2011 Joint Workshop on Hands-free Speech Communication and Microphone Arrays, HSCMA'11, 2011)
The introduction of partly decorrelated features into the REMOS (REverberationMOdeling for Speech recognition) concept for distant-talking speech recognition [1] is discussed. REMOS combines a hidden Markov model (HMM), trained on clean speech, with a reverberation model capturing certain room characteristics. The most likely contributions of both models to a reverberant observation are determined by an inner optimization problem. In HMM frameworks, decorrelated features are assumed when diagonal covariance matrices are used in the output densities. However, in REMOS, only highly correlated logmelspec (logarithmic mel-spectral) features have been used so far, which has been limiting the recognition performance. In this work, we extend the RE-MOS concept and introduce a new set of partly decorrelated features derived from the frequency filtering [2]. Recognition experiments with connected digits show a consistent relative reduction in word error rate of up to 29% compared to the former logmelspec implementation. © 2011 IEE
Keywords: Automatic speech recognition; distant-talking; feature decorrelation; frequency filtering; reverberation-robust

Extrapolation of Bandlimited Multidimensional Signals from Continuous Measurements - Frankenbach, Cornelius; Martínez-Nuevo, Pablo; Møller, Martin; Kellermann, Walter (European Signal Processing Conference (EUSIPCO), 2021)
Conventional sampling and interpolation commonly rely on discrete measurements. In this paper, we develop a theoretical framework for extrapolation of signals in higher dimensions from knowledge of the continuous waveform on bounded high-dimensional regions. In particular, we propose an iterative method to reconstruct bandlimited multidimensional signals based on truncated versions of the original signal to bounded regions-herein referred to as continuous measurements. In the proposed method, the reconstruction is performed by iterating on a convex combination of region-limiting and bandlimiting operations. We show that this iteration consists of a firmly nonexpansive operator and prove strong convergence for multidimensional bandlimited signals. In order to improve numerical stability, we introduce a regularized iteration and show its connection to Tikhonov regularization. The method is illustrated numerically for two-dimensional signals.
CRIS-Team Scopus Importer:2021-01-22
Keywords: Papoulis' algorithm; Signal extrapolation; Signal reconstruction; Tikhonov regularization

Facial Expression Analysis for Model-Based Coding of Video Sequences - Eisert, Peter; Girod, Bernd (Picture Coding Symposium (PCS 97), 1997)
LMS::416

Fading Techniques for Error Concealment in Block-Based Decoding Systems - Friebe, Markus; Kaup, André (IEEE Transactions on Broadcasting, 2007)
Error concealment has a great importance within block-based video decoding systems especially for mobile transmission systems like DVB-T or DVB-H. In this paper, we propose a spatio-temporal and spatio-bi-temporal fading technique for recovering lost or erroneously received macroblock samples. For spatio-temporal fading based on a boundary error criterion obtained from temporal error concealment, either spatial, temporal, or fading of both methods is used for recovering lost image samples in one macroblock. A weighted absolute difference between well received macroblock boundary samples from the current frame and motion compensated macroblock boundary samples from the previous frame represents the boundary error. The weights for fading are interpolated from the boundary error. For spatio-bi-temporal fading, one lost macroblock can individually be recovered pel-wise spatially from the current or bi-temporally from the previous and the future frame by weighted averaging three different estimations for the lost macroblock. The simulation results have shown that for concealing erroneously received uncompressed and compressed video data the proposed methods outperform the reference methods both in subjective and objective video quality. © 2007 IEE
Keywords: Error concealment; Video coding and transmission; Video restoration

Fast adaptation of frequency-domain volterra filters using inherent recursions of iterated coefficient updates - Zeller, Marcus; Kellermann, Walter (15th European Signal Processing Conference, EUSIPCO 2007, 2007)
Adaptive Volterra filters are a popular model for compensating distortions caused by nonlinear structures with memory such as low-quality loudspeakers. This paper proposes a fast version of the recently investigated repeated coefficient updates for the partitioned block frequency-domain adaptive Volterra filter. Exploiting inherent recursions of the iteration procedure yields an efficient realization with a very low additional complexity compared to the usual LMS adaptation. Experimental results for both noise and speech demonstrate a significant acceleration of the filter convergence and overall echo cancellation for realistic nonlinear AEC scenarios. © 2007 EURASI

Fast and robust adaptation of DFT-domain Volterra filters in diagonal coordinates using iterated coefficient updates - Zeller, Marcus; Kellermann, Walter (IEEE Transactions on Signal Processing, 2010)
This paper presents a novel data-reusing technique for implementing adaptive discrete Fourier transform (DFT) domain Volterra filters in diagonal coordinates having arbitrary nonlinear order. In general, a major drawback of such nonlinear filters is the large number of parameters. Thus, a weak excitation of higher-order kernels results in a slow convergence for system identification tasks. In order to exploit the available innovation of the signals more effectively, each data frame is processed for a specified number of iterations in an overlap-save scheme, thereby enhancing the convergence performance. Due to inherent recursions of the repeated filtering and updating steps, this concept also lends itself to a fast algorithm, requiring only a moderate increase in algorithmic complexity over a baseline implementation. A detailed comparison of the required number of multiplications is given for several different algorithm versions. Despite the relatively simple concept, the outlined iteration algorithm exhibits significant gains in both adaptation speed and steady-state convergence. This is demonstrated by various experiments for both stationary noise and speech input considering an application of nonlinear acoustic echo cancellation. Finally, investigations on the tracking behavior for time-variant nonlinearities and robustness against misdetection of double-talk situations confirm the promising benefits and good trade-off of the proposed iterated coefficient updates Volterra filters approach. © 2010 IEE
Keywords: Adaptive filters; Echo cancellation; Identification; Iterative methods; Nonlinear systems

Fast and robust spatio-temporal image alignment for inter-sequence error concealment - Tröger, Tobias; Kaup, André (2010 17th IEEE International Conference on Image Processing, ICIP 2010, 2010)
Terrestrial broadcasting of digital TV is often error-prone due to suboptimal channels. Inter-sequence error concealment is a novel technique to reconstruct lost image blocks of video signals by inserting error-free blocks of reference signals. Considering multi-broadcast receivers, reference signals are typically scaled, cropped and delayed. Spatio-temporal image alignment therefore is the crucial point of inter-sequence error concealment. A well-known intensity-based approach utilizing a numerical optimization technique shows supreme reconstruction quality outperforming state-of-the-art concealment methods by up to 15 dB PSNRY. In this paper, we present a novel feature-based approach which significantly reduces complexity. By using scale-invariant features for image alignment, the computation time can be reduced by a factor of up to 32.4. As reconstruction quality is almost kept constant, robustness of our algorithm is high. The maximum loss is 0.8 dB in terms of PSNRY. © 2010 IEE
Keywords: Error concealment; Image alignment; Image registration; Scale-invariant feature transform; Synchronization

Fast CU Split Decisions for HEVC Inter Coding Using Support Vector Machines - Heindel, Andreas; Haubner, Thomas; Kaup, André (Picture Coding Symposium (PCS), 2016)
High Efficiency Video Coding (HEVC) is known to be the state of the art in video compression. However, aiming at real-time applications, its very high number of possible coding options makes it necessary to approximate the selection of coding modes by fast algorithms. For this purpose Coding Unit (CU) split decisions are made by Support Vector Machine (SVM) classifiers in this paper. We use one SVM model per depth and the traditional Rate-Distortion Optimization (RDO) is used as fallback method in the case of uncertain SVM decisions. Regarding the sequence classes ClassB to ClassE, the proposed algorithm achieves encoding time reductions of more than 60%, accompanied by only 1.8% additional bitrate (lowdelay main configuration). Considering all sequence classes from ClassA to ClassF, the additional bitrate increases to 4% for the lowdelay main and about 3% for the randomaccess main configuration, with a similar encoding time reduction of more than 60

Fast Dynamic Simulation of Flexible and Rigid Bodies with Kinematic Constraints - Hummel, Achim; Girod, Bernd (ACM Symposium on Virtual Reality Software and Technology, 1997)
LMS::406

Fast Edge-Aware Occlusion Detection in the Context of Multispectral Camera Arrays - Sippel, Frank; Seiler, Jürgen; Kaup, André (2024 IEEE International Conference on Image Processing, 2024)
Multispectral imaging is very beneficial in diverse applications, like healthcare and agriculture, since it can capture absorption bands of molecules in different spectral areas. A promising approach for multispectral snapshot imaging are camera arrays. Image processing is necessary to warp all different views to the same view to retrieve a consistent multispectral datacube. This process is also called multispectral image registration. After a cross spectral disparity estimation, an occlusion detection is required to find the pixels that were not recorded by the peripheral cameras. In this paper, a novel fast edge-aware occlusion detection is presented, which is shown to reduce the runtime by at least a factor of 12. Moreover, an evaluation on ground truth data reveals better performance in terms of precision and recall. Finally, the quality of a final multispectral datacube can be improved by more than 1.5 dB in terms of PSNR as well as in terms of SSIM in an existing multispectral registration pipeline. The source code is available at https://github.com/FAU-LMS/fast-occlusion-detection.<br

Faster IVA: Update Rules for Independent Vector Analysis based on Negentropy and the Majorize-Minimize Principle - Brendel, Andreas; Kellermann, Walter (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2021)

Fast Estimation of an Expressive Global Motion Description in Block Coded Sequences - Heuer, J.; Kaup, André (Workshop on Image Analysis for Multimedia Interactive Services, 1999)
LMS::458

Fast Exclusion of Angular Intra Prediction Modes in HEVC Using Reference Sample Variance - Heindel, Andreas; Kaup, André (IEEE International Symposium on Circuits and Systems (ISCAS), 2016)
Intra picture coding using HEVC is very efficient and is applied to video as well as single image compression. However, the cost for this high compression efficiency is the complexity caused by the high number of 35 available coding modes. Existing methods for fast mode decision estimate the mode costs based on the prediction error samples. This paper proposes a smart method to exclude the 33 angular prediction modes of HEVC even before prediction, if they are unlikely to deliver a small prediction error. The sensitivity of excluding these modes can be adjusted by a threshold. Completely disregarding the angular modes leads to a very high bitrate increase compared to the unmodified encoder of almost 18% with a time saving of 51%. However, excluding the modes using an exemplary threshold value, the encoding time can be reduced by more than 20%, accompanied by a bitrate increase of only 0.7

Fast image filters as an alternative to reconstruction kernels in Computed Tomography - Flohr, T.; Schaller, Stefan; Stadler, A.; Brandhuber, Wolfgang; Niethammer, M.; Klingenbeck-Regn, K.; Steffen, Peter (SPIE: Medical Engineering, 2001)

Fast intra mode decision in HEVC using early distortion estimation - Heindel, Andreas; Kaup, André (IEEE China Summit and International Conference on Signal and Information Processing, ChinaSIP 2015, 2015)
The latest video coding standard HEVC has the potential to rapidly conquer many areas of video compression due to the significant boost in coding efficiency compared to its predecessor. Unfortunately there is the drawback that this gain in efficiency comes at the expense of complexity. Encoders have to select the best possible coding mode from a huge set of available modes offered by the standard. In this paper the intra mode selection process is accelerated by early estimation of the distortion introduced by a certain mode, which may be applied in addition to other fast mode decision schemes. Due to the online adaptation of our proposed estimator, the approach is tolerant to different coding settings and input sequences. Using early distortion estimation results in a potential encoder speed-up of 24.83% on average, accompanied by an average bitrate increase of 2.92
Keywords: Distortion estimation; HEVC; mode selection; rate-distortion optimization (RDO)

Fast orthogonality deficiency compensation for improved frequency selective image extrapolation - Seiler, Jürgen; Kaup, André (2008 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2008)
The purpose of this paper is to introduce a very efficient algorithm for signal extrapolation. It can widely be used in many applications in image and video communication, e. g. for concealment of block errors caused by transmission errors or for prediction in video coding. The signal extrapolation is performed by extending a signal from a limited number of known samples into areas beyond these samples. Therefore a finite set of orthogonal basis functions is used and the known part of the signal is projected onto them. Since the basis functions are not orthogonal regarding the area of the known samples, the projection does not lead to the real portion a basis function has of the signal. The proposed algorithm efficiently copes with this non-orthogonality resulting in very good objective and visual extrapolation results for edges, smooth areas, as well as structured areas. Compared to an existent implementation, this algorithm has a significantly lower computational complexity without any degradation in quality. The processing time can be reduced by a factor larger than 100. ©2008 IEE
Keywords: Error concealment; Image processing; Prediction; Signal extrapolation

Fast Public-Key Watermarking of Compressed Video - Hartung, Frank; Girod, Bernd (Proceedings IEEE International Conference on Image Processing (ICIP 97), 1997)
LMS::413

Fast Reconstruction of Three-Quarter Sampling Measurements Using Recurrent Local Joint Sparse Deconvolution and Extrapolation - Grosche, Simon; Regensky, Andy; Sinn, Alexander; Seiler, Jürgen; Kaup, André (IEEE Transactions on Circuits and Systems II: Express Briefs, 2022)

Fast Search for Long-Term Memory Motion-Compensated Prediction - Wiegand, Thomas; Lincoln, Bo; Girod, Bernd (IEEE International Conference on Image Processing, 1998)
LMS::338

Fast transcoding of intra frames between H.263 and H.264 - Bialkowski, Jens; Kaup, André; Illgner, K. (2004 International Conference on Image Processing, ICIP 2004, 2004)
In this paper we present algorithms developed for transcoding predicted intra macroblocks between H.263 and H.264. It can be used in video servers for fast adaption of video content in networks supplying different coding standards, e.g. internet and UMTS. It is shown that due to basic pattern similarities between frequency domain prediction of H.263 and spatial prediction of H.263 it is possible to reuse predictor side information. This side information is used to simplify the mode and direction decision for intra prediction. It results in a high reduction of complexity for mode and direction estimation at comparable PSNR and a slightly increased rate. © 2004 IEEE.

Fast Video Transcoding from H.263 To H. 264/AVC - Bialkowski, Jens; Barkowsky, Marcus; Kaup, André (Multimedia Tools and Applications, 2007)
In the past 10 years detailed works on different video transcoders have been published. However, the new ITU-T Recommendation H.264-also adapted as ISO/IEC MPEG-4 Part 10 (AVC)-provides many new encoding options for the prediction processes that lead to difficulties for low complexity transcoding. In this work we present very fast transcoding techniques to convert H.263 bitstreams into H.264/AVC bitstreams. We will give reasoning, why the proposed pixel domain approach is advantageous in this scenario instead of using a DCT domain transcoder. Our approach results in less than 9% higher data rate at equivalent PSNR quality compared to a full-search approach. But this rate loss allows the reduction of the search complexity by a factor of over 200 for inter frames and still a reduction of over 70% for intra frames. A comparison to a fast search algorithm is given. We also provide simulation results that our algorithm works for transcoding MPEG-2 to H.264/AVC in the aimed scenario. © 2007 Springer Science+Business Media, LL
Keywords: H.263; H.264/MPEG-4 AVC; Low complexity coding; Video transcoding

Feedback-Based Error Control for Robust Video Transmission - Färber, Niko (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2000)

Feedback Connection for Deep Neural Network-Based Acoustic Modeling - Tran, Dung; Delcroix, Marc; Ogawa, Atsunori; Hümmer, Christian; Nakatani, Tomohiro (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017)
The use of auxiliary features is an effective way to improve the performance of deep neural network (DNN)-based acoustic models. Most approaches use auxiliary features that represent the speaker or the environment. These auxiliary features are usually computed independently of the acoustic model. This paper investigates a types of auxiliary features obtained from the output of a hidden layer that feeds back to the input layer of the network. Since the auxiliary features are extracted from the hidden layer of the network no external information is required such as the speaker or the environment. Experimentally, by forcing the extraction of the auxiliary features from the same networks, we can further improve the performance of the overall network and reduce the total number of parameters used. We tested this approach with different deep neural network architectures including: deep neural networks, convolutional neural networks and unfolded recurrent convolutional networks. We confirmed the effectiveness of this approach on the CHiME3 datase
LMS::1712
Keywords: deep neural network; auxiliary features; bottleneck features; acoustic modeling

Feedback Structures for a Transfer Function Model of a Circular Vibrating Membrane - Schäfer, Maximilian; Rabenstein, Rudolf; Schlecht, Sebastian (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2019)
The attachment of feedback loops to physical or musical systems enables a large variety of possibilities for the modification of the system behavior. Feedback loops may enrich the echo density of feedback delay networks (FDN), or enable the realization of complex boundary conditions in physical simulation models for sound synthesis. Inspired by control theory, a general feedback loop is attached to a model of a vibrating membrane. The membrane model is based on the modal expansion of an initial-boundary value problem formulated in a state-space description. The possibilities of the attached feedback loop are shown by three examples, namely by the introduction of additional mode wise damping; modulation and damping inspired by FDN feedback loops; time-varying modification of the system behavior.<br
Keywords: Sound synthesis; Functional transformations; Feedback loops; Vibrating membrane

Fehlerverschleierung digitaler Fernsehbildsignale bei mobilem Mehrfachempfang - Tröger, Tobias; Heiber, H.; Schmitt, A.; Kaup, André (ITG Fachtagung für Elektronische Medien, 2009)

Fehlerverschleierung für paketierte Sprachübertragung durch Zeitdehnung und Phasenanpassung - Stenger, Alex; Rabenstein, Rudolf; Girod, Bernd (9. Aachener Kolloquium, 1997)
LMS::385

Filterstrukturen für zeitvariante rekursive Systeme - Rabenstein, Rudolf (Mathematische Methoden in der Signalverarbeitung, 1984)
LMS::1586

Finger-Trace Generator for SWYPE Touch Screen Testing Applications - Pöllot, Martin; Springer, Dominic; Schleifer, Ralph; Nitsch, Monika; Kaup, André (7th IEEE International Conference On Consumer Electronics ICCE, 2017)
With the introduction of touch-screens for an increasing number of applications such as smart phones, vending machines and automotive displays, the need for testing input possibilities for such displays arose. For many different test scenarios not only synthetic, but also human-like inputs are desired. This paper describes an approach capable of creating human-like finger-traces for the input method including, but not limited to swyping. In the process, the algorithm will calculate splines that imitate human finger-traces and run through specified control points. These control points are lying on a given keyboard layout and are analyzed by image processing algorithms. Providing different user profiles for varying input styles—such as precise and sloppy, slow and fast—automated test runs on the robustness and accuracy of applications can be realized, considering the finger-trace that was utilize
Keywords: touch screen; finger-trace; spline interpolation; automated testing; swype; image processing; synthetic input method

Finite Difference Time Domain Modeling of Phase Grating Diffusion - Kowalczyk, Konrad; van Walstijn, M. (Proceedings of the 4th International Symposium on Communications, Control and Signal Processing, ISCCSP 2010, 2010)
In this paper, a method for modeling diffusion caused by non-smooth boundary surfaces in simulations of room acoustics using finite difference time domain (FDTD) technique is investigated. The proposed approach adopts the well-known theory of phase grating diffusers to efficiently model sound scattering from rough surfaces. The variation of diffuser well-depths is attained by nesting allpass filters within the reflection filters from which the digital impedance filters used in the boundary implementation are obtained. The presented technique is appropriate for modeling diffusion at high frequencies caused by small surface roughness and generally diffusers that have narrow wells and infinitely thin separators. The diffusion coefficient was measured with numerical experiments for a range of fractional Brownian diffuser
LMS::940
Keywords: acoustic signal processing; acoustic scattering; acoustic refraction; architectural acoustics; digital filters; fractals

First- and Second-order Adaptive Differential Microphone Arrays - Teutsch, Heinz; Elko, G.W. (7th International Workshop on Acoustic Echo and Noise Control, 2001)
LMS::190

Fisheye Video Processing Using Distortion Corrected Block-Based Motion Compensation - Eichenseer, Andrea (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2018)
Surveying wide areas with a single camera is a typical scenario in surveillance and automotive applications. Conventional perspective camera lenses, however, are not able to cover very large fields of view. This is where ultra wide-angle fisheye cameras come into play. By employing fisheye lenses, the standard pinhole model is abandoned in favor of a fisheye projection that is able to accommodate fields of view that may well surpass 180 degrees. The captured fisheye video sequences exhibit image characteristics that deviate notably from conventional rectilinear imagery as obtained by perspective pinhole cameras. These characteristics are not taken into account by traditional video processing techniques such as motion estimation and its subsequent compensation. This thesis therefore investigates fisheye video sequences in the context of motion estimation and various video processing tasks based thereon. After introducing an extensive fisheye data set, a first analysis examines the performance of a state-of-the-art video compression scheme when faced with fisheye input sequences. The main part of the thesis is then dedicated to motion estimation based on block matching and the development of an adapted approach for fisheye video sequences. By incorporating suitable projections, the translational motion model that would otherwise only hold for perspective video sequences may be exploited, thus facilitating the motion search and yielding improved motion compensation results. Exhaustive simulations and experiments on synthetic and real-world fisheye video data demonstrate the merit of the adaptation. Building upon fisheye motion estimation, four motion compensated video processing applications are investigated and adapted to provide improved results for fisheye input data. The considered applications include temporal error concealment, intermediate view synthesis, and temporal as well as spatial resolution enhancement.

FishUI: Interactive Fisheye Distortion Visualization - Regensky, Andy; Herglotz, Christian; Kaup, André (IEEE International Conference on Visual Communications and Image Processing (VCIP), 2020)
Fisheye lenses provide major benefits for many applications due to their large field of view. However, they come at the cost of strong radial distortions leading to problems in a variety of signal processing tasks which have been developed with perspective lenses in mind. As such, while state-of-the-art image and video codecs excel in reducing redundancy and irrelevance in content captured with perspective lenses, the coding gain reduces significantly when fisheye lenses are applied. To improve the understanding of distortions introduced by fisheye lenses with respect to perspective lenses, we provide an interactive user interface for the visualization of fisheye block distortions.<br

Fixed pattern noise column drift compensation (CDC) for digital moving picture cameras - Schoeberl, Michael; Foessel, Siegfried; Kaup, André (2010 17th IEEE International Conference on Image Processing, ICIP 2010, 2010)
In CMOS image sensors tiny semiconductor variations cause a distortion of the image known as fixed pattern noise (FPN). With a good FPN compensation CMOS sensors can deliver very good image quality. Compensation algorithms differ in complexity, maximum frame rate, and compensation quality with thermal drift. This paper analyzes existing approaches to offset FPN compensation and their drawbacks. We also show a new compensation algorithm that offers fine grained per-pixel compensation and is able to compensate temperature variations while still operating at the maximum sensor frame rate. This enables the construction of motion picture cameras without active cooling or even temperature measurements while still delivering a high image quality at high frame rates. © 2010 IEE
Keywords: Cameras; Image generation; Imaging; Noise

Forensic analysis of AI-compression traces in spatial and frequency domain - Bergmann, Sandra; Moussa, Denise; Brand, Fabian; Kaup, André; Rieß, Christian (Pattern Recognition Letters, 2024)
The classical JPEG compression is a rich source of cues for forensic image analysis. However, this compression standard will in the near future be complemented by a new, highly efficient learning-based compression standard called JPEG AI. JPEG AI is fundamentally different from classical JPEG. Hence, its forensic traces can also be expected to be fundamentally different. We argue that there is a pressing need for image forensics research to investigate these traces. In this work, we characterize forensic compression traces of different AI compression algorithms. Our analysis investigates AI compression artifacts in frequency domain and in spatial domain. Both domains exhibit similar artifacts that likely stem from upsampling operations of the decoders. Additionally, we report for one AI codec another artifact in homogeneous regions. We also investigate the artifact detectability in several scenarios including unseen AI compression traces and postprocessing. Here, frequency and autocorrelation features are better on additive noise and classical JPEG post-compression, while RGB features perform better on blurred and downsampled images.

Forensic License Plate Recognition with Compression-Informed Transformers - Moussa, Denise; Maier, Anatol; Spruck, Andreas; Seiler, Jürgen; Rieß, Christian (2022 IEEE International Conference on Image Processing (ICIP), 2022)
Forensic license plate recognition (FLPR) remains an open challenge in legal contexts such as criminal investigations, where unreadable license plates (LPs) need to be deciphered from highly compressed and/or low resolution footage, e.g., from surveillance cameras. In this work, we propose a side-informed Transformer architecture that embeds knowledge on the input compression level to improve recognition under strong compression. We show the effectiveness of Transformers for license plate recognition (LPR) on a low-quality real-world dataset. We also provide a synthetic dataset that includes strongly degraded, illegible LP images and analyze the impact of knowledge embedding on it. The network outperforms existing FLPR methods and standard state-of-the art image recognition models while requiring less parameters. For the severest degraded images, we can improve recognition by up to 8.9 percent points.<br
Keywords: License Plate Recognition; Image Forensics

Formangepasste diskrete Cosinus Transformation für die Prädiktionsverbesserung im HEVC - Wige, Eugen; Heindel, Andreas; Kaup, André (15. ITG-Fachtagung für Elektronische Medien, 2013)
Um die Kompression in der Videocodierung zu verbessern, führen wir eine explizite Referenzbildentrauschung in die Codierschleife eines Videocodecs ein. Motiviert durch den Gedanken, dass die Leistung des Prädiktionsfehlers höher sein kann, falls Rauschen in dem zu codierenden Video vorhanden ist, wird die Bewegungskompensation durch die eingeführten Module verbessert. Es wird gezeigt wie man einen solchen Ansatz für die Codierung bei sehr kleinen Einstellungen des Quantisierungsparameters aber auch bei sehr groben Quantisierungseinstellungen verwenden kann. Die entwickelten Algorithmen wurden in der Referenzsoftware des aktuellen HEVC-Standards getestet. Die Simulationsergebnisse zeigen, dass mit der vorgeschlagenen Vorgehensweise maximale Bitratenersparnisse von bis zu 10 % für niedrige als auch hohe Quantisierungsparametereinstellungen erreicht werden können. Im Durchschnitt wurden Bitratenersparnisse von 7 % für hohe Qualität und 5 % für niedrige Qualität bei Codierung der ClassB-Sequenzen erreich

Formulation of the REMOS concept from an uncertainty decoding perspective - Maas, Roland; Sehr, Armin; Yoshioka, Takuya; Delcroix, Marc; Kinoshita, Keisuke; Nakatani, Tomohiro; Kellermann, Walter (International Conference on Digital Signal Processing (DSP), 2013)
In this paper, we introduce a new formulation of the REMOS (REverberation MOdeling for Speech recognition) concept from an uncertainty decoding perspective. Based on a convolutive observation model that relaxes the conditional independence assumption of hidden Markov models, REMOS effectively adapts automatic speech recognition (ASR) systems to noisy and strongly reverberant environments. While uncertainty decoding approaches are typically designed to operate irrespectively of the employed decoding routine of the ASR system, REMOS explicitly considers the additional information provided by the Viterbi decoder. In contrast to previous publications of the REMOS concept, we provide a conclusive derivation of its decoding routine using a Bayesian network representation in order to prove its inherent uncertainty decoding character. © 2013 IEE
Keywords: Automatic speech recognition; Noise robustness; Reverberation robustness; Uncertainty decoding; Viterbi decoding

Fourieranalyse als mathematisches Modell zur Auswertung und Darstellung von postoperativen Hornhauttopographiedaten nach nichtmechanischer perforierender Keratoplastik - Langenbucher, A.; Seitz, B.; Kus, M.; Steffen, Peter; Naumann, G. (Klinische Monatsblätter für Augenheilkunde, 1997)
LMS::403

Fractional Montgomery effect: a self-imaging phenomenon - Lohmann, Adolf; Knuppertz, H.; Jahns, J. (Journal of the Optical Society of America A-Optics Image Science and Vision, 2005)
LMS::652

Frame rate conversion from higher to lower frame rates in motion picture - Foessel, S.; Schöberl, Michael (SMPTE Annual Technical Conference - Symposium on High Frame Rates for Digital Cinema, 2012)
LMS::1371

Frame Rate Up-Conversion Using Key Point Agnostic Frequency-Selective Mesh-to-Grid Resampling - Heimann, Viktoria; Spruck, Andreas; Kaup, André (IEEE International Conference on Acoustics, Speech and Signal Processing, 2021)
High frame rates are desired in many fields of application. As in many cases the frame repetition rate of an already captured video has to be increased, frame rate up-conversion (FRUC) is of high interest. We conduct a motion compensated approach. From two neighboring frames, the motion is estimated and the neighboring pixels are shifted along the motion vector into the frame to be reconstructed. For displaying, these irregularly distributed mesh pixels have to be resampled onto regularly spaced grid positions. We use the model-based key point ag- nostic frequency-selective mesh-to-grid resampling (AFSMR) for this task and show that AFSMR works best for applications that contain irregular meshes with varying densities. AFSMR gains up to 3.2 dB in contrast to the already high performing frequency-selective mesh-to-grid resampling (FSMR). Addition- ally, AFSMR increases the run time by a factor of 11 relative to FSM

Frame-wise HMM adaptation using state-dependent reverberation estimates - Sehr, Armin; Maas, Roland; Kellermann, Walter (36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 2011)
A novel frame-wise model adaptation approach for reverberation-robust distant-talking speech recognition is proposed. It adjusts the means of static cepstral features to capture the statistics of reverberant feature vector sequences obtained from distant-talking speech recordings. The means of the HMMs are adapted during decoding using a state-dependent estimate of the late reverberation determined by joint use of a feature-domain reverberation model and optimum partial state sequences. Since the parameters of the HMMs and the reverberation model can be estimated completely independently, the approach is very flexible with respect to changing acoustic environments. Due to the frame-wise model adaptation, some of the HMM limitations are relieved, and recognition results surpassing that of matched reverberant training are obtained at the cost of a moderately increased decoding complexity. © 2011 IEE
Keywords: acoustic modeling; distant-talking ASR; frame-wise HMM adaptation; reverberation modeling; Robust speech recognition

Framewise repeated coefficient updates for enhanced nonlinear AEC by diagonal coordinate volterra filters - Zeller, Marcus; Kellermann, Walter (2008 Hands-free Speech Communication and Microphone Arrays, HSCMA 2008, 2008)
In this paper, we extend the approach of iterated coefficient updates to DFT-domain Volterra filters in diagonal coordinates. This technique, which has recently been successfully applied to the partitioned block frequency-domain Volterra filter, repeatedly uses the same block of data of the DFT-domain LMS adaptation and thus performs several updates per frame. Therefore, the available signal innovation is exploited more efficiently as this procedure can be implemented using a recursive scheme which demands a relatively moderate increase in computational power compared to the significantly improved performance. The benefits of such an approach are demonstrated by experimental results regarding both the echo return loss enhancement (ERLE) and the coefficient error norms of higher-order kernels for both noise and real speech signals. ©2008 IEE
Keywords: Adaptive volterra filter; Iterative methods; Nonlinear system identification

Freeze detection in 2D navigation video sequences by matching of extracted line segments - Yammine, Gilbert; Wige, Eugen; Simmet, F.; Niederkorn, D.; Kaup, André (2012 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2012, 2012)
In this paper, we present an automated error/artifact detection framework that monitors the infotainment system of the car. Specifically, this paper focuses on the detection of freezing artifacts that could occur in the navigation system of the car during field tests. Knowing that the motion in the navigation system could also stop when the car stops, it is crucial to differentiate between this situation and a real freezing artifact. We propose an algorithm that reliably detects map jumps which occur after the freezing artifacts. The proposed algorithm extracts lines from the possibly frozen frame and the frame following the freeze event and matches the extracted lines together in order to output a geometric transformation matrix that describes the motion between both frames. If the motion is larger than normal, a map jump is detected and a freeze event is signaled. © 2012 IEE

Freeze detection in 2D navigation video sequences overlaid with real satellite images - Yammine, Gilbert; Khairat, A.; Simmet, F.; Niederkorn, D.; Kaup, André (2012 IEEE 14th International Workshop on Multimedia Signal Processing, MMSP 2012, 2012)
This paper focuses on the detection of freezing artifacts that could occur in the navigation system of the car during field tests. Knowing that the motion in the navigation system could also stop when the car stops, it is crucial to differentiate between this situation and a real freezing artifact. We propose an algorithm that reliably detects map jumps which occur after the freezing artifacts. The proposed algorithm extracts invariant features from the possibly frozen frame and the frame following the freeze event and matches the features in order to output a geometric transformation that describes the motion between both frames. If the motion is larger than normal, a map jump is detected and a freeze event is signaled. © 2012 IEE

Frequency-Domain Analysis of Traces for the Detection of AI-based Compression - Bergmann, Sandra; Moussa, Denise; Brand, Fabian; Kaup, André; Rieß, Christian (2023 International Workshop on Biometrics and Forensics (IWBF), 2023)
The JPEG algorithm is the most popular compression method on the internet. Its properties have been extensively studied in image forensics for examining image origin and authenticity. However, the JPEG standard will in the near future be extended with AI-based compression. This approach is fundamentally different from the classic JPEG algorithm, and requires an entirely new set of forensics tools. As a first step towards forensic tools for AI compression, we present a first investigation of forensic traces in HiFiC, the current state-of-the-art AI-based compression method. We investigate the frequency space of the compressed images, and identify two types of traces, which likely arise from GAN upsampling and in homogeneous areas. We evaluate the detectability on different patch sizes and unseen postprocessing, and report a detectability of 96.37%. Our empirical results also suggest that further, yet unidentified, compression traces can be expected in the spatial domain.<br
Keywords: AI-based compression; frequency analysis

Frequency-domain integration of acoustic echo cancellation and a generalized sidelobe canceller with improved robustness - Herbordt, Wolfgang; Kellermann, Walter (European Transactions on Telecommunications, 2002)
For hands-free acoustic human/machine interfaces, as required, e.g., for automatic speech recognition, teleconferencing, and other multimedia services, microphone arrays using generalized sidelobe cancellers (GSCs) in conjunction with acoustic echo cancellation (AEC) can be efficiently applied for optimum communication. This contribution first devises a new structure for combining AEC and GSC in the frequency domain. We show that computational complexity is reduced by more than a factor of ten compared to a time-domain arrangement. Second, robustness issues of the GSC adaptation mechanism are addressed. We illustrate how robust adaptation is assured in the new structure. Third, we propose alternatives for the fixed beamformer part of the GSC, which allow to specify (a) the width of the target tracking region and (b) improve the suppression of low frequencies. GSC robustness constraints are included using linear optimization with configurable penalty functions.

Frequency-Domain Mimo Acoustic Echo Cancellation Based on A Kronecker Product Approximation - Halimeh, Mhd Modar; Kellermann, Walter (International Workshop on Acoustic Signal Enhancement, IWAENC 2022 - Proceedings, 2022)
As the number of loudspeakers and microphones in modern devices grows, the task of Acoustic Echo Cancellation (AEC) becomes increasingly computationally expensive as the echo canceller requires the optimization of a large number of adaptive filters. In this paper, an efficient frequency-domain Multiple-Input/Multiple-Output (MIMO) AEC approach is introduced using a Kronecker product-based approximation of the echo paths. In the proposed approach, the trade-off between the approximation accuracy and the computational complexity is controlled by a tuning parameter. This approach is verified for both, time-varying and time-invariant rendering systems and the results show that the proposed approach achieves steady-state performance similar to that of a conventional method at a much lower computational cost.
CRIS-Team Scopus Importer:2022-11-25

Frequency selective extrapolation with residual filtering for image error concealment - Koloda, Jan; Seiler, Jürgen; Kaup, André; Sanchez, V.; Peinado, A. (2014 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2014, 2014)
The purpose of signal extrapolation is to estimate unknown signal parts from known samples. This task is especially important for error concealment in image and video communication. For obtaining a high quality reconstruction, assumptions have to be made about the underlying signal in order to solve this underdetermined problem. Among existent reconstruction algorithms, frequency selective extrapolation (FSE) achieves high performance by assuming that image signals can be sparsely represented in the frequency domain. However, FSE does not take into account the low-pass behaviour of natural images. In this paper, we propose a modified FSE that takes this prior knowledge into account for the modelling, yielding significant PSNR gains. © 2014 IEE
Keywords: error concealment; Image processing

Frequency-Selective Geometry Upsampling of Point Clouds - Heimann, Viktoria; Spruck, Andreas; Kaup, André (IEEE International Conference on Image Processing (ICIP) 2022, 2022)

Frequency Selective Mesh-to-Grid Resampling for Image Communication - Koloda, Jan; Seiler, Jürgen; Kaup, André (IEEE Transactions on Multimedia, 2017)
This paper presents a novel approach for image reconstruction from pixels located at arbitrary non-integer positions, called mesh. This task forms an intrinsic part of various multimedia applications, including super-resolution, fisheye imaging or generations of new views in multicamera systems, among others. We propose a new frequency selective mesh-to-grid resampling (FSMR) algorithm that aims at producing high quality reconstructions. It is inspired by the existing frequency selective reconstruction (FSR) algorithm that is known to exhibit high performance when pixels are located on the regular two-dimensional grid. However, if samples that are located at non-integer positions are involved, a severe overfitting problem arises from the fact that non-orthogonal weighted bases sampled at non-integer positions are used for signal modelling. In order to overcome this issue, we propose a novel stabilising mechanism that is based on a set of adaptively weighted initial estimates, called key points. We also show that Fourier basis, used in the classic grid-based FSR, yields complex signals when non-integer positions are involved. Since digital images are real-valued, we propose to employ two-dimensional cosine transform basis. Experimental results show the superiority of the proposed approach over a wide range of existing reconstruction technique
Keywords: Image reconstruction; non-integer pixel positions; frequency selective resampling

Frequency-Selective Mesh-to-Mesh Resampling for Color Upsampling of Point Clouds - Heimann, Viktoria; Spruck, Andreas; Kaup, André (Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP), 2021)
With the increased use of virtual and augmented reality applications, the importance of point cloud data rises. High-quality capturing of point clouds is still expensive and thus, the need for point cloud super-resolution or point cloud upsampling techniques emerges. In this paper, we propose an interpolation scheme for color upsampling of three-dimensional color point clouds. As a point cloud represents an object’s surface in three-dimensional space, we first conduct a local transform of the surface into a two-dimensional plane. Secondly, we propose to apply a novel Frequency-Selective Mesh-to-Mesh Resampling (FSMMR) technique for the interpolation of the points in 2D. FSMMR generates a model of weighted superpositions of basis functions on scattered points. This model is then evaluated for the final points in order to increase the resolution of the original point cloud. Evaluation shows that our approach outperforms common interpolation schemes. Visual comparisons of the jaguar point cloud underlines the quality of our upsampling results. The high performance of FSMMR holds for various sampling densities of the input point clou
Keywords: point cloud; color upsampling; frequency- selective

Frequency Selective Reconstruction of Non-Regularly Sampled Image and Video Data - Jonscher, Markus (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2018)
Video data with both a high spatial and a high temporal resolution is desired for many applications. The processing throughput of state-of-the-art cameras, however, is limited to a certain number of pixels per second which leads for most of these cameras to a trade-off between acquiring a video sequence at a high spatial resolution but a low temporal resolution or vice versa. In order to overcome these limitations, a non-regular sampling using a subsequent reconstruction algorithm may be applied. This thesis focuses on the reconstruction of non-regularly sampled image and video data. Various enhancements for the applied reconstruction algorithm are introduced that improve the reconstruction quality on the one hand or reduce the computational complexity on the other hand. Techniques that exploit temporal or spatial correlations when using a non-regular sampling sensor in multi-frame or multi-view scenarios show promising results. Additionally, a novel dynamic non-regular sampling strategy is presented that allows the readout of varying sampling patterns over time and in combination with the aforementioned reconstruction algorithm, significant gains in reconstruction quality may be achieved.

Frequency selective signal extrapolation with applications to error concealment in image communication - Kaup, André; Meisinger, Katrin; Aach, T. (AEÜ - International Journal of Electronics and Communications, 2005)
Signal extrapolation is an important topic in image communication where it can be used for various prediction purposes such as concealment of image data corrupted by transmission errors. This contribution outlines a method for extrapolating a signal beyond a limited number of known samples. The known signal samples are approximated by a set of basis functions which are defined over an area covering known as well as unknown samples. By minimizing a suitable error criterion and successively selecting the most dominant basis functions, a non band-limited signal extrapolation can be obtained. It is shown that this extrapolation can successfully be used for concealment of transmission errors in video communication as well as elimination of defective pixels in X-ray imaging. © 2005 Elsevier GmbH. All rights reserve
Keywords: Error concealment; Image and video communication; Signal extrapolation; X-ray imaging

Fresnel similarity - Lohmann, Adolf; Ojeda-Castaneda, J. (Optics Communications, 2005)
We study Fresnel diffraction, in particular the similarity between an object, at plane z = 0, and its diffraction pattern, at z > 0. We define a degree of similarity, σ(z), which depends on the object and the distance z. Perfect self-imaging is associated with a degree of similarity equal to unity, while imperfect self-imaging is classified by a degree between zero and one. We derive the theory that allows us to find σ(z), if the object is known. And we can synthesize an object that exhibits a wanted σ(z). We describe some novel applications. © 2005 Published by Elsevier B.
LMS::648

Fricative phoneme detection using deep neural networks and its comparison to traditional methods - Yurt, Metehan; Kantharaju, Pavan; Disch, Sascha; Niedermeier, Andreas; Escalante-B, Alberto N.; Morgenshtern, Veniamin (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2021)
Accurate phoneme detection and processing can enhance speech intelligibility in hearing aids and audio & speech codecs. As fricative phonemes have an important part of their energy concentrated in high frequency bands, frequency lowering algorithms are used in hearing aids to improve fricative intelligibility for people with high-frequency hearing loss. In traditional audio codecs, while processing speech in blocks, spectral smearing around fricative phoneme borders results in pre and post echo artifacts. Hence, detecting the fricative borders and adapting the processing accordingly could enhance the quality of speech. Until recently, phoneme detection and analysis were mostly done by extracting features specific to the class of phonemes. In this paper, we present a deep learning based fricative phoneme detection algorithm that exceeds the state-of-the-art fricative phoneme detection accuracy on the TIMIT speech corpus. Moreover, we compare our method to other approaches that employ classical signal processing for fricative detection and also evaluate it on the TIMIT files coded with AAC codec followed by bandwidth limitation. Reported results of our deep learning approach on original TIMIT files are reproducible and come with an easy to use code that could serve as a baseline for any future research on this topic.
CRIS-Team Scopus Importer:2021-11-26
Keywords: Deep neural networks; Fricative phonemes; Machine learning; Phoneme detection; TIMIT speech corpus

From Blind to Semi-Blind Acoustic Source Separation based on Independent Component Analysis - Brendel, Andreas (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2022)

From direction of arrival estimates to localization of planar reflectors in a two dimensional geometry - Canclini, A.; Annibale, Paolo; Antonacci, F.; Sarti, A.; Rabenstein, Rudolf; Tubaro, S. (36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 2011)
In this paper we propose a novel technique to localize planar obstacles through the measurement of the Direction of Arrival by a microphone array. The measurement of the Direction of Arrival of the reflected path is turned into a quadratic constraint where the unknowns are the line parameters of the reflector. A cost function that combines multiple constraints is then derived. A parametric description of the obstacle is found by minimization of the cost function. Some simulations and experimental results show the feasibility of the proposed method. © 2011 IEE
Keywords: Direction of Arrival; Microphone Array; Reflector Localization

Full-Duplex Communication Systems with Loudspeaker Arrays and Microphone Arrays - Buchner, Herbert; Spors, Sascha; Kellermann, Walter; Rabenstein, Rudolf (2002 IEEE International Conference on Multimedia and Expo, ICME 2002, 2002)
For high-quality multimedia communication systems, such as teleconferencing, or tele-teaching (especially of music), multichannel sound reproduction is highly desirable. While current approaches still rely on a restrained listening area, the sweet spot, a volume solution for a large listening space is offered by the wave field synthesis (WFS) method, where arrays of loudspeakers generate a prespecified sound field. On the recording side of the two-way systems, the use of microphone arrays is an effective approach to cope with undesired signal components in the receiving room. However, before full-duplex communication can be deployed, efficient approaches to the acoustic echo cancellation (AEC) problem in this challenging scenario have to be found. We investigate different options for system integration, after a brief discussion of the current state of the art. We then present a first real-time solution on a regular PC platform, based on an efficient AEC for MIMO (multi-input and multi-output) systems in the frequency-domain.

Full-duplex multichannel communication: Real-time implementations in a general framework - Herbordt, Wolfgang; Buchner, Herbert; Kellermann, Walter; Rabenstein, Rudolf; Spors, Sascha; Teutsch, Heinz (IEEE Int. Conf. on Multimedia and Expo, 2003)
In this contribution, we embed full-duplex multichannel communication interfaces for tele-presence systems into a general framework. On the reproduction side, we consider a wide range of multichannel acoustic rendering techniques including traditional stereophony, '5.1' systems, and wave field synthesis using loudspeaker arrays for sound immersion. On the recording side, microphone arrays are discussed for capturing clean desired signals with spatial information. Based on this general framework, real-time implementations of such full-duplex multichannel communication systems are then described. We combine wave field synthesis with multichannel acoustic echo cancellation and adaptive beamforming and discuss a real-time implementation on standard desktop and laptop PCs.

Full-Duplex Systems for Sound Field Recording and Auralization Based on Wave Field Synthesis - Buchner, Herbert; Spors, Sascha; Kellermann, Walter (Conf. Rec. AES 116th Convention, 2004)
LMS::487

Fundamental Approaches to Robust Differential Beamforming With High Directivity Factors - Huang, Gongping; Benesty, Jacob; Chen, Jingdong (IEEE/ACM Transactions on Audio, Speech and Language Processing, 2022)
Differential beamforming, which measures the spatial derivatives of the acoustic pressure field, can be used in a wide range of small devices that require high-fidelity sound and speech acquisition as it can achieve frequency-invariant spatial responses with high directivity factors (DFs). Since a differential process is inherently sensitive to sensors' self noise and other array imperfections, the most challenging problem in the design of any differential beamformer is how to achieve the maximum possible DF while maintaining a proper level of robustness for practical usage. While significant efforts have been made on this topic, the problem remains unsolved and further study is indispensable. This paper is devoted to dealing with this challenging problem. It presents a study on theory and methods to achieve the optimal and fundamental compromise between the white noise gain (WNG), which quantifies how robust is the beamformer, and the DF in differential beamforming. The major contributions of this work are as follows. 1) We show and prove that any null constrained fixed beamformer can be decomposed as the sum of two orthogonal filters, i.e., the maximum WNG (MWNG) beamformer and a reduced-rank one. Based on this decomposition, we develop three kinds of differential beamformers from the WNG perspective, which can achieve a flexible and optimal compromise between DF and WNG. 2) We show that a transformed null constrained beamformer can also be decomposed as the sum of two orthogonal filters, i.e., the transformed maximum DF (MDF) beamformer and another reduced-rank one. Based on this decomposition, we also develop three kinds of differential beamformers, which can obtain the desired level of DF while using the rest of the degrees of freedom to maximize the WNG. Simulations are performed to validate the theoretical analysis and developed differential beamformers.
CRIS-Team Scopus Importer:2022-10-21
Keywords: Acoustics; Array signal processing; Beampattern; differential beamforming; directivity factor; microphone arrays; Microphone arrays; quadratic eigenvalue problem; robust beamforming; Robustness; Sensor arrays; Sensors; White noise; white noise gain

Fundamental Performance Limits of Power-Spectrum Condition-Compliant Watermarks - Su, Jonathan; Girod, Bernd (Proc. SPIE Security and Watermarking of Multimedia Contents II, 2000)
LMS::136

Fusion of Multisensor Data - Strobel, Norbert (Principles of 3D Image Analysis and Synthesis, 2000)

Generalized Coherence-based Signal Enhancement - Löllmann, Heinrich; Brendel, Andreas; Kellermann, Walter (Internationl Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020)
This contribution presents a novel approach for coherence-based signal enhancement. An estimator for the coherent-to-diffuse ratio (CDR) is devised, which exploits the concept of generalized magnitude coherence and thus, unlike common state-of-the-art schemes, can simultaneously take advantage of more than two microphones. Moreover, the speech enhancement by CDR-based spectral weighting is not performed as a post-filtering step, but by enhancing the most appropriate microphone signal. This signal is implicitly determined as part of the CDR estimation such that the presented technique does not depend on an estimation of the direction-of-arrival (DOA) or similar side-information about the desired source. The application of the new approach to binaural hearings aids shows that it achieves a consistently better speech enhancement performance than comparable state-of-the-art approaches.<br
Keywords: CDR estimation; generalized coherence; multichannel signal enhancement; dereverberation; hearing aids

Generalized multichannel frequency-domain adaptive filtering: Efficient realization and application to hands-free speech communication - Buchner, Herbert; Benesty, J.; Kellermann, Walter (Signal Processing, 2005)
In unknown environments where we need to identify, model, or track unknown and time-varying channels, adaptive filtering has been proven to be an effective tool. In this contribution, we focus on multichannel algorithms in the frequency domain that are especially well suited for input signals which are not only auto-correlated but also highly cross-correlated among the channels. These properties are particularly important for applications like multichannel acoustic echo cancellation. Most frequency-domain algorithms, as they are well known from the single-channel case, are derived from existing time-domain algorithms and are based on different heuristic strategies, e.g, for stepsize normalization. Here, we present a new rigorous derivation of a whole class of multichannel adaptive filtering algorithms in the frequency domain based on a recursive least-squares criterion. Then, from the normal equation, we derive a generic adaptive algorithm in the frequency domain. Due to the rigorous approach, the proposed framework inherently takes the coherence between all input signal channels into account. An analysis of this multichannel algorithm shows that the mean-squared error convergence is independent of the input signal statistics (i.e., both auto-correlation and cross-correlation). A useful approximation provides interesting links between some well-known algorithms for the single-channel case and the general multichannel framework. We also give design rules for important parameters to optimize the performance in practice. The computational complexity is kept low by introducing several new techniques, such as a robust recursive Kalman gain computation in the frequency domain and efficient fast Fourier transform (FFT) computation tailored to overlapping data blocks. Simulation results and real-time performance for applications such as multichannel acoustic echo cancellation show the high efficiency of the approach. © 2004 Elsevier B.V. All rights reserve
Keywords: Adaptive filtering; Beamforming; Echo cancellation; Frequency domain; Multichannel

Generalized Nonlinear System Identification using Adaptive Volterra Filters with Evolutionary Kernels - Zeller, Marcus (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2013)
Goal: This research was on adaptive filter structures that are suitable for a generalized nonlinear system identification scheme. Particular focus was on efficient implementations in both time and DFT domain, especially for applications in the field of acoustic echo cancellation (e.g. including nonlinear distortions by low-cost loudspeakers/microphones). However, the developed approach of self-configuring, transversal adaptive filters based on Volterra structures is applicable to a large class of nonlinear system identification problem

Generalized wave-domain transforms for listening room equalization with azimuthally irregularly spaced loudspeaker arrays - Hofmann, Christian; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2016)
In reverberant environments, Listening Room Equalization (LRE) by pre-filtering of loudspeaker signals is highly desirable for premium sound reproduction systems with a high number of loudspeakers. In this contribution, the efficient concept of LRE by wave-domain adaptive filtering is extended by deriving generalized loudspeaker signal transforms for loudspeaker arrays with an irregular azimuthal spacing. In particular, a matrix approximation allows to restore the unitarity property of a transform matrix which lost the unitarity when applied to irregular arrays instead of regular ones. Simulations of adaptive LRE systems with irregular loudspeaker arrays confirm the efficacy of the proposed, novel, generalized transform
Keywords: WDAF; Wave Domain; Azimuthally Irregular Arrays; Spatial Transforms; Equalization

Generation of dual N-Channel Filterbanks with Hilbert-Transformed Mother Wavelets - Steffen, Peter; Brandhuber, Wolfgang (Proceedings European Signal Processing Conference (EUSIPCO), 2008)
We reconsider the problem of generating a dual filterbank to a given one with N channels such that its N - 1 mother wavelets will be related to those of the original one by the Hilbert-transform. Known solutions treat the case of unitary filterbanks, while treats the biorthogonal case and the overcomplete case. We present a solution which is valid for arbitrary filterbanks and obtain identical results. This fact indicates that the proposition of unitarity is not necessary at all. Moreover, the phase condition is obtained in a constructive way and is given by a closed form expressio
LMS::814

Geometrical interpretation of the PCA subspace approach for overdetermined blind source separation - Winter, Stefan; Sawada, H.; Makino, Shoji (EURASIP J APPL SIG P, 2006)
We discuss approaches for blind source separation where we can use more sensors than sources to obtain a better performance. The discussion focuses mainly on reducing the dimensions of mixed signals before applying independent component analysis. We compare two previously proposed methods. The first is based on principal component analysis, where noise reduction is achieved. The second is based on geometric considerations and selects a subset of sensors in accordance with the fact that a low frequency prefers a wide spacing, and a high frequency prefers a narrow spacing. We found that the PCA-based method behaves similarly to the geometry-based method for low frequencies in the way that it emphasizes the outer sensors and yields superior results for high frequencies. These results provide a better understanding of the former metho
LMS::596

Geometrical Interpretation of the PCA Subspace Method for Overdetermined Blind Source Separation - Winter, Stefan; Sawada, H.; Makino, Shoji (Proc. IEEE Int. Symposium on Independent Component Analysis and Blind Signal Separation (ICA), 2003)
LMS::534

GEOMETRICALLY CONSTRAINED SOURCE EXTRACTION AND DEREVERBERATION BASED ON JOINT OPTIMIZATION - Yang, Yichen; Wang, Xianrui; Brendel, Andreas; Zhang, Wen; Kellermann, Walter; Chen, Jingdong (European Signal Processing Conference, 2023)
Source extraction, which aims at extracting the target source signals from the observed reverberant mixtures, plays an important role in voice communication and human-machine interfaces. Among the numerous source extraction methods that have been developed, the geometrically constrained (GC) one, which incorporates the direction-of-arrival (DOA) information of the target signals, has demonstrated great potential. However, this method generally suffers from significant performance degradation in strong reverberant environments since it is challenging to obtain in such environments accurate DOA estimates that are needed by the algorithm. To address this problem, we present in this work an iterative algorithm, which integrates the source-wise weighted prediction error (WPE)based dereverberation principle with the geometrically constrained source extraction method. We show that this algorithm is able to improve the DOA estimation accuracy as well as the source extraction performance.
CRIS-Team Scopus Importer:2023-12-15
Keywords: Blind source separation; dereverberation; geometrical constraint; semi-blind source extraction

Geometrically constrained TRINICON-based relative transfer function estimation in underdetermined scenarios - Reindl, Klaus; Markovich-Golan, Shmulik; Barfuß, Hendrik; Gannot, Sharon; Kellermann, Walter (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2013)
Speech extraction in a reverberant enclosure using a linearly-constrained minimum variance (LCMV) beamformer usually requires reliable estimates of the relative transfer functions (RTFs) of the desired source to all microphones. In this contribution, a geometrically constrained (GC)-TRINICON concept for RTF estimation is proposed. This approach is applicable in challenging multiple-speaker scenarios and in underdetermined situations, where the number of simultaneously active sources outnumbers the number of available microphone signals. As a most practically relevant and distinctive feature, this concept does not require any voice-activity-based control mechanism. It only requires coarse reference information on the target direction of arrival (DoA). The proposed GC-TRINICON method is compared to a recently proposed subspace method for RTF estimation relying on voice-activity control. Experimental results confirm the effectiveness of GC-TRINICON in realistic conditions. © 2013 IEE
Keywords: Blind source separation (BSS); generalized sidelobe canceller (GSC); linearly-constrained minimum variance (LCMV); relative transfer function estimation

Geometrical Understanding of the PCA Subspace Method for Overdetermined Blind Source Separation - Winter, Stefan; Sawada, H.; Makino, Shoji (Conf. Rec. Intl. Workshop on Acoustic Echo and Noise Control (IWAENC), 2003)

Geometrical Understanding of the PCA Subspace Method for Overdetermined Blind Source Separation - Winter, Stefan; Sawada, H.; Makino, Shoji (Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), 2003)

Geometric inference of the room geometry under temperature variations - Annibale, Paolo; Filos, Jason; Naylor, Patrick A.; Rabenstein, Rudolf (5th International Symposium on Communications Control and Signal Processing, ISCCSP 2012, 2012)
Geometric inference is an approach for localizing reflectors in a closed acoustic space. It is based on a simple observation that turns time differences of arrival (TDOA) or time of arrival (TOA) measurements from the signals of a microphone array into a geometric constraint. The reflector localization methodology relies on accurate TDOA which is directly dependent on speed of sound information. Estimating the actual speed of sound at the ambient temperature therefore greatly improves the accuracy of the reflector localization in uncontrolled environments. This manuscript shows how to use the geometric inference jointly with the speed of sound estimation for a more accurate reflector localization. Simulations and experiments show the validity of the proposed approach. © 2012 IEE
Keywords: channel estimation; Geometric inference; speed of sound estimation; TDOA; TOA

Geometry-Adaptive Multi-View Coding Techniques for Image-based Rendering - Magnor, Marcus (Friedrich-Alexander-Universität Erlangen-Nürnberg, 2001)

Geometry-Corrected Geodesic Motion Modeling with Per-Frame Camera Motion for 360-Degree Video Compression - Regensky, Andy; Kaup, André (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024)

Global Motion Estimation in Image Sequences Using Robust Motion Vector Field Segmentation - Heuer, J.; Kaup, André (ACM Multimedia 99, 1999)
LMS::461

Gradient intra prediction for coding of computer animated video - Li, Xiang; Oertel, Norbert; Kaup, André (2006 IEEE 8th Workshop on Multimedia Signal Processing, MMSP 2006, 2006)
The increasing interest in computer animations initiated the need of efficient coding for such applications. This paper proposes two gradient based approaches to improve the efficiency of Intra Prediction. Taking advantage of a special property of computer animations, namely the gradient distribution of intensity on object surfaces, the devised methods provided a better performance over H.264/AVC. According to the simulations, up to 0.3 dB gain in PSNR has been achieved. © 2006 IEE
Keywords: Computer animation; H.264/AVC; Intra coding

Graph-Based Compensated Wavelet Lifting for 3-D+t Medical CT Data - Lanz, Daniela; Kaup, André (Picture Coding Symposium, 2016)
An efficient scalable data representation is an important task especially in the medical area, e.g. for volumes from Computed Tomography (CT) or Magnetic Resonance Tomography (MRT), when a downscaled version of the original signal is needed. Image and video coders based on wavelet transforms provide an adequate way to naturally achieve scalability. This paper presents a new approach for improving the visual quality of the lowpass band by using a novel graph-based method for motion compensation, which is an important step considering data compression. We compare different kinds of neighborhoods for graph construction and demonstrate that a higher amount of referenced nodes increases the quality of the lowpass band while the mean energy of the highpass band decreases. We show that for cardiac CT data the proposed method outperforms a traditional mesh-based approach of motion compensation by approximately 11 dB in terms of PSNR of the lowpass band. Also the mean energy of the highpass band decreases by around 30

Graph-Based Compensated Wavelet Lifting for Scalable Lossless Coding of Dynamic Medical Data - Lanz, Daniela; Kaup, André (IEEE Transactions on Image Processing, 2020)

GSAEC - Acoustic echo cancellation embedded into the generalized sidelobe canceller - Herbordt, Wolfgang; Kellermann, Walter (Proc. European Signal Processing Conference (EUSIPCO), 2000)
In this paper, we examine the integration of acoustic echo cancellation (AEC) with a robust generalized side-lobe canceller (GSC) [2, 3] with regard to exploitable synergies. Since the AEC is located behind the fixed beamformer (FBF) of the GSC only one AEC is required for an arbitrary number of array elements. System analysis shows that ideal suppression of local interferers and of acoustic echoes cannot be achieved simultaneously. However, experiments confirm that the interference reduction (IR) is degraded by less than 3 dB compared to a cascade of AEC and GSC and that the echo reduction is improved by more than 13 dB relative to the GSC in echoic environments.

H.263 to H.264 transconding using data mining - Fernández-Escribano, G.; Bialkowski, Jens; Kalva, H.; Cuenca, P.; Orozco-Barbosa, L.; Kaup, André (14th IEEE International Conference on Image Processing, ICIP 2007, 2007)
In this paper, we propose the use of data mining algorithms to create a macroblock partition mode decision algorithm for inter-frame prediction, to be used as part of a high-efficient H.263 to H.264 transcoder. We use machine learning tools to exploit the correlation and derive decision trees to classify the incoming H.263 MC residual into one of the several coding modes in H.264. The proposed approach reduces the H.264 MB mode computation process into a decision tree lookup with very low complexity. Experimental results show that the proposed approach reduces the inter-prediction complexity by as much as 60% while maintaining the coding efficiency. ©2007 IEE
Keywords: Data mining; H.263; H.264

H.264/AVC-compatible coding of dynamic light fields using transposed picture ordering - Fecker, Ulrich; Kaup, André (13th European Signal Processing Conference, EUSIPCO 2005, 2005)
A dynamic light field or a multi-view video sequence requires capturing an object or a scene with multiple cameras. This allows viewing the scene from arbitrary viewpoints without the need for geometric information. As this leads to very high data rates, efficient compression is necessary. In this paper, we present an approach to exploiting the spatial correlation between the different camera views in addition to the temporal correlation within each view. A simple resorting scheme is introduced, which allows the usage of an off-the-shelf video coder to compress multi-view video data. Coding results are shown using the H.264/AVC video coding standard. It is discussed how the coding efficiency depends on the frame rate and the camera distance of the sequenc

Halfband Filters and Hilbert Transformers - Schüßler, Hans Wilhelm; Steffen, Peter (Circuits Systems and Signal Processing, 1998)
LMS::332

Hands-free speech recognition using a reverberation model in the feature domain - Sehr, Armin; Zeller, Marcus; Kellermann, Walter (14th European Signal Processing Conference, EUSIPCO 2006, 2006)
A novel approach for robust hands-free speech recognition in highly reverberant environments is proposed. Unlike conventional HMMbased concepts, it implicitly accounts for the statistical dependence of successive feature vectors due to the reverberation. This property is attained by a combined acoustic model consisting of a conventional HMM, modeling the clean speech, and a reverberation model. Since the HMM is independent of the acoustic environment, it needs to be trained only once using the usual Baum-Welch reestimation procedure. The training of the reverberation model is based on a set of room impulse responses for the corresponding acoustic environment and involves only a negligible computational effort. Thus, the recognizer can be adapted to new environments with moderate effort. In a simulation of an isolated digit recognition task in a highly reverberant room, the proposed method achieves a 60% reduction of the word error rate compared to a conventional HMM trained on reverberant speech, at the cost of an increased decoding complexit

Head Orientation Estimation from Multiple Microphone Arrays - Felsheim, Rebecca; Brendel, Andreas; Naylor, Patrick A.; Kellermann, Walter (European Signal Processing Conference (EUSIPCO), 2020)

Hierarchical clustering applied to overcomplete BSS for convolutive mixtures - Winter, Stefan; Sawada, H.; Araki, S.; Makino, Shoji (ISCA Tutorial and Research Workshop on Statistical and Perceptual Audio Processing, 2004)
LMS::527

Hierarchical Coding of Light Fields with Disparity Maps - Magnor, Marcus; Girod, Bernd (IEEE Int. Conf. on Image Processing, 1999)

High-dimensional regression with gaussian mixtures and partially-latent response variables - Deleforge, Antoine; Forbes, Florence; Horaud, Radu (Statistics and Computing, 2014)
In this work we address the problem of approximating high-dimensional data with a low-dimensional representation. We make the following contributions. We propose an inverse regression method which exchanges the roles of input and response, such that the low-dimensional variable becomes the regressor, and which is tractable. We introduce a mixture of locally-linear probabilistic mapping model that starts with estimating the parameters of inverse regression, and follows with inferring closed-form solutions for the forward parameters of the high-dimensional regression problem of interest. Moreover, we introduce a partially-latent paradigm, such that the vector-valued response variable is composed of both observed and latent entries, thus being able to deal with data contaminated by experimental artifacts that cannot be explained with noise models. The proposed probabilistic formulation could be viewed as a latent-variable augmentation of regression. We devise expectation-maximization (EM) procedures based on a data augmentation strategy which facilitates the maximum-likelihood search over the model parameters. We propose two augmentation schemes and we describe in detail the associated EM inference procedures that may well be viewed as generalizations of a number of EM regression, dimension reduction, and factor analysis algorithms. The proposed framework is validated with both synthetic and real data. We provide experimental evidence that our method outperforms several existing regression technique
LMS::1522

High dynamic range video by spatially non-regular optical filtering - Schöberl, Michael; Belz, Jürgen; Seiler, Jürgen; Foessel, S.; Kaup, André (2012 19th IEEE International Conference on Image Processing, ICIP 2012, 2012)
We present a new method for capturing high dynamic range video (HDRV). Our method is based on spatially varying exposures, where individual pixels are covered with filters for different optical attenuation. For preventing the loss in resolution we use a new non-regular arrangement of the attenuation pattern. Subsequent image reconstruction based on the sparsity assumption allows the reconstruction of natural images with high detail. © 2012 IEE
Keywords: Digital Camera; High Dynamic Range Image Sensor; Resolution Enhancement; Sparsity

High Dynamic Range Video Cameras Based on Single Shot Non-Regular Sampling - Keinert, J.; Schöberl, Michael; Ziegler, M.; Zilly, F.; Foessel, S. (International Broadcast Convention Conference (IBC), 2013)
Compared with traditional video acquisition, high dynamic range video increases the visual experience, enhances opportunities in post-production and simplifies studio lighting. However, existing techniques for high dynamic range video capture are too expensive for low and medium budget cameras. Furthermore, they are not suitable for handling the most challenging lighting conditions. Against this background, this paper presents a novel technique for capturing high dynamic range video. It is based on a non-regular mask of attenuation filters placed in front of a standard sensor. Compared to a traditional sensor with the same resolution, the captured image shows nearly the same amount of detail, but has a much higher dynamic range. The achievable dynamic range increment depends on the underlying sensor capabilities. For highest quality requirements, a 12 f-stop sensor can be extended by 2 to 5 f-stops and a 14 f-stop sensor, by 4, to 7 f-stops. The obviation of bracketing avoids motion artefacts and reduces the data volume read from the senso
LMS::1382

High-Dynamic Range Video Cameras Based on Single Shot Non-Regular Sampling - Keinert, J.; Schöberl, Michael; Ziegler, M.; Zilly, F.; Foessel, S. (The Best of IET and IBC, 2013)
Compared to traditional video acquisition, high-dynamic range video increases the visual experience, enhances the possibilities in post-production and simplifies stage lighting. However, existing techniques for high-dynamic range video capture are too expensive for low and medium budget cameras. Furthermore, they are not suitable to handle the most challenging lighting conditions. Against this background, the paper presents a novel technique for capturing high-dynamic range video. It is based on a non-regular mask of attenuation filters placed in front of a standard sensor. Compared to a traditional sensor with the same resolution, the captured image shows nearly the same amount of detail, but it has a much higher dynamic range. The achievable dynamic range increment depends on the underlying sensor capabilities. For highest quality requirements, a 12 f-stop sensor can be extended by 2 to 5 f-stops, a 14 f-stop sensor can be extended by 4 to 7 f-stops. The renouncement of bracketing avoids motion artifacts and reduces the data volume read from the senso
LMS::1381

High-Dynamic Range Video Cameras Based on Single Shot Non-Regular Sampling - Keinert, J.; Schöberl, Michael; Ziegler, M.; Zilly, F.; Foessel, S. (SMPTE Motion Imaging Journal, 2014)
LMS::1501

High Dynamic Range Video Reconstruction from a Stereo Camera Setup - Bätz, Michel; Richter, Thomas; Garbas, Jens-Uwe; Papst, Anton; Seiler, Jürgen; Kaup, André (Signal Processing-Image Communication, 2014)
To overcome the dynamic range limitations in images taken with regular consumer cameras, several methods exist for creating high dynamic range (HDR) content. Current low-budget solutions apply a temporal exposure bracketing which is not applicable for dynamic scenes or HDR video. In this article, a framework is presented that utilizes two cameras to realize a spatial exposure bracketing, for which the different exposures are distributed among the cameras. Such a setup allows for HDR images of dynamic scenes and HDR video due to its frame by frame operating principle, but faces challenges in the stereo matching and HDR generation steps. Therefore, the modules in this framework are selected to alleviate these challenges and to properly handle under- and oversaturated regions. In comparison to existing work, the camera response calculation is shifted to an offline process and a masking with a saturation map before the actual HDR generation is proposed. The first aspect enables the use of more complex camera setups with different sensors and provides robust camera responses. The second one makes sure that only necessary pixel values are used from the additional camera view, and thus, reduces errors in the final HDR image. The resulting HDR images are compared with the quality metric HDR-VDP-2 and numerical results are given for the first time. For the Middlebury test images, an average gain of 52 points on a 0-100 mean opinion score is achieved in comparison to temporal exposure bracketing with camera motion. Finally, HDR video results are provide
UnivIS-Import:2015-03-09:Pub.2013.tech.IE.LEN.highdy_3
Keywords: high dynamic range video; stereo camera setup; stereo matching

Higher-order listening room compensation with additive compensation signals - Hofmann, Christian; Günther, Michael; Bürger, Michael; Kellermann, Walter (IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2016)
The performance of sound reproduction systems for spatial audio is impaired by time-variant, reverberant listening environments. To tackle this issue, the Loudspeaker-Enclosure-Microphone System (LEMS) between the loudspeakers and reference microphones in the listening environment can be identified adaptively to allow an LEMS-specific pre-processing of the loudspeaker signals. This contribution introduces a broadband implementation of a narrowband Listening Room Compensation (LRC) method with additive compensation signals, recently proposed by Talagala et al. [1], it extends the concept to higher-order compensation, and compares LRC to Listening Room Equalization (LRE) analytically. Evaluations in an image-source environment confirm the efficacy of higher-order LRC and its suitability as a complexity-reduced alternative to LRE
Keywords: listening room; compensation; equalization; dereverberation; approximate inversion

Highly Efficient Simulation and Visualization of Acoustic Wave Fields with the Functional Transformation Method - Petrausch, Stefan; Rabenstein, Rudolf (Simulation and Visualization, 2005)
The Functional Transformation Method is an established method for digital sound synthesis via physical modeling. It starts from a physical description of the sounding object in form of a partial differential equation and solves it by suitable integral transformation in the frequency domain. After discretization, the resulting algorithms are fast enough to be implemented in full audio sample-rate. In this paper new results are presented, that extend the evaluation of the model into an efficient visualization of all physical processes with constant coefficients over regions with separable boundary conditions. In particular the simulation of wave fields is shown. A simulation program is presented, that creates animations of two-dimensional wave propagation in real-time. Furthermore, an application to the analysis of a new multi-channel sound reproduction method is shown.
LMS::549

High-Order Motion Compensation for Low Bit-Rate Video - Girod, Bernd; Wiegand, Thomas; Steinbach, Eckehard; Flierl, Markus; Zhang, Xiaozheng (European Signal Processing Conference, 1998)
LMS::347

High-Quality Acoustic Rendering with Wave Field Synthesis - Spors, Sascha; Teutsch, Heinz; Rabenstein, Rudolf (Vision, Modeling, and Visualization, 2002)

High-resolution hyperspectral video imaging using a hexagonal camera array - Sippel, Frank; Seiler, Jürgen; Kaup, André (Journal of the Optical Society of America A: Optics and Image Science, and Vision, 2024)
Retrieving the reflectance spectrum from objects is an essential task for many classification and detection problems, since many materials and processes have a unique spectral behavior. In many cases, it is highly desirable to capture hyperspectral images due to the high spectral flexibility. Often, it is even necessary to capture hyperspectral videos or at least to be able to record a hyperspectral image at once, also called snapshot hyperspectral imaging, to avoid spectral smearing. For this task, a high-resolution snapshot hyperspectral camera array using a hexagonal shape is introduced. The hexagonal array for hyperspectral imaging uses off-the-shelf hardware, which enables high flexibility regarding employed cameras, lenses, and filters. Hence, the spectral range can be easily varied by mounting a different set of filters. Moreover, the concept of using off-the-shelf hardware enables low prices in comparison to other approaches with highly specialized hardware. Since classical industrial cameras are used in this hyperspectral camera array, the spatial and temporal resolution is very high, while recording 37 hyperspectral channels in the range from 400 to 760 nm in 10 nm steps. As the cameras are at different spatial positions, a registration process is required for near-field imaging, which maps the peripheral camera views to the center view. It is shown that this combination using a hyperspectral camera array and the corresponding image registration pipeline is superior in comparison to other popular snapshot approaches. For this evaluation, a synthetic hyperspectral database is rendered. On the synthetic data, the novel approach, to our knowledge, outperforms its best competitor by more than 3 dB in reconstruction quality. This synthetic data is also used to show the superiority of the hexagonal shape in comparison to an orthogonal-spaced one. Moreover, a real-world high-resolution hyperspectral video database with 10 scenes is provided for further research in other application

Histogram-based prefiltering for luminance and chrominance compensation of multiview video - Fecker, Ulrich; Barkowsky, Marcus; Kaup, André (IEEE Transactions on Circuits and Systems For Video Technology, 2008)
Significant advances have recently been made in the coding of video data recorded with multiple cameras. However, luminance and chrominance variations between the camera views may deteriorate the performance of multiview codecs and image-based rendering algorithms. A histogram matching algorithm can be applied to efficiently compensate for these differences in a prefiltering step. A mapping function is derived which adapts the cumulative histogram of a distorted sequence to the cumulative histogram of a reference sequence. If all camera views of a multiview sequence are adapted to a common reference using histogram matching, the spatial prediction across camera views is improved. The basic algorithm is extended in three ways: a time-constant calculation of the mapping function, RGB color conversion, and the use of global disparity compensation. The best coding results are achieved when time-constant histogram calculation and RGB color conversion are combined. In this case, the usage of histogram matching prior to multiview encoding leads to substantial gains in the coding efficiency of up to 0.7 dB for the luminance component and up to 1.9 dB for the chrominance components. This prefiltering step can be combined with block-based illumination compensation techniques that modify the coder and decoder themselves, especially with the approach implemented in the multiview reference software of the Joint Video Team (JVT). Additional coding gains up to 0.4 dB can be observed when both methods are combined. © 2008 IEE
Keywords: Image-based rendering; Multiview video; Video coding; Video signal processing

Holography and the Wigner function - Lohmann, Adolf; Testorf, Markus; Ojeda-Castaneda, J. (Society of Photo-Optical Instrumentation Engineers (Proceedings), 2002)
LMS::649

How blind are we in Blind Signal Processing? - Kellermann, Walter (4th Joint Meeting of Acoustical Society of America and Acoustical Society of Japan, 2006)
Blind signal processing algorithms aim at blindly solving classical signal processing problems such as system identification, signal separation, or source localization. Here it is investigated to which extent some of the more popular blind signal processing concepts are really blind when they are used for signal acquisition in the acoustic domain. Blindness regarding the source configuration and the source signal properties, blindness with respect to channel properties, and, finally, blindness regarding the microphone configuration are investigated. It turns out that many algorithms are explicitly nonblind with respect to important properties of the source configuration, such as the number of sources and their pointlike nature. The source signals typically have to fulfill certain statistical properties or have to meet sparseness constraints. Some algorithms rely on a dominant direct acoustic path between sources and sensors, and linearity of the transmission channel model is implied with all convolutive mixture models. Finally, some popular algorithms are actually aiming at direction of arrival estimation, which always requires knowledge on the microphone array geometry. In essence, blind algorithms are wellinformed in many respects and are blind only with regard to a few, although decisive, properties.© A
LMS::676

HRTF-based robust least-squares frequency-invariant beamforming - Barfuß, Hendrik; Hümmer, Christian; Lamani, Gleni; Schwarz, Andreas; Kellermann, Walter (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2015)
In this work, an HRTF-based RLSFI beamformer design is proposed. The HRTF-based RLSFI beamformer accounts for the influence of a robot's head on the sound field. The performance of the new HRTF-based RLSFI beamformer is evaluated using signal-based measures and word error rates for an off-the-shelf speech recognizer, and is compared to the performance of the original free-field RLSFI beamformer design. The experimental results confirm the efficacy of the proposed HRTF-based beamformer design for robot auditio
Keywords: Spatial filtering; robust superdirective beamforming; white noise gain; signal enhancement; robot audition

HRTF-based robust least-squares frequency-invariant polynomial beamforming - Barfuß, Hendrik; Müglich, Marcel; Kellermann, Walter (International Workshop on Acoustic Echo and Noise Control (IWAENC), 2016)
In this work, we propose a robust Head-Related Transfer Function (HRTF)-based polynomial beamformer design which accounts for the influence of a humanoid robot's head on the sound field. In addition, it allows for a flexible steering of our previously proposed robust HRTF-based beamformer design. We evaluate the HRTF-based polynomial beamformer design and compare it to the original HRTF-based beamformer design by means of signal-independent measures as well as word error rates of an off-the-shelf speech recognition system. Our results confirm the effectiveness of the polynomial beam-former design, which makes it a promising approach to robust beam-forming for robot auditio
Keywords: Spatial filtering; robust superdirective beamforming; polynomial beamforming; white noise gain; signal enhancement; robot audition; head-related transfer functions

HRTF-based two-dimensional robust least-squares frequency-invariant beamformer design for robot audition - Barfuß, Hendrik; Bürger, Michael; Podschus, Jasper; Kellermann, Walter (Joint Workshop on Hands-free Speech Communication and Microphone Arrays (HSCMA), 2017)
In this work, we propose a two-dimensional Head-Related Transfer Function (HRTF)-based robust beamformer design for robot audition, which allows for explicit control of the beamformer response for the entire three-dimensional sound field surrounding a humanoid robot. We evaluate the proposed method by means of both signal-independent and signal-dependent measures in a robot audition scenario. Our results confirm the effectiveness of the proposed two-dimensional HRTF-based beamformer design, compared to our previously published one-dimensional HRTF-based beamformer design, which was carried out for a fixed elevation angle onl

Hybrid particle filtering based on an elitist resampling scheme - Halimeh, Mhd Modar; Hümmer, Christian; Brendel, Andreas; Kellermann, Walter (Sensor Array and Multichannel Signal Processing Workshop (SAM), 2018)
In this paper, the elitist resampling particle filter (ERPF) is introduced. The ERPF is an approach to combine generic particle filters based on an evolutionary selection of particles, which introduces a long-term memory to the selected group of the particles. Thereby, the ERPF aims at fusing the advantages, e.g., robustness and computational efficiency, and mitigating the drawbacks of each filter, e.g., degeneracy and samples impoverishment. Two variants of the ERPF are presented in this paper: the discrete ERPF (DERPF) and the continuous ERPF (CERPF), which to the authors' knowledge represent new forms of particle filters. The two proposed hybrids are compared to the corresponding original filters using two well-established benchmark models to illustrate that the evolutionary combinations provide a better estimation accuracy and exhibit an increased robustness against outlier
Keywords: Particle filter; nonlinear system identification; tracking; Bayesian filtering

Hybrid person detection and tracking in H.264/AVC video streams - Wojaczek, Philipp; Laumer, Marcus; Amon, P.; Hutter, Andreas; Kaup, André (10th International Conference on Computer Vision Theory and Applications, VISAPP 2015, 2015)
In this paper we present a new hybrid framework for detecting and tracking persons in surveillance video streams compressed according to the H.264/AVC video coding standard. The framework consists of three stages and operates in both the compressed and the pixel domain of the video. The combination of compressed and pixel domain represents the hybrid character. Its main objective is to significantly reduce the amount of computation required, in particular for frames and image regions with few people present. In its first stage the proposed framework evaluates the header information for each compressed frame in the video sequence, namely the macroblock type information. This results in a coarse binary mask segmenting the frame into foreground and background. Only the foreground regions are processed further in the second stage that searches for persons in the image pixel domain by applying a person detector based on the Implicit Shape Model. The third stage segments each detected person further with a newly developed method that fuses information from the first two stages. This helps obtaining a finer segmentation for calculating a color histogram suitable for tracking the person using the mean shift algorithm. The proposed framework was experimentally evaluated on a publicly available test set. The results demonstrate that the proposed framework reliably separates frames with and without persons such that the computational load is significantly reduced while the detection performance is kep
Keywords: Color histogram; Compressed domain; Compression; H.264/avc; Hsv; Hue; Macroblocks; Object detection; Person detection; Pixel domain; Segmentation; Tracking

Hybrid Super-Resolution Combining Example-Based Single-Image and Interpolation-Based Multi-Image Reconstruction Approaches - Bätz, Michel; Eichenseer, Andrea; Seiler, Jürgen; Jonscher, Markus; Kaup, André (2015 IEEE International Conference on Image Processing (ICIP), 2015)
Achieving a higher spatial resolution is of particular interest in many applications such as video surveillance and can be realized by employing higher resolution sensors or applying super-resolution methods. Traditional super-resolution algorithms are based on either a single low resolution image or on multiple low resolution frames. In this paper, a hybrid super-resolution method is proposed which combines both a single-image and a multi-image approach using a soft decision mask. The mask is computed from the motion information utilized in the multi-image super-resolution part. This concept is shown to work for one particular setup but is also extensible toward other combinations of single-image and multi-image super-resolution algorithms as well as other merging metrics. Simulation results show an average luminance PSNR gain of up to 0.85 dB and 0.59 dB for upscaling factors of 2 and 4, respectively. Visual results substantiate the objective result
Keywords: Resolution Enhancement; Image Processing; Single-Image Super-Resolution; Multi-Image Super-Resolution

Hyper-spectral image analysis with partially-latent regression - Deleforge, Antoine; Forbes, Florence; Horaud, Radu (22nd European Signal Processing Conference (EUSIPCO), 2014)
The analysis of hyper-spectral images is often needed to recover physical properties of planets. To address this inverse problem, the use of learning methods have been considered with the advantage that, once a relationship between physical parameters and spectra has been established through training, the learnt relationship can be used to estimate parameters from new images underpinned by the same physical model. Within this framework, we propose a partially-latent regression method which maps high-dimensional inputs (spectral images) onto low-dimensional responses (physical parameters). We introduce a novel regression method that combines a Gaussian mixture of locally-linear mappings with a partially-latent variable model. While the former makes high-dimensional regression tractable, the latter enables to deal with physical parameters that cannot be observed or, more generally, with data contaminated by experimental artifacts that cannot be explained with noise models. The method is illustrated on images collected from the Mars plane
LMS::1517
Keywords: hyper-spectral images; regression; dimension reduction; mixture models; latent variable model

Hyperspectral Image Reconstruction from Multispectral Images Using Non-Local Filtering - Sippel, Frank; Seiler, Jürgen; Kaup, André (23rd International Workshop on Multimedia Signal Processing, 2021)

Identification of active sources in single-channel convolutive mixtures using known source models - Sundar, Harshavardhan; Sreenivas, Thippur V.; Kellermann, Walter (IEEE Signal Processing Letters, 2012)
We address the problem of identifying the constituent sources in a single-sensor mixture signal consisting of contributions from multiple simultaneously active sources. We propose a generic framework for mixture signal analysis based on a latent variable approach. The basic idea of the approach is to detect known sources represented as stochastic models, in a single-channel mixture signal without performing signal separation. A given mixture signal is modeled as a convex combination of known source models and the weights of the models are estimated using the mixture signal. We show experimentally that these weights indicate the presence/absence of the respective sources. The performance of the proposed approach is illustrated through mixture speech data in a reverberant enclosure. For the task of identifying the constituent speakers using data from a single microphone, the proposed approach is able to identify the dominant source with up to 8 simultaneously active background sources in a room with RT60= 250 ms, using models obtained from clean speech data for a Source to Interference Ratio (SIR) greater than 2 dB. © 1994-2012 IEE
Keywords: Latent variable approach; multiple source identification; single channel; Student's-t Mixture Models (tMMs)

Identification of Irregular Motion in Automotive Navigation Systems Using Novelty Detection - Pöllot, Martin; Springer, Dominic; Schleifer, Ralph; Niederkorn, Dieter; Kaup, André (IEEE Symposium Series on Computational Intelligence SSCI, 2016)
Automated display testing for visual unpleasant and erroneous navigation sequences is an important step to preserve a high quality standard for premium vehicle manufacturers. This paper presents a novel error detection algorithm for navigation sequences based on novelty detection on motion parameters obtained from real world navigation sequences. Motion parameters are accumulated through key point matching using BRISK and subsequent homography calculation. With these parameters one is able to describe the motion between two successive frames. Combinations of translational and rotational components allow the novelty detection algorithm to predict outliers. These outliers either show positioning errors or abnormal motion behaviour which are both unacceptable for high quality. Experimental results demonstrate that this algorithm works significantly better than state of the art, where one has to know errors before analysing the data set in order to determine thresholds for particular errors. The gains in precision and recall are 49.67% and 6.06% respectively, the accuracy is 1.37% higher compared to optimized threshold result
Keywords: Novelty Detection; Error Detection; Automated Display Testing; Navigation Systems; Machine Learning

Illustration of the Duality Between Channel Coding and Rate Distortion with Side Information - Su, Jonathan; Eggers, Joachim; Girod, Bernd (34th Asilomar Conference Signals, Systems, Computers, 2000)
LMS::203

Image and Video Coding - Girod, Bernd; Gray, R.; Kovacevic, J.; Vetterli, M. (IEEE Signal Processing Magazine, 1998)
LMS::329

Image-Based Synchronization in Mobile TV for a Multi-Broadcast-Receiver - Tröger, Tobias; Heiber, H.; Schmitt, A.; Kaup, André (9th Workshop Digital Broadcasting, 2008)
In this paper we propose a novel technique for imagebased synchronization of two mobile received TV sequences having different spatial resolutions, deviant image qualities, and possibly lost image blocks. The sequences are aligned spatially and temporally based on an affine transform. The optimal transformation parameters are determined numerically by the LevenbergMarquardt-Algorithm due to complexity reasons. The presented algorithm leads to accurate synchronization of mobile TV and is robust even in case of image cropping and arbitrarily shaped error pattern
LMS::760
Keywords: DVB-H; DVB-T; Mobile TV; Multi-Broadcast Reception; Synchronization; T-DMB

Image Restoration for Frame- and Object-Based Video Coding Using an Adaptive Constrained Least Squares Approach - Kaup, André (Signal Processing, 2000)
LMS::463

Image Segmentation for Improved Lossless Screen Content Compression - Uddehal, Shabhrish Reddy; Strutz, Tilo; Och, Hannah; Kaup, André (2023 IEEE International Conference on Acoustics, Speech and Signal Processing, 2023)
In recent years, it has been found that screen content images (SCI) can be effectively compressed based on appropriate probability modelling and suitable entropy coding methods such as arithmetic coding. The key objective is determining the best probability distribution for each pixel position. This strategy works particularly well for images with synthetic (textual) content. However, usually screen content images not only consist of synthetic but also pictorial (natural) regions. These images require diverse models of probability distributions to be optimally compressed. One way to achieve this goal is to separate synthetic and natural regions. This paper proposes a segmentation method that identifies natural regions enabling better adaptive treatment. It supplements a compression method known as Soft Context Formation (SCF) and operates as a pre-processing step. If at least one natural segment is found within the SCI, it is split into two subimages (natural and synthetic parts) and the process of modelling and coding is performed separately for both. For SCIs with natural regions, the proposed method achieves a bit-rate reduction of up to 11.6% and 1.52% with respect to HEVC and the previous version of the S
Keywords: screen content images; lossless coding; segmentation; probability distribution; soft context formation

Image Super-Resolution Using T-Tetromino Pixels - Grosche, Simon; Regensky, Andy; Seiler, Jürgen; Kaup, André (Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023)

Impact of coding noise on the convergence of blind source separation - Meier, Stefan; Kellermann, Walter (11. ITG Conference on Speech Communication, 2014)
In networks consisting of distributed microphones, signals from several spatially distant microphones are exploited for multichannel signal processing. This requires transmission of at least one microphone signal to another network node. In this paper, the effect of a predictive source coding scheme on the TRINICON-based Blind Source Separation (BSS) algorithm is evaluated. First, the propagation of the quantization noise into the separated output signals is analyzed and verified experimentally. Moreover, the effect of coding on the convergence of the algorithm itself is evaluated. For the experimental part of the analysis, the application of BSS in a binaural hearing aid scenario is considered. BSS proves to be very robust against small coding word lengths and exhibits a minor sensitivity on noise shapin

Implementation of arbitrary linear sound synthesis algorithms by digital wave guide structures - Petrausch, Stefan; Rabenstein, Rudolf (8th International Conference on Digital Audio Effects, DAFx 2005, 2005)
The Digital Wave Guide (DWG) method is one of the most popular techniques for digital sound synthesis via physical modeling. Due to the inherent solution of the wave equation by the structure of the DWG method, it provides a highly efficient algorithm for typ ical physical modeling problems. In this paper it will be shown, that it is possible to use this efficient structure for any existing linear sound synthesis algorithm. By a consequent description of discrete implementations with State Space Structures (SSSs), suit able linear state space transformations can be used to achieve the typical DWG structure from any given system. The proposed ap proach is demonstrated with two case studies, where a modal solu tion achieved with the Functional Transformation Method (FTM) is transformed to a DWG implementation. In the first example the solution of the lossless wave equation is transformed to a DWG structure, yielding an arbitrary size fractional delay filter. In an other example a more elaborated model with dispersion and damp ing terms is transformed, resulting in a DWG model with parame ter morphing feature

Implementation of Digital 3-D IIR Filters for Stream-Processing Applications - Runze, Gerhard (IEE Colloquium on Multidimensional Systems: Problems and Solutions, 1998)

Implementing Ambisonics on a 48 Channel Circular Loudspeaker Array - Kratschmer, M.; Rabenstein, Rudolf (Ambisonics Symposium 2009, 2009)
This contribution describes an implementation of near fieldcoding higher order Ambisonics on an existing circular array of 48 loudspeakers. Static and moving source s can be reproduced with a variable Ambisonics order and variable number of loudspeakers. Informal listening testswere performed to evaluate the influence of Ambisonics order and loudspeaker number on the perception quality and locali zation accurac

Implicit discretization of linear partial differential equations and repetitive processes - Rabenstein, Rudolf; Steffen, Peter (2009 International Workshop on Multidimensional (nD) Systems, nDS 2009, 2009)
Implicit schemes are a popular approach to the discretization of linear partial differential equations by finite differences. They require to solve a linear set of equations in each time step. Since finite difference discretizations lead to a local coupling, these systems of equations are sparse and can be effectively solved by iterative procedures. Numerical procedures of this type are known in control theory as repetitive processes. They have mostly been used to describe control algorithms for processes where passes of finite length are repeated over and over. This contribution shows how the implicit discretization of partial differential equations can be cast into the framework of repetitive processes. Thus it establishes a link between yet unrelated results in numerical mathematics and control theory. © 2009 IEE

Improved acoustic echo cancellation for low SNR based on blockwise combination of filters - Azpicueta-Ruiz, Luis A.; Zeller, Marcus; Figueiras-Vidal, Aníbal R.; Arenas-García, Jerónimo; Kellermann, Walter (20th International Congress on Acoustics 2010, ICA 2010 - Incorporating the 2010 Annual Conference of the Australian Acoustical Society, 2010)
Acoustic echo cancellers (AEC) are becoming increasingly important because of the widespread use of hands-free devices. Due to their simplicity, most of the cancellers rely on NLMS-type adaptive filters to model and track the time-varying echo path. Recently, adaptive combinations of filters are gaining increasing popularity as a flexible and versatile approach to overcome compromises inherent to adaptive filters, thus enhancing the overall performance. Regarding AEC scenarios, such filter combinations have already been proposed for, e.g., improving the trade-off between convergence speed and steady-state error or for reducing the dependency on varying ratios of linear and nonlinear distortions. In this paper, we present a new AEC approach, showing improved performance for unknown or time-varying signalto- noise ratios (SNR). The proposed scheme exploits the fact that the coefficient energy of a typical echo path is not uniformly distributed, but decays exponentially. Under this condition, an NLMS filter will introduce significant estimation errors for less significant filter taps due to gradient noise. Since the number of affected coefficients strongly depends on the SNR and hence on the implied noise floor, the cancellation performance may degrade considerably for low SNRs. In order to relieve the coefficient noise, the adaptive impulse response is split into a number of nonoverlapping blocks, each of which is combined with a virtual 'zero-block', having fixed zero coefficients, with timevarying relative weights for both the nonzero and the zero-block. In practice, this results in a possibly biased estimation of some of the filter coefficients. However, it has been shown that such estimates can yield advantages in terms of mean-square error, especially for low SNRs. The combination of each block is implemented by a convex mixing, where the control parameter is updated according to a stochastic gradient descent method so as to minimize the global error of the AEC. For moderate block numbers, the increment in computational cost over a conventional NLMS canceller is negligible. In particular, this contribution investigates the operation of the blockwise combined filter for low SNR conditions, comparing its performance with standard NLMS- and PNLMS-type filters. The robustness and benefits of the proposed approach are thereby experimentally verified for noise and speech inputs. Moreover, the influence of the number of blocks and the mixing parameters is also studied and indications on future work (as e.g. accounting for impulsive noise or an extension to nonlinear filters) are given. Copyright © (2010) by the International Congress on Acoustic

Improved Encoding of DCT Coefficients for Low Bit-Rate Video Coding Using Multiple VLC Tables - Hartung, Frank; Girod, Bernd (International Conference on Image Processing ICIP-99, Kobe, Japan, 1999)
LMS::312

Improved Kalman gain computation for multichannel frequency-domain adaptive filtering and application to acoustic echo cancellation - Buchner, Herbert; Kellermann, Walter (2002 IEEE International Conference on Acoustic, Speech and Signal Processing, 2002)
For multichannel adaptive filtering applications like acoustic echo cancellation, sophisticated, but efficient algorithms are necessary that take both auto- and cross-correlations of the excitation signals into account. Based on a recently published frequency-domain framework, we examine in this contribution the computation of the so-called frequency-domain Kalman gain, which is crucial in the multichannel case to meet the above requirements for correlated input signals. We present an extended scheme with increased computational efficiency for a higher number of input channels. Robust convergence behaviour during perturbation of the desired signal is ensured by a new dynamical regularization. Application of the new algorithm to multichannel acoustic echo cancellation with real-world signals achieves improved performance.

Improved mode selection in hybrid error concealment for multi-broadcast-reception - Tröger, Tobias; Heiber, Henning; Schmitt, Andreas; Kaup, André (Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 2010)
Digital TV signals are often degraded by lost macro blocks or slices if received with mobile devices. Recently, we introduced a technique for inter-sequence error concealment in multi-broadcast scenarios which reconstruct lost image blocks of video signals by inserting error-free blocks of corresponding reference signals. The algorithm outperforms state-of-the-art techniques by up to 15 dB PSNR. The quality of reconstructed image blocks can he further enhanced by combining intersequence error concealment with a temporal concealment approach as we showed in 2009. To enhance the robustness of this hybrid error concealment, we propose a novel metric for mode selection in this paper which takes into account local image statistics. While keeping the complexity of hybrid error concealment nearly constant, a mean gain of up to 2.1 dB and a peak gain of up to 10.1 dB in terms of PSNR can be achieved additionally. ©2010 IEE

Improved Motion Plane Adaptive 360-Degree Video Compression Using Affine Motion Models - Ritthaler, Marina; Regensky, Andy; Kaup, André (ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025)

Improved Screen Content Coding in VVC Using Soft Context Formation - Och, Hannah; Uddehal, Shabhrish Reddy; Strutz, Tilo; Kaup, André (2024 IEEE International Conference on Acoustics, Speech and Signal Processing, 2024)
Keywords: image compression; screen content; VVC; segmentation; soft context formation

Improved wideband blind adaptive system identification using decorrelation filters for the localization of multiple speakers - Lombard, Anthony; Buchner, Herbert; Kellermann, Walter (2007 IEEE International Symposium on Circuits and Systems, ISCAS 2007, 2007)
This paper addresses the TDOA extraction problem for localizing multiple sources in noisy and reverberant environments with emphasis on speech excitation. TDOAs are estimated by performing blind adaptive MIMO system identification using a gradient-based BSS variant of the TRINICON framework. We present a novel method to improve the TDOA estimation for signals with lowpass-like spectral characteristics such as speech, for which the standard approach achieves only imperfect system identification. To this end, we propose to combine the BSS algorithm with decorrelation filters, thereby achieving a greatly improved wideband identification of the acoustical system. The approach is verified in a number of scenarios, where it provides more accurate TDOA estimates for the speaker localization at a negligible additional computational cost. © 2007 IEE

Improving block-based compensated wavelet lifting by reconstructing unconnected pixels - Schnurrer, Wolfgang; Seiler, Jürgen; Kaup, André (2013 International Symposium on Signals, Circuits and Systems, ISSCS 2013, 2013)
This paper presents a new approach for improving the visual quality of the lowpass band of a compensated wavelet transform. A high quality of the lowpass band is very important as it can then be used as a downscaled version of the original signal. To adapt the transform to the signal, compensation methods can be implemented directly into the transform. We propose an improved inversion of the block-based motion compensation by processing unconnected pixels by a reconstruction method. We obtain a better subjective visual quality while furthermore saving up to 2.6% of bits for lossless coding. © 2013 IEE

Improving HEVC Encoding of Rendered Video Data Using True Motion Information - Herglotz, Christian; Müller, David; Weinlich, Andreas; Bauer, Frank; Ortner, Michael; Stamminger, Marc; Kaup, André (20th IEEE International Symposium on Multimedia (ISM), 2018)
This paper shows that motion vectors representing the true motion of an object in a scene can be exploited to improve the encoding process of computer generated video sequences. Therefore, a set of sequences is presented for which the true motion vectors of the corresponding objects were generated on a per-pixel basis during the rendering process. In addition to conventional motion estimation methods, it is proposed to exploit the computer generated motion vectors to enhance the rate-distortion performance. To this end, a motion vector mapping method including disocclusion handling is presented. It is shown that mean rate savings of 3.78% can be achieved.<br
Keywords: HEVC; motion vectors; blender; renderer; encoding

Improving Mesh-Based Motion Compensation by Using Edge Adaptive Graph-Based Compensated Wavelet-Lifting for Medical Data Sets - Lanz, Daniela; Kaup, André (IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2017)
Medical applications like Computed Tomography (CT) or Magnetic Resonance Tomography (MRT) often require an efficient scalable representation of their huge output volumes in the further processing chain of medical routine. A downscaled version of such a signal can be obtained by using image and video coders based on wavelet transforms. The visual quality of the resulting lowpass band, which shall be used as a representative, can be improved by applying motion compensation methods during the transform. This paper presents a new approach of using the distorted edge lengths of a mesh-based compensated grid instead of the approximated intensity values of the underlying frame to perform a motion compensation. We will show that an edge adaptive graph-based compensation and its usage for compensated wavelet lifting improves the visual quality of the lowpass band by approximately 2.5 dB compared to the traditional mesh-based compensation, while the additional filesize required for coding the motion information doesn't chang
LMS::1708
Keywords: discrete wavelet transforms; motion compensation; scalability; signal processing on graphs; computed tomography

Improving Spherical Image Resampling through Viewport-Adaptivity - Regensky, Andy; Heimann, Viktoria; Zhang, Ruoyu; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2023)
The conversion between different spherical image and video projection formats requires highly accurate resampling techniques in order to minimize the inevitable loss of information. Suitable resampling algorithms such as nearest neighbor, linear or cubic resampling are readily available. However, no generally applicable resampling technique exploits the special properties of spherical images so far. Thus, we propose a novel viewport-adaptive resampling (VAR) technique that takes the spherical characteristics of the underlying resampling problem into account. VAR can be applied to any mesh-to-mesh capable resampling algorithm and shows significant gains across all tested techniques. In combination with frequency-selective resampling, VAR outperforms conventional cubic resampling by more than 2 dB in terms of WS-PSNR. A visual inspection and the evaluation of further metrics such as PSNR and SSIM support the positive results.<br
Keywords: 360-degree; spherical; viewport-adaptive; frequency-selective; image resampling

Improving the Image Quality of Blockbased Video Coders by Exploiting Interblock Redundancy - Kutka, R.; Kaup, André; Hager, M. (First International Workshop on Wireless Image/Video Communications, Loughborough, 1996)
LMS::1026

Improving the prediction efficiency for multi-view video coding using histogram matching - Fecker, Ulrich; Barkowsky, Marcus; Kaup, André (25th PCS: Picture Coding Symposium 2006, PCS2006, 2006)
Applications for video data recorded with a setup of several cameras are currently attracting increasing interest. For such multi-view sequences, efficient coding is crucial to handle the enormous amount of data. However, significant luminance and chrominance variations between the different views, which often originate from imperfect camera calibration, are able to reduce the coding efficiency and the rendering quality. In this paper, we suggest the usage of histogram matching to compensate these differences in a pre-filtering step. After a description of the proposed algorithm, it is explained how histogram matching can be applied to multi-view video data. The effect of histogram matching on the coding performance is evaluated by statistically analysing prediction from temporal as well as from spatial references. For several test sequences, results are shown which indicate that the amount of spatial prediction across different camera views can be increased by applying histogram matchin
Keywords: Image filtering; Multi-view video; Video coding

Improving the Rate-Distortion Model of HEVC Intra by Integrating the Maximum Absolute Error - Jaskolka, Karina; Kaup, André (IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), 2019)

Increasing camera dynamic range through in-sensor multi-exposure white balancing - Schöberl, Michael; Schnurrer, Wolfgang; Foessel, S.; Kaup, André (2011 18th IEEE International Conference on Image Processing, ICIP 2011, 2011)
In typical image sensors the spectral sensitivity of color channels is fixed. The illumination spectrum in natural scenes can vary to a great extent. This leads to an unbalanced response in color channels and hence a reduction in dynamic range. We propose a new method to adjust the relative sensitivity of color channels based on multi-exposure frame combination. Instead of a single long exposure we capture a different number of gapless exposures in each color channel and combine them. This offers a digital option for reducing sensitivity for some color channels and aligns the color channels. The method preserves motion blur and can be used in any long exposure or moving picture photography. We can now get a higher dynamic range from a camera system under any unfavorable illumination conditions with very little effort. © 2011 IEE
Keywords: dynamic range; image sensor; white balance

Increasing imaging resolution by covering your sensor - Schöberl, Michael; Seiler, Jürgen; Foessel, S.; Kaup, André (2011 18th IEEE International Conference on Image Processing, ICIP 2011, 2011)
Up to now, an increase in camera resolution required image sensors with more and more pixels. However, acquisition systems are limited in their pixels per second throughput given as power and complexity constraints. Simply capturing more pixels in a given system is often not possible. We propose a new non-regular imaging architecture that samples only few pixels and reconstructs a high resolution image afterwards. Our sampling is optimized to provide non-regular spatial sampling from a sensor with regular readout circuits. An existing slow image acquisition system can then be used to capture the data. The image reconstruction is performed with a local sparsity-based approach. The result is a high resolution image that requires a much smaller effort during acquisition. © 2011 IEE
Keywords: Digital Camera; Image Sensor; Resolution Enhancement; Sparsity

Increasing Imaging Resolution by Non-Regular Sampling and Joint Sparse Deconvolution and Extrapolation - Seiler, Jürgen; Jonscher, Markus; Ussmueller, Thomas; Kaup, André (IEEE Transactions on Circuits and Systems For Video Technology, 2018)
Increasing the resolution of image sensors has been a never ending struggle since many years. In this paper, we propose a novel image sensor layout which allows for the acquisition of images at a higher resolution and improved quality. For this, the image sensor makes use of non-regular sampling which reduces the impact of aliasing. Therewith, it allows for capturing details which would not be possible with state-ofthe-art sensors of the same number of pixels. The non-regular sampling is achieved by rotating prototype pixel cells in a nonregular fashion. As not the whole area of the pixel cell is sensitive to light, a non-regular spatial integration of the incident light is obtained. Based on the sensor output data, a high-resolution image can be reconstructed by performing a deconvolution with respect to the integration area and an extrapolation of the information to the insensitive regions of the pixels. To solve this challenging task, we introduce a novel joint sparse deconvolution and extrapolation algorithm. The union of non-regular sampling and the proposed reconstruction allows for achieving a higher resolution and therewith an improved imaging qualit

Increasing the Accuracy of a Neural Network Using Frequency Selective Mesh-to-Grid Resampling - Spruck, Andreas; Heimann, Viktoria; Kaup, André (IEEE International Symposium on Circuits and Systems (ISCAS), 2022)

Influence of the presentation time on subjective votings of coded still images - Barkowsky, Marcus; Eskofier, B.; Bialkowski, Jens; Kaup, André (2006 IEEE International Conference on Image Processing, ICIP 2006, 2006)
The quality of coded images is often assessed by a subjective test. Usually the viewers get as much time as they need to find a stable result. In video sequences however, the viewer has to judge the quality in a shorter time that is defined by the changing content or a following scene cut. Therefore it is desirable to know the influence of a shorter presentation time on the perceptibility of distortions. In this paper we present the results of a suitable subjective test on coded still images. The images were presented for six different durations, ranging from 200 ms to 3 s. Special care was taken to avoid the memorization effect usually present after short presentations. The results show that the viewers tend to avoid extreme votings at short durations. The variance of the votings is also discussed in detail. Based on the result of the voting for the longest presentation time, we propose a prediction model for the voting of the shorter durations using a logistic curve fit. This Presentation Time Model (PTM) is presented and analysed in detail. ©2006 IEE
Keywords: Cognitive science; Multimedia communication; Subjective test; Temporal effects; Visual system

Informed ego-noise suppression using motor data-driven dictionaries - Schmidt, Alexander; Kellermann, Walter (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019)

Informed Source Extraction based on Independent Vector Analysis using Eigenvalue Decomposition - Brendel, Andreas; Kellermann, Walter (European Signal Processing Conference (EUSIPCO), 2020)

Informed spatial filtering based on constrained independent component analysis - Barfuß, Hendrik; Reindl, Klaus; Kellermann, Walter (Audio Source Separation, 2018)
In this work, we present a linearly constrained signal extraction algorithm which is based on a Minimum Mutual Information (MMI) criterion that allows to exploit the three fundamental properties of speech and audio signals: Nonstationarity, Nonwhiteness, and Nongaussianity. Hence, the proposed method is very well suited for signal processing of nonstationary nongaussian broadband signals like speech. Furthermore, from the linearly constrained MMI approach, we derive an efficient realization in a Generalized Sidelobe Canceler structure. To estimate the relative transfer functions between the microphones, which are needed for the set of linear constraints, we use an informed time-domain independent component analysis algorithm, which exploits some coarse direction-of-arrival information of the target source. As a decisive advantage, this simplifies the otherwise challenging control mechanism for simultaneous adaptation of the GSC's blocking matrix und interference and noise canceler coefficients. Finally, we establish relations between the proposed method and other well-known multichannel linear filter approaches for signal extraction based on second-order-statistics, and demonstrate the effectiveness of the proposed signal extraction method in a multispeaker scenari
Keywords: Independent Component Analysis; Blind Source Separation; Linearly Constrained Minimum Variance; Generalized Sidelobe Canceler; Informed Spatial Processing; Minimum Mutual Information; Relative Transfer Function; TRINICON

Infrastructure-based Perception with Cameras and Radars for Cooperative Driving Scenarios - Tsaregorodtsev, Alexander; Buchholz, Michael; Belagiannis, Vasileios (IEEE Intelligent Vehicles Symposium, Proceedings, 2024)
Roadside infrastructure has enjoyed widespread adoption for various tasks such as traffic surveillance, traffic monitoring, control of traffic flow, and prioritization of public transit and emergency vehicles. As automated driving functions and vehicle communications continue to be researched, cooperative and connected driving scenarios can now be realized. Cooperative driving, however, imposes stringent environmental perception and model requirements. In particular, road users, including pedestrians and cyclists, must be reliably detected and accurately localized. Furthermore, the perception framework must have low latency to provide up-to-date information. In this work, we present a refined, camera-based reference point detector design that does not rely on annotated infrastructure datasets and incorporates fusion with cost-effective radar sensor data to increase system reliability, if available. The reference point detector design is realized with box and instance segmentation object detector models to extract object ground points. In parallel, objects are extracted from radar target data through a clustering pipeline and fused with camera object detections. To demonstrate the real-world applicability of our approaches for cooperative driving scenarios, we provide an extensive evaluation of data from a real test site.
CRIS-Team Scopus Importer:2024-08-09

In-loop denoising of reference frames for lossless coding of noisy image sequences - Wige, Eugen; Amon, Peter; Hutter, Andreas; Kaup, André (2010 17th IEEE International Conference on Image Processing, ICIP 2010, 2010)
The major gain in video coding applications compared to single image coding is the use of temporal prediction, which exploits the correlation between adjacent frames. However, in high quality video coding, especially lossless video coding, the compression gain of P-frames over I-Frames becomes very small. The reason for that is that the reference frame for inter prediction is not good enough, and therefore the amount of inter-predicted blocks in a P-Frame becomes relatively small compared to the number of intra-predicted blocks. In order to generate a better predictor for inter-prediction, we propose to remove additive noise from the reference frame using an adaptive Wiener filter. This way, we could achieve a maximum compression gain of 4.6% and an average compression gain of 3.3% in contrast to the H.264/AVC standard for lossless coding of high quality image sequences without affecting the encoding time noticeably. © 2010 IEE
Keywords: Lossless video compression; Motion compensated prediction; Predictor denoising

In-loop noise-filtered prediction for high efficiency video coding - Wige, Eugen; Yammine, Gilbert; Amon, P.; Hutter, Andreas; Kaup, André (IEEE Transactions on Circuits and Systems For Video Technology, 2014)
In this paper, we focus on optimization and enhancement of High Efficiency Video Coding (HEVC) with respect to professional applications. In most of the professional video applications, noise dominates the compression performance. We therefore theoretically and practically analyze the denoising performance of an HEVC codec and show that, especially for low to medium quantization parameters (QPs), the source noise is still present in the reconstructed frame. This motivates us to use an explicit in-loop reference frame denoising filter for this quality range. We show how the noise, which badly influences the prediction, can be modeled and estimated. Two approaches are introduced, in which either the noise is calculated from prior knowledge or estimated automatically without prior knowledge. A denoising filter that exploits the HEVC core transform is introduced. Finally, a coding mode adaptive scheme for motion compensation using the noise-filtered reference frame is described. The compression results show that especially for low to medium QP settings as well as lossless compression considerable bitrate savings can be achieved using the proposed schemes. © 1991-2012 IEEE.

Input signal decorrelation applied to adaptive second-order Volterra filters in the time domain - Küch, Fabian; Zeller, Marcus; Kellermann, Walter (2006 IEEE 12th Digital Signal Processing Workshop and 4th IEEE Signal Processing Education Workshop, DSPWS, 2006)
The compensation of nonlinear signal distortions as caused by small, low-cost loudspeakers driven at high volume by second-order Volterra filters has been investigated in recent works. In the field of acoustic echo cancellation, such undesired signal components are removed by adaptive filtering. Since the performance of the usually applied LMS adaptation suffers from correlations of the input data, preceding decorrelation of the excitation signal is desirable in order to increase the speed of convergence. This contribution discusses an efficient configuration for incorporating a decorrelation filter into nonlinear adaptive filtering scenarios. Simulation results for both noise and speech indicate an increase in convergence speed by providing decorrelated signals as filter input. © 2006 IEE
Keywords: Adaptive Volterra filter; Decorrelation; Nonlinear AEC

Intelligent multichannel signal processing for future audio reproduction systems - Brandenburg, Karlheinz; Schneider, Martin; Franck, Andreas; Kellermann, Walter; Brix, Sandra (AES International Conference 2013: Sound Field Control, 2013)
It is possible to improve audio reproduction, active noise reduction systems, and sound field control by using multi-loudspeaker systems. Distributed arrays of transducers with intelligent audio signal processing need sophisticated signal processing algorithms. The paper looks both at some of the psychoacoustic fundamentals of auditory illusion (still partly unsolved) and proposals to build solutions based on the combination of wave field synthesis (WFS) and wave domain adaptive filtering (WDAF). We include the sketch of application scenarios for future systems with a large number of loudspeakers. Copyright © (2013) by the Audio Engineering Societ

Inter-Camera Color Correction for Multispectral Imaging with Camera Arrays Using a Consensus Image - Kossira, Katja; Seiler, Jürgen; Kaup, André (International Workshop on Multimedia Signal Processing (MMSP), 2024)
This paper introduces a novel method for inter-camera color calibration for multispectral imaging with camera arrays using a consensus image. Capturing images using multispectral camera arrays has gained importance in medical, agricultural, and environmental processes. Due to fabrication differences, noise, or device altering, varying pixel sensitivities occur, influencing classification processes. Therefore, color calibration between the cameras is necessary. In existing methods, one of the camera images is chosen and considered as a reference, ignoring the color information of all other recordings. Our new approach does not just take one image as reference, but uses statistical information such as the location parameter to generate a consensus image as basis for calibration. This way, we managed to improve the PSNR values for the linear regression color correction algorithm by 1.15 dB and the improved color difference (iCID) values by 2.8
Keywords: color calibration; multispectral imaging; camera arrays; consensus image

Interconnection of state space structures and wave digital filters - Petrausch, Stefan; Rabenstein, Rudolf (IEEE Transactions on Circuits and Systems II: Express Briefs, 2004)
State space structures (SSSs) and wave digital filters (WDFs) are two major paradigms for the realization of digital filters. Both approaches are well established, but there are no proven methods for a mixed design of digital filters consisting of parts which are realized as SSSs and parts realized as WDFs. This contribution shows how to add a wave port to the conventional SS representation. This wave port allows to interconnect SSSs and WDFs without creating delay-free loops. Such interconnections allow to build discrete-time structures by reusing existing designs from both classes of digital filters. © 2005 IEEE.
Keywords: State space structures (SSSs); Wave digital filters (WDFs)

Internet Video-Streaming - Girod, Bernd; Färber, Niko; Stuhlmüller, Klaus (Proc. of 10th Tyrrhenian Int. Workshop on Digital Communications, 1998)
LMS::354

Inter-scale prediction of motion information for a wavelet-based scalable video coder - Garbas, Jens-Uwe; Kaup, André (26th Picture Coding Symposium, PCS 2007, 2007)
In this paper we present an efficient inter-scale motion vector prediction scheme for a wavelet based scalable video codec which uses inter-scale wavelet prediction. The scalable video codec considered in our work belongs to the class of 2D+t+2D schemes which applies 2-D spatial wavelet transform on the input data first, motion compensated temporal transform second and further spatial decomposition at the last stage. Although inter-scale wavelet coefficient prediction is proven to be very efficient in this coding scheme, inter-scale motion prediction has not yet been analyzed in this context. We propose a prediction scheme that enables exploitation of inter-scale motion redundancy and significantly enhances coding efficiency. PSNR increase of up to 1.45 dB and side information rate savings of up to 13% can be denoted, particularly for fast moving sequence
Keywords: Motion prediction; Scalable video coding

Inter-Sequence Error Concealment for Diversity Reception of TV Sequences - Tröger, Tobias; Heiber, Henning; Schmitt, Andreas; Kaup, André (17th European Signal Processing Conference, EUSIPCO 2009, 2009)
In this paper a new technique for error concealment of digital TV signals in a diversity reception scenario is proposed. Inter-sequence error concealment utilizes one or more reference sequences to reconstruct lost image blocks or slices of a video sequence. Based on the general approach of image combination for uncompressed video sequences, inter-sequence error concealment of coded TV sequences is evaluated in detail. The presented technique especially considers error propagation by motion-compensated prediction in hybrid video coding. The algorithm for error concealment is introduced on example of DVB-T signals which are transmitted in multi-frequency networks. In practice, applications like multi-standard-reception of DVB-T, DVB-H and T-DMB seem also natural. © EURASIP, 200

Inter-sequence error concealment of high-resolution video sequences in a multi-broadcast-reception scenario - Tröger, Tobias; Garbas, Jens-Uwe; Heiber, Henning; Schmitt, Andreas; Kaup, André (16th European Signal Processing Conference, EUSIPCO 2008, 2008)
In this paper, a new approach is proposed for the concealment of lost samples in a high-resolution, high-quality video utilizing a low-resolution, highly compressed video with equal content. It is shown, that this inter-sequence error concealment is a very robust and flexible technique which outperforms conventional methods even for low bit rates. By adopting an affine motion model, the proposed technique also performs well in case of different image sizes, cropped image content and arbitrarily shaped loss area. An optimization method limits the computational complexity and maximizes the image restoration quality. If two or more low-resolution reference video sequences are available, the algorithm can be easily expanded. A typical application for inter-sequence error concealment is the restoration of a DVB-T video sequence in a terrestrial multi-broadcast-scenario with DVB-T, DVB-H and T-DMB. copyright by EURASI

Inter-sequence error concealment techniques for multi-broadcast TV reception - Tröger, Tobias; Kaup, André (IEEE Transactions on Broadcasting, 2011)
Mobile reception of digital TV signals often suffers from lost slices due to suboptimal channel characteristics. In this contribution we propose novel inter-sequence error concealment techniques which reconstruct lost blocks of distorted high-resolution TV signals by inserting error-free blocks of low-resolution reference signals. The proposed methods are well-suited for future automotive multi-broadcast receivers. As the reference signals are typically delayed, scaled and possibly cropped versions of the distorted TV signals, spatio-temporal image alignment is a crucial point. We first focus on pixel-based alignment utilizing a numerical optimization scheme. To reduce complexity a feature-based approach is introduced alternatively taking advantage of the scale-invariant feature transform. Simulations show that the proposed techniques can achieve supreme reconstruction quality outperforming conventional techniques by up to 15.0 dB PSNR. The pixel-based approach mostly reaches the maximum reconstruction quality for both raw and compressed video signals. Feature-based concealment performs slightly worse but reduces the computation time by a factor of up to 36.6 at a maximum loss of 1.0 dB. A novel hybrid concealment technique combining the proposed methods with a temporal approach further increases the reconstruction quality. Applying one simple and one sophisticated mode selection scheme, a gain of 3.2 dB and 4.6 dB can be achieved, respectively. © 2011 IEE
Keywords: Error concealment; image alignment; image registration; numerical optimization; scale-invariant feature transform; synchronization; video reconstruction

Intra-Frame Coding Using a Conditional Autoencoder - Brand, Fabian; Seiler, Jürgen; Kaup, André (IEEE Journal of Selected Topics in Signal Processing, 2020)
Exploiting spatial redundancy in images is responsible for a large gain in the performance of image and video compression. The main tool to achieve this is called intra-frame prediction. In most state-of-the-art video coders, intra prediction is applied in a block-wise fashion. Up to now angular prediction was dominant, providing a low-complexity method covering a large variety of content. With deep learning, however, it is possible to create prediction methods covering a wider range of content, being able to predict structures which traditional modes can not predict accurately. Using the conditional autoencoder structure, we are able to train a single artificial neural network which is able to perform multi-mode prediction. In this paper, we derive the approach from the general formulation of the intra-prediction problem and introduce two extensions for spatial mode prediction and for chroma prediction support. Moreover, we propose a novel latent-space-based cross component prediction. We show the power of our prediction scheme with visual examples and report average gains of 1.13% in Bjontegaard delta rate in the luma component and 1.21% in the chroma component compared to VTM using only traditional modes.<br

Intra Frame Prediction for Video Coding Using a Conditional Autoencoder Approach - Brand, Fabian; Seiler, Jürgen; Kaup, André (Picture Coding Symposium (PCS), 2019)
Intra prediction is a vital component of most modern image and video codecs. State of the art video codecs like High Efficiency Video Coding (HEVC) or the upcoming Versatile Video Coding (VVC) use a high number of directional modes. With the recent advances in deep learning, it is now possible to use artificial neural networks for intra frame prediction. Previously published approaches usually add additional ANN based modes or replace all modes by training several networks. In our approach, we use a single autoencoder network to first compress the original with help of already transmitted pixels to four parameters. We then use the parameters together with this support area to generate a prediction for the block. This way, we are able to replace all angular intra modes by a single ANN. In the experiments we compare our method with the intra prediction method currently used in the VVC Test Model (VTM). Using our method, we are able to gain up to 0.85 dB prediction PSNR with a comparable amount of side information or reduce the amount of side information by 2 bit per prediction unit with similar PSNR.<br
Keywords: Intra Prediction; Autoencoder; Video Coding

Intra To Inter: Towards Intra Prediction for Learning-Based Video Coders Using Optical Flow - Brand, Fabian; Seiler, Jürgen; Kaup, André (Proc. 2021 IEEE International Conference on Image Processing (ICIP), 2021)
Traditional video coders often rely on a block structure for transmission. Here each block is coded separately and sequentially and for each block the encoder can decide whether to use intra or inter prediction. This way, inter and intra prediction can be mixed within a single frame. This has advantages when new areas are uncovered, which were not present in the reference frame, and can hence not be predicted well. These areas are typically predicted using intra prediction. Currently much research goes into end-to-end-trained video coders which do not operate on a block level and typically use dense motion fields for inter prediction. There it is more difficult to incorporate intra prediction for uncovered regions. In this paper we propose a novel concept which enables us to reinterpret classical angular intra prediction in a way that we can transmit it as part of the dense motion field. We can save an average of 18% rate for the transmission of the motion vectors for the same quality of the prediction imag
Keywords: Inter Coding; Optical Flow; Non-Block-Based Video Coding; Intra Prediction

Introducing Latent Space Correlation to Conditional Autoencoders for Intra Prediction - Brand, Fabian; Seiler, Jürgen; Kaup, André (2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), 2020)

Introduction to the Issue on Acoustic Source Localization and Tracking in Dynamic Real-Life Scenes - Gannot, S.; Haardt, M.; Kellermann, Walter; Willett, P. (IEEE Journal of Selected Topics in Signal Processing, 2019)
CRIS-Team Scopus Importer:2019-05-02

Introduction to the issue on recent advances in video processing for consumer displays - Schonfeld, D.; Biemond, J.; De Haan, G.; Kaup, André (IEEE Journal of Selected Topics in Signal Processing, 2011)
The 12 papers in this special issue on recent advances in video processing for consumer displays fall into four areas: image enhancement and spatial resolution; video rate conversion; video deinterlacing and compression; and human vision and depth map

Introduction to the special issue on processing reverberant speech: Methodologies and applications - Nakatani, Tomohiro; Kellermann, Walter; Naylor, Patrick; Miyoshi, Masato; Juang, Biing Hwang (Fred) (IEEE Transactions on Audio Speech and Language Processing, 2010)
The 17 papers in this special issue focus on the methodologies and applications of processing reverberant speech. The issue highlights some major aspects of the recent progress in the fiel

Irregular Motion Detection in Automotive Navigation Systems with Soft Map Rotation - Pöllot, Martin; Springer, Dominic; Schleifer, Ralph; Nitsch, Monika; Kaup, André (Proceedings of the 37th IEEE International Conference On Consumer Electronics ICCE, 2019)
Visually unpleasant and erroneous positioning in navigation scenes is an issue for, but not limited to, premium vehicle manufacturers. In order to assure consumers high quality products, intelligent automated display testing is required. This paper presents an enhanced version of an error detection algorithm for navigation sequences based on novelty detection. Therefore, motion parameters obtained from real world navigation sequences are used. These are taken directly from the display using a screen-grabbing device. The proposed algorithm works solely on the information provided in these images. Several image processing methods extract information describing the overall motion between two consecutive frames which is then fed into a novelty detection algorithm to predict outliers. In this case, these outliers describe motions that should not occur in the displayed scene as they display impossible movements of the car. Experimental results demonstrate the improvement over the state of the art, being able to deal with recent changes in the displayed scene in order to hide or conceal present positioning errors. Evaluating the performance of the proposed algorithm, a precision of 76.2%, which is significantly higher than the state of the art method, could be reached while not decreasing the recall at all.<br
Keywords: soft rotation; navigation; novelty detection; screen content testing; automotive

Iterated coefficient updates of partitioned block frequency domain second-order volterra filters for nonlinear AEC - Zeller, Marcus; Kellermann, Walter (2007 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '07, 2007)
This paper presents the benefits of iterated coefficient updates for the adaptation of partitioned block frequency domain second-order Volterra filters when applied to nonlinear acoustic echo cancellation. In order to increase the convergence speed of an NLMS algorithm with separate kernel normalization, each input frame is used for several coefficient updates. This procedure effectively accelerates the convergence of the employed adaptive Volterra filters and is shown to be superior to processing with increased data overlap. The advantages of this novel approach are illustrated by experimental results for noise and speech input and guidelines for determining suitable numbers of iterations for the filter kernels are given. © 2007 IEE
Keywords: Adaptive signal processing; Echo suppression; Iterative methods; Volterra filters

Iterative Denoising-Based Mesh-to-Grid Reconstruction with Hyperparametric Adaptation - Koloda, Jan; Bätz, Michel; Kaup, André (IEEE International Workshop on Multimedia Signal Processing, 2017)
This paper presents a new method for the reconstruction of images from samples located at non-integer mesh positions. This is a common scenario for many image processing applications such as multi-image super-resolution, frame-rate up-conversion, or virtual view synthesis in multi-camera systems. The proposed method consists of an iterative procedure that employs adaptive denoising in order to reduce the reconstruction error. The denoising strength is controlled by a novel hyper-parametric adaptation mechanism that aims at maximizing the reconstruction quality in each iteration. Furthermore, the usage of the proposed hyperparametric model drastically reduces the number of necessary parameters that need to be trained or stored. In terms of luminance PSNR, the proposed approach improves the reconstruction quality by more than 1 dB with respect to the initial estimate and outperforms current state-of-the-art denoising-based reconstruction schemes by up to 0.5 d

Iterative DFT-domain inverse filter determination for adaptive listening room equalization - Schneider, Martin; Kellermann, Walter (International Workshop on Acoustic Signal Enhancement (IWAENC), 2012)
Listening room equalization (LRE) is used to remove the unwanted linear distortion of a reproduced acoustic scene by the listening room. This task comprises the identification of the loudspeaker-enclosure-microphone system (LEMS) and the determination of the pre-equalizing filters for the identified LEMS. For the latter subtask, filtered-X adaptive filtering algorithms allow an iterative optimization of the equalizers, whereby approximations do only affect the convergence speed, but not the optimality of the determined equalizer coefficients. Alternative approaches determine the equalizers by directly solving a system of linear equations, where the obtained equalizer coefficients may be suboptimal due to approximations which potentially facilitate an online implementation for MIMO LEMSs. In this paper, we present an alternative algorithm which allows an iterative optimization of the obtained equalizers, without any filtering of the excitation signals for the adaptation algorithm. Similar to the filtered-X generalized frequency-domain adaptive filtering (FxGFDAF), this approach uses a DFT-domain approximation with low computational cost. The evaluation results show an improved convergence behavior compared to the FxGFDA
Keywords: Adaptive filtering; DFT domain; Filtered-X; Inverse filtering; Listening room equalization

Iterative Optimization of Quarter Sampling Masks for Non-Regular Sampling Sensors - Grosche, Simon; Seiler, Jürgen; Kaup, André (25th IEEE International Conference on Image Processing (ICIP), 2018)

ItTakesTwo: Leveraging Peer Representations for Semi-supervised LiDAR Semantic Segmentation - Liu, Yuyuan; Chen, Yuanhong; Wang, Hu; Belagiannis, Vasileios; Reid, Ian; Carneiro, Gustavo (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2025)
The costly and time-consuming annotation process to produce large training sets for modelling semantic LiDAR segmentation methods has motivated the development of semi-supervised learning (SSL) methods. However, such SSL approaches often concentrate on employing consistency learning only for individual LiDAR representations. This narrow focus results in limited perturbations that generally fail to enable effective consistency learning. Additionally, these SSL approaches employ contrastive learning based on the sampling from a limited set of positive and negative embedding samples. This paper introduces a novel semi-supervised LiDAR semantic segmentation framework called ItTakesTwo (IT2). IT2 is designed to ensure consistent predictions from peer LiDAR representations, thereby improving the perturbation effectiveness in consistency learning. Furthermore, our contrastive learning employs informative samples drawn from a distribution of positive and negative embeddings learned from the entire training set. Results on public benchmarks show that our approach achieves remarkable improvements over the previous state-of-the-art (SOTA) methods in the field. https://github.com/yyliu01/IT2.
CRIS-Team Scopus Importer:2024-10-25
Keywords: LiDAR Semantic Segmentation; Semi-supervised Learning

Joint Acoustic Echo Cancellation and Blind Source Extraction based on Independent Vector Extraction - Haubner, Thomas; Koldovsky, Zbynek; Kellermann, Walter (International Workshop on Acoustic Signal Enhancement (IWAENC), 2022)

Joint Audio-Video Object Localization and Tracking - Strobel, Norbert; Spors, Sascha; Rabenstein, Rudolf (IEEE Signal Processing Magazine, 2001)
LMS::161

Joint audio-video object localization using a recursive multi-state multi-sensor estimator - Strobel, Norbert; Spors, Sascha; Rabenstein, Rudolf (2000 IEEE Interntional Conference on Acoustics, Speech, and Signal Processing, 2000)
Object localization based on audio and video information is important for the analysis of dynamic scenes such as video conferences or traffic situations. In this paper, we view the dynamic audio-video object localization problem as a joint recursive estimation problem. It is solved using a decentralized Kalman filter fusing both audio and video position estimates. To better take into account different object maneuvers, multiple state-space equations are also incorporated. The result is a recursive multi-state multi-sensor estimator. Experiments show that it yields significantly improved joint position estimates compared to results achieved by using either an audio or a video system only.

Joint Audio-Video Object Tracking - Spors, Sascha; Rabenstein, Rudolf; Strobel, Norbert (IEEE International Conference on Image Processing (ICIP), 2001)
A method for joint localization of a single object using an audio sensor and a video sensor were outlined. These localizers are best described using different coordinate systems as the audio position estimates are nonlinearly related to the state. Whereas video sensor system operates in Cartesian coordinates. The local Kalman filters consists of three main components: local Kalman filter at the microphone array; a local Kalman filter at the video camera; and a global Kalman filter at the fusion center. The loss of optimality is the price to pay for replacing the linear Kalman filter with an extended Kalman filter. Quantitative experiments have shown that such a strategy can improve object localization and tracking.

Joint Audio-Video Signal Processing for Object Localization and Tracking - Strobel, Norbert; Spors, Sascha; Rabenstein, Rudolf (Microphone Arrays, 2001)

Joint audiovisual signal processing - Rabenstein, Rudolf (IEEE Signal Processing Magazine, 2000)
LMS::1608

Joint audiovisual signal processing II - Rabenstein, Rudolf (IEEE Signal Processing Magazine, 2001)

Joint consistent graph synthesis and speed of sound estimation for acoustic localization in multi-source reverberant environments - Annibale, Paolo; Rabenstein, Rudolf; Kreissig, Martin; Yang, Bin (8th International Workshop on Multidimensional Systems (nDS 2013), 2013)
Acoustic source localization using microphone arrays is often based on the Time Difference of Arrivals (TDOA). The estimation of Time Differences of Arrivals (TDOAs) relies on the detection of the peak in suitable cross-correlations between microphone signals. Unfortunately, in real environments a number of spurious peaks occur in the cross-correlations due to multiple sources, environment reflections, source signal periodicity and other interferences. Thus, techniques have been proposed that aim at selecting the correct direct-path peak and at assigning it to the desired source. Such techniques are known in the literature as TDOA-disambiguation. In this paper we propose a novel disambiguation approach based on the combination of two criteria: consistent TDOA-graph synthesis and acceptable speed of sound. Simulations and experimental results are reported to show the effectiveness of the proposed approach in reverberant environment

Joint Content-Adaptive Dictionary Learning And Sparse Selective Extrapolation For Cross-Spectral Image Reconstruction - Genser, Nils; Seiler, Jürgen; Kaup, André (2020 IEEE International Conference on Image Processing (ICIP), 2020)
Numerous applications deal with distorted images, e.g., during transmission over lossy channels in image coding, or to reconstruct occlusions in multi-color multi-view imaging scenarios. In many cases, not all spectral channels are distorted, or the losses distribute differently between the channels. Recently, efforts were made to develop reference guided approaches to reconstruct distorted spectral content. However, these methods are only able to exploit information from a single reference, even if there are multiple spectral components available that could be used for reconstruction. To overcome this limitation, a novel method is proposed in this paper, which introduces a content-adaptive dictionary learning approach that is applicable to an arbitrary number of references. With the novel approach, an average PSNR gain of approx. 1.5 dB is achieved in comparison to the best recently published state-of-the-art methods for a single reference component. Moreover, the presence of multiple guidance channels can enhance the reconstruction by another 6 dB on average. At the same time, the complexity of the novel algorithm is significantly lower than for previously published method

Joint DOA and TDOA estimation for 3D localization of reflective surfaces using eigenbeam MVDR and spherical microphone arrays - Sun, Haohai; Mabande, Edwin; Kowalczyk, Konrad; Kellermann, Walter (36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, 2011)
Methods of 3D direction of arrival (DOA) estimation, coherent source detection and reflective surface localization are studied, based on recordings by a spherical microphone array. First, the spherical harmonics domain minimum variance distortionless response (EB-MVDR) beamformer is employed for the localization of broadband coherent sources, which is characterized by simpler frequency focusing matrices than the corresponding element-space implementation, and by a higher resolution than conventional spherical array beamformers. After the DOA estimation step, the source signals are extracted by EB-MVDRs. Then, by computing the crosscorrelation functions between the extracted signals, the coherent sources are detected and their time differences of arrival (TDOA) are estimated. Given the positions of the array and the reference source, and the estimated DOA and TDOA of the coherent sources, the positions of the major reflectors can be inferred. Experimental results in a real room validate the proposed method. © 2011 IEE
Keywords: DOA; localization of reflectors; spherical harmonics; Spherical microphone array; TDOA

Jointly Resampling and Reconstructing Corrupted Images for Image Classification using Frequency-Selective Mesh-to-Grid Resampling - Heimann, Viktoria; Spruck, Andreas; Kaup, André (2022 IEEE 24TH INTERNATIONAL WORKSHOP ON MULTIMEDIA SIGNAL PROCESSING (MMSP), 2022)
CRIS-Team WoS Importer:2022-12-30

Joint Optimization of Acoustic Echo Cancellation and Adaptive Beamforming - Herbordt, Wolfgang; Kellermann, Walter; Nakamura, S. (Topics in Acoustic Echo and Noise Control, 2006)

Joint optimization of LCMV beamforming and acoustic echo cancellation - Herbordt, Wolfgang; Kellermann, Walter; Nakamura, S. (12th European Signal Processing Conference, EUSIPCO 2004, 2004)
Full-duplex hands-free acoustic human/machine interfaces often require the combination of acoustic echo cancellation and speech enhancement in order to suppress acoustic echoes, local interference, and noise. In order to optimally exploit positive synergies between acoustic echo cancellation and speech enhancement, we present in this contribution a combined least-squares (LS) optimization criterion for the integration of acoustic echo cancellation and adaptive linearly-constrained minimum variance (LCMV) beamforming. Based on this optimization criterion, we derive a computationally efficient system based on the generalized sidelobe canceller (GSC), which effectively deals with scenarioes with time-varying acoustic echo paths and simultaneous presence of double-talk of acoustic echoes, local interference, and desired speakers.

Joint optimization of LCMV beamforming and acoustic echo cancellation for automatic speech recognition - Herbordt, Wolfgang; Nakamura, S.; Kellermann, Walter (2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05, 2005)
For full-duplex hands-free acoustic human/machine interfaces, often a combination of acoustic echo cancellation and speech enhancement in order to suppress acoustic echoes, local interference, and noise is required. In order to optimally exploit positive synergies between acoustic echo cancellation and speech enhancement, we presented in an earlier work a combined least-squares (LS) optimization criterion for the integration of acoustic echo cancellation and adaptive linearly-constrained minimum variance (LCMV) beamforming [1]. In this contribution, we illustrate the efficiency of the proposed solution in situations with high levels of background noise and with time-varying echo paths and frequent double-talk by speech recognition experiments. ©2005 IEE

Joint Optimization of Rate, Distortion, and Decoding Energy for HEVC Intraframe Coding - Herglotz, Christian; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2016)
This paper presents a novel algorithm that aims at minimizing the required decoding energy by exploiting a general energy model for HEVC-decoder solutions. We incorporate the energy model into the HEVC encoder such that it is capable of constructing a bit stream whose decoding process consumes less energy than the decoding process of a conventional bit stream. To achieve this, we propose to extend the traditional Rate-Distortion-Optimization scheme to a Decoding-Energy-Rate-Distortion approach. To obtain fast encoding decisions in the optimization process, we derive a fixed relation between the quantization parameter and the Lagrange multiplier for energy optimization. Our experiments show that this concept is applicable for intraframe-coded videos and that for local playback as well as online streaming scenarios, up to 15% of the decoding energy can be saved at the expense of a bitrate increase of approximately the same magnitud
Keywords: HEVC; energy; decoder; model; rate-distortion-optimization

Joint Optimization of Rate, Distortion, and Maximum Absolute Error for Compression of Medical Volumes Using HEVC Intra - Jaskolka, Karina; Kaup, André (Picture Coding Symposium, 2018)
Many visual quality metrics are used to measure the quality of lossy compressed images and videos, and are integrated in the rate-distortion optimization of hybrid video codecs. However, most of the metrics focus on the average objective quality in a picture. In certain applications, like medical image processing, the maximum absolute error should be more weighted. In this paper, the rate-distortion optimization of HEVC is extended by integrating this error metric. Thus, rate, average error, and maximum absolute error are jointly optimized. Furthermore, a weighting factor α is included in the calculation of the optimization for balancing the ratio between average and maximum absolute error. For HEVC intra with α = 0.25 an average maximum absolute error reduction of -25.63 can be achieved, while the bitrate increases slightly by 0.59%. Furthermore, the visual quality of the medical volumes improves and the data fidelity increases, i.e. less block artifacts appear and less structure disappear.<br

Joint Out-of-Distribution Detection and Uncertainty Estimation for Trajectory Prediction - Wiederer, Julian; Schmidt, Julian; Kressel, Ulrich; Dietmayer, Klaus; Belagiannis, Vasileios (2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023)
Despite the significant research efforts on trajectory prediction for automated driving, limited work exists on assessing the prediction reliability. To address this limitation we propose an approach that covers two sources of error, namely novel situations with out-of-distribution (OOD) detection and the complexity in in-distribution (ID) situations with uncertainty estimation. We introduce two modules next to an encoder-decoder network for trajectory prediction. Firstly, a Gaussian mixture model learns the probability density function of the ID encoder features during training, and then it is used to detect the OOD samples in regions of the feature space with low likelihood. Secondly, an error regression network is applied to the encoder, which learns to estimate the trajectory prediction error in supervised training. During inference, the estimated prediction error is used as the uncertainty. In our experiments, the combination of both modules outperforms the prior work in OOD detection and uncertainty estimation, on the Shifts robust trajectory prediction dataset by 2.8 % and 10.1%, respectively. The code is publicly available44project page: https://github.com/againerju/joodu.
CRIS-Team Scopus Importer:2024-12-16

Joint Regression Modeling and Sparse Spatial Refinement for High-Quality Reconstruction of Distorted Color Images - Genser, Nils; Seiler, Jürgen; Kaup, André (IEEE International Conference on Image Processing (ICIP), 2019)
High quality algorithms are demanded to reconstruct distorted color images in a variety of applications. For example, distortions can result during transmission over lossy channels in image coding or in multi-view imaging scenarios. In general, not all color channels are equally affected and the losses distribute differently in-between channels. However, state-of-the-art methods process color channels independently and do not take the cross color information into account. Thus, a novel and powerful reconstruction algorithm is formulated in this contribution that exploits color as well as spatial information. Therefore, an initial model is estimated for the distorted area using a reference channel. Then, its quality is estimated and a spatial weighting model is set-up. Afterwards, the initial inter channel prediction is refined by generating a sparse model that takes the spatial correlations into account, as well. Consequently, the proposed method achieves an outstanding quality compared to state-of-the-art methods.<br
Keywords: Image reconstruction; error concealment; RGB imaging; multi-color imaging; signal extrapolation

Joint Shape and Centroid Adaptive Frequency Selective Extrapolation for the Reconstruction of Arbitrarily Shaped Loss Areas - Bätz, Michel; Schnurrer, Wolfgang; Koloda, Jan; Eichenseer, Andrea; Kaup, André (Picture Coding Symposium, 2016)
Reconstructing missing areas of arbitrary shape and size is particularly important in error-prone communication as well as in applications where motion compensation is conducted such as multi-image super-resolution or framerate up-conversion. To that end, frequency selective extrapolation is an effective image reconstruction technique. This approach was originally designed for block losses and has recently been enhanced by a centroid adaptation to improve the reconstruction quality in the case of arbitrarily shaped loss areas. In this paper, we reuse the idea of centroid adaptation and introduce a novel shape adaptation so as to better assign higher weights to more relevant pixels. Moreover, we propose to combine both the shape adaptation and the centroid adaptation into a joint solution to further improve the reconstruction quality. To evaluate the proposed method, three different loss patterns are used. Simulation results yield an average gain in luminance PSNR of up to 0.2 dB for the high quality profile and 3.4 dB for the high efficiency profile, respectively. A visual comparison confirms these result

Joint Temporal and Inter-Sequence Error Concealment for Multi-Broadcast-Reception - Tröger, Tobias; Heiber, H.; Schmitt, A.; Kaup, André (10th Workshop Digital Broadcasting, 2009)
Digital TV signals being transmitted over error-prone channels often suffer from lost macro blocks or slices. In literature, a wide range of approaches for error concealment is described exploiting spatial and/or temporal references. In 2008, we introduced a new paradigm for error concealment utilizing several representations of a particular TV signal which are avail-able in multi-broadcast-reception scenarios. This inter-sequence error concealment provides supreme reconstruction quality in case of low compression of the reference signals outperforming state-of-the-art techniques by up to 15 dB PSNRY. However, inter-sequence error concealment may fail if the reference signals are characterized by degraded image quality. In this paper, we propose a combination of temporal error concealment approaches and inter-sequence error concealment which can cope with this drawback. Joint temporal and inter-sequence error concealment outnumbers inter-sequence error concealment by up to 5.8 dB PSNRY on average. A peak gain of up to 14.9 dB PSNRY can be achieved. Finally, the proposed algorithm can also cope with lost parts of the reference signals in practical application scenario
LMS::824

Kernel-based MMSE Multimedia Signal Reconstruction and its Application to Spatial Error Concealment - Koloda, Jan; Peinado, A.M.; Sánchez, V. (IEEE Transactions on Multimedia, 2014)
This paper proposes a novel approach for multimedia signal reconstruction based on kernel density estimation (KDE). We make use of a vector formalism in which vectors consist of a first subvector containing a set of missing samples and a second one containing a set of available context samples. The missing subvector is reconstructed by a minimum mean square error estimator which employs a probability density function (pdf) obtained by KDE. As in any kernel-based method, the main issue to deal with is the estimation of an appropriate kernel bandwidth. We propose an adaptive procedure for bandwidth estimation (BE) especially conceived for signal reconstruction. Thus, unlike general KDE or kernel-based regression, which try to obtain a general fit, the focus of this BE procedure is on the specific missing subvector. Also, in order to exploit local signal correlations, our BE proposal adopts a scaling approach in which the bandwidth is computed as the local covariance matrix scaled by two factors. These two scale factors are obtained by minimization of two different approximations to the reconstruction error. The resulting reconstruction methodology is tested on a spatial error concealment (EC) application in which intracoded images have been transmitted through an error prone channel. The experimental results show the superiority of the proposed approach over a wide range of existing EC technique
LMS::1533
Keywords: bandwidth estimation; kernel density estimation; multimedia signal reconstruction; spatial error concealment

Key Point Agnostic Frequency-Selective Mesh-to-Grid Image Resampling using Spectral Weighting - Heimann, Viktoria; Genser, Nils; Kaup, André (IEEE International Workshop on Multimedia Signal Processing (MMSP), 2020)
Many applications in image processing require re- sampling of arbitrarily located samples onto regular grid po- sitions. This is important in frame-rate up-conversion, super- resolution, and image warping among others. A state-of-the-art high quality model-based resampling technique is frequency- selective mesh-to-grid resampling which requires pre-estimation of key points. In this paper, we propose a new key point agnostic frequency-selective mesh-to-grid resampling that does not depend on pre-estimated key points. Hence, the number of data points that are included is reduced drastically and the run time decreases significantly. To compensate for the key points, a spectral weighting function is introduced that models the optical transfer function in order to favor low frequencies more than high ones. Thereby, resampling artefacts like ringing are supressed reliably and the resampling quality increases. On average, the new AFSMR is conceptually simpler and gains up to 1.2 dB in terms of PSNR compared to the original mesh-to-grid resampling while being approximately 14.5 times faster.
Keywords: image reconstruction; resampling; scattered data

Knowing What to Label for Few Shot Microscopy Image Cell Segmentation - Dawoud, Youssef; Bouazizi, Arij; Ernst, Katharina; Carneiro, Gustavo; Belagiannis, Vasileios (Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023, 2023)
In microscopy image cell segmentation, it is common to train a deep neural network on source data, containing different types of microscopy images, and then fine-tune it using a support set comprising a few randomly selected and annotated training target images. In this paper, we argue that the random selection of unlabelled training target images to be annotated and included in the support set may not enable an effective fine-tuning process, so we propose a new approach to optimise this image selection process. Our approach involves a new scoring function to find informative unlabelled target images. In particular, we propose to measure the consistency in the model predictions on target images against specific data augmentations. However, we observe that the model trained with source datasets does not reliably evaluate consistency on target images. To alleviate this problem, we propose novel self-supervised pretext tasks to compute the scores of unlabelled target images. Finally, the top few images with the least consistency scores are added to the support set for oracle (i.e., expert) annotation and later used to fine-tune the model to the target images. In our evaluations that involve the segmentation of five different types of cell images, we demonstrate promising results on several target test sets compared to the random selection approach as well as other selection approaches, such as Shannon's entropy and Monte-Carlo dropout.
CRIS-Team Scopus Importer:2023-03-10
Keywords: Applications: Biomedical/healthcare/medicine