3D Rendering Framework for Data Augmentation in Optical Character Recognition - Spruck, Andreas; Hawesch, Maximiliane; Maier, Anatol; Rieß, Christian; Seiler, Jürgen; Kaup, André (ISSCS 2021 - International Symposium on Signals, Circuits and Systems, 2021)
In this paper, we propose a data augmentation framework for Optical Character Recognition (OCR). The proposed framework is able to synthesize new viewing angles and illumination scenarios, effectively enriching any available OCR dataset. Its modular structure allows to be modified to match individual user requirements. The framework enables to comfortably scale the enlargement factor of the available dataset. Furthermore, the proposed method is not restricted to single frame OCR but can also be applied to video OCR. We demonstrate the performance of our framework by augmenting a 15% subset of the common Brno Mobile OCR dataset. Our proposed framework is capable of leveraging the performance of OCR applications especially for small datasets. Applying the proposed method, improvements of up to 2.79 percentage points in terms of Character Error Rate (CER), and up to 7.88 percentage points in terms of Word Error Rate (WER) are achieved on the subset. Especially the recognition of challenging text lines can be improved. The CER may be decreased by up to 14.92 percentage points and the WER by up to 18.19 percentage points for this class. Moreover, we are able to achieve smaller error rates when training on the 15% subset augmented with the proposed method than on the original nonaugmented full dataset.
CRIS-Team Scopus Importer:2021-09-17

Deep Learning Based Cross-Spectral Disparity Estimation For Stereo Imaging - Genser, Nils; Spruck, Andreas; Seiler, Jürgen; Kaup, André (2020 IEEE International Conference on Image Processing (ICIP), 2020)
Recently, cross-spectral stereo-camera setups found their way from special applications to mass market, especially in smartphones, automotive systems, or drones. In the following, a novel concept is introduced to bring stereo cameras and cross-spectral disparity estimation together. So far, either monomodal stereo algorithms exist that are not suitable for cross-spectral image registration, or structural template matching is applied that achieves a low quality. To overcome these limitations, a technique is proposed to synthesize arbitrary spectral components from widely available color stereo databases, and to retrain mono-modal deep learning methods. In this contribution, the estimation of spectral bands based on random processes is shown together with noise models, which also allow for a robust registration of narrowband components. The theoretical examination is completed by an extensive evaluation, including a self-manufactured cross-spectral camera setup. In comparison to state-of-the-art techniques, the end-point error is on average reduced by a factor of seve

Forensic License Plate Recognition with Compression-Informed Transformers - Moussa, Denise; Maier, Anatol; Spruck, Andreas; Seiler, Jürgen; Rieß, Christian (2022 IEEE International Conference on Image Processing (ICIP), 2022)
Forensic license plate recognition (FLPR) remains an open challenge in legal contexts such as criminal investigations, where unreadable license plates (LPs) need to be deciphered from highly compressed and/or low resolution footage, e.g., from surveillance cameras. In this work, we propose a side-informed Transformer architecture that embeds knowledge on the input compression level to improve recognition under strong compression. We show the effectiveness of Transformers for license plate recognition (LPR) on a low-quality real-world dataset. We also provide a synthetic dataset that includes strongly degraded, illegible LP images and analyze the impact of knowledge embedding on it. The network outperforms existing FLPR methods and standard state-of-the art image recognition models while requiring less parameters. For the severest degraded images, we can improve recognition by up to 8.9 percent points.<br
Keywords: License Plate Recognition; Image Forensics

Frame Rate Up-Conversion Using Key Point Agnostic Frequency-Selective Mesh-to-Grid Resampling - Heimann, Viktoria; Spruck, Andreas; Kaup, André (IEEE International Conference on Acoustics, Speech and Signal Processing, 2021)
High frame rates are desired in many fields of application. As in many cases the frame repetition rate of an already captured video has to be increased, frame rate up-conversion (FRUC) is of high interest. We conduct a motion compensated approach. From two neighboring frames, the motion is estimated and the neighboring pixels are shifted along the motion vector into the frame to be reconstructed. For displaying, these irregularly distributed mesh pixels have to be resampled onto regularly spaced grid positions. We use the model-based key point ag- nostic frequency-selective mesh-to-grid resampling (AFSMR) for this task and show that AFSMR works best for applications that contain irregular meshes with varying densities. AFSMR gains up to 3.2 dB in contrast to the already high performing frequency-selective mesh-to-grid resampling (FSMR). Addition- ally, AFSMR increases the run time by a factor of 11 relative to FSM

Frequency-Selective Geometry Upsampling of Point Clouds - Heimann, Viktoria; Spruck, Andreas; Kaup, André (IEEE International Conference on Image Processing (ICIP) 2022, 2022)

Frequency-Selective Mesh-to-Mesh Resampling for Color Upsampling of Point Clouds - Heimann, Viktoria; Spruck, Andreas; Kaup, André (Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP), 2021)
With the increased use of virtual and augmented reality applications, the importance of point cloud data rises. High-quality capturing of point clouds is still expensive and thus, the need for point cloud super-resolution or point cloud upsampling techniques emerges. In this paper, we propose an interpolation scheme for color upsampling of three-dimensional color point clouds. As a point cloud represents an object’s surface in three-dimensional space, we first conduct a local transform of the surface into a two-dimensional plane. Secondly, we propose to apply a novel Frequency-Selective Mesh-to-Mesh Resampling (FSMMR) technique for the interpolation of the points in 2D. FSMMR generates a model of weighted superpositions of basis functions on scattered points. This model is then evaluated for the final points in order to increase the resolution of the original point cloud. Evaluation shows that our approach outperforms common interpolation schemes. Visual comparisons of the jaguar point cloud underlines the quality of our upsampling results. The high performance of FSMMR holds for various sampling densities of the input point clou
Keywords: point cloud; color upsampling; frequency- selective

Increasing the Accuracy of a Neural Network Using Frequency Selective Mesh-to-Grid Resampling - Spruck, Andreas; Heimann, Viktoria; Kaup, André (IEEE International Symposium on Circuits and Systems (ISCAS), 2022)

Jointly Resampling and Reconstructing Corrupted Images for Image Classification using Frequency-Selective Mesh-to-Grid Resampling - Heimann, Viktoria; Spruck, Andreas; Kaup, André (2022 IEEE 24TH INTERNATIONAL WORKSHOP ON MULTIMEDIA SIGNAL PROCESSING (MMSP), 2022)
CRIS-Team WoS Importer:2022-12-30

Motion-Adapted Three-Dimensional Frequency Selective Extrapolation - Spruck, Andreas; Jonscher, Markus; Seiler, Jürgen; Kaup, André (IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019)
It has been shown, that high resolution images can be acquired using a low resolution sensor with non-regular sampling. Therefore, post-processing is necessary. In terms of video data, not only the spatial neighborhood can be used to assist the reconstruction, but also the temporal neighbor-hood. A popular and well performing algorithm for this kind of problem is the three-dimensional frequency selective extrapolation (3D-FSE) for which a motion adapted version is introduced in this paper. This proposed extension solves the problem of changing content within the area considered by the 3D-FSE, which is caused by motion within the sequence. Because of this motion, it may happen that regions are em-phasized during the reconstruction that are not present in the original signal within the considered area. By that, false content is introduced into the extrapolated sequence, which affects the resulting image quality negatively. The novel extension, presented in the following, incorporates motion data of the sequence in order to adapt the algorithm accordingly, and compensates changing content, resulting in gains of up to 1.75 dB compared to the existing 3D-FS

Quality Assurance of Weld Seams Using Laser Triangulation Imaging and Deep Neural Networks - Spruck, Andreas; Seiler, Jürgen; Roll, Michael; Dudziak, Thomas; Eckstein, Jürgen; Kaup, André (2020 IEEE International Workshop on Metrology for Industry 4.0 & IoT, 2020)

Reliability Scoring for the Recognition of Degraded License Plates - Maier, Anatol; Moussa, Denise; Spruck, Andreas; Seiler, Jürgen; Rieß, Christian (2022 18th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2022)
Criminal investigations oftentimes need the identification of license plates of escape vehicles. The vehicles may be recorded by low-quality cameras in the wild. Their license plates may be unreadable for police officers. Recent efforts aim to use machine learning to forensically decipher license plates from such low-quality images. These methods operate near the information-theoretic limit of recognition and hence show quite high error rates. Unfortunately, it is unclear when such prediction errors occur, which makes it difficult to use these methods in practice. In this work, we propose a Bayesian Neural Network to inherently incorporate a reliability measure into the classifier. We additionally propose to integrate multiple estimations with an entropy weight to further improve the reliability. Our experiments show that this uncertainty metric dramatically reduces the number of false predictions while preserving most of the true predictions.<br

Resampling and Reconstructing Corrupted Images for Image Classification - Heimann, Viktoria; Spruck, Andreas; Kaup, André (IEEE International Workshop on Multimedia Signal Processing, 2022)