Application of SORT on Multi-Object Tracking and Segmentation - Köferl, Franz; Link, Johannes; Eskofier, Björn (Conference on Computer Vision and Pattern Recognition; 5th BMTT MOTChallenge Workshop: Multi-Object Tracking and Segmentation, 2020)
Multiple object tracking and segmentation (MOTS) on monocular images using object detectors without any end-to-end learning of the tracking step has been a common problem historically. Including the posterior of the object detector into the tracking step proves to be difficult, because features from the detection step are reduced to only segmentation mask, object probability, and class information. Based on this, solving tasks like combining of segmentation masks and the actual tracking step is still the main challenge. We adapt an existing simple online tracking method (SORT) based on bounding boxes. The tracking process predicts trajectory using a Kalman filter and matches tracks to detections using a simple IOU metric. The sMOTSA score on the test set of KITTI-MOTS are 64.1 (cars), 54.5 (pedestrian) and on the test set of MOTS20 is 56.8 (pedestrian).<br
Keywords: Tracking; Detection; Segmentation

Interactive Segmentation of RGB-D Indoor Scenes using Deep Learning - Rüthlein, Maximilian; Köferl, Franz; Mehringer, Wolfgang; Eskofier, Björn (International Conference on Machine Learning; 2nd ICML 2020 Workshop on Human in the Loop Learning, 2020)
Human segmentation of point clouds for the creation of datasets for deep learning is a tedious and especially time-consuming task. Interactive segmentation methods from the domain of RGB images reduce this time effort by using an iterative scheme of deep neural networks and human labelers. We apply this interactive segmentation scheme to the point cloud domain, using PointNet as our backbone and exploiting user marker-information by label propagation. The evaluation was based on a user study and suggests a significant increase in segmentation speed, which is traded off for mask quality. On average, our approach decreased segmentation time by 21.1%. We consider this work as an important first step into the domain of interactive point cloud segmentation.<br

Just a Matter of Scale? Reevaluating Scale Equivariance in Convolutional Neural Networks - Altstidl, Thomas Robert; Nguyen, An; Schwinn, Leo; Köferl, Franz; Mutschler, Christopher; Eskofier, Björn; Zanca, Dario (Proc. Intl. Joint Conf. Neural Netw. (IJCNN), 2023)
The widespread success of convolutional neural networks may largely be attributed to their intrinsic property of translation equivariance. However, convolutions are not equivariant to variations in scale and fail to generalize to objects of different sizes. Despite recent advances in this field, it remains unclear how well current methods generalize to unobserved scales on real-world data and to what extent scale equivariance plays a role. To address this, we propose the novel Scaled and Translated Image Recognition (STIR) benchmark based on four different domains. Additionally, we introduce a new family of models that applies many re-scaled kernels with shared weights in parallel and then selects the most appropriate one. Our experimental results on STIR show that both the existing and proposed approaches can improve generalization across scales compared to standard convolutions. We also demonstrate that our family of models is able to generalize well towards larger scales and improve scale equivariance. Moreover, due to their unique design we can validate that kernel selection is consistent with input scale. Even so, none of the evaluated models maintain their performance for large differences in scale, demonstrating that a general understanding of how scale equivariance can improve generalization and robustness is still lacking.<br
Keywords: Image recognition; Neural networks; Benchmark testing; Robustness; Convolutional neural networks; Kernel; Standards

Lightweight, generative variant exploration for high-performance graphics applications - Selgrad, Kai; Lier, Alexander; Köferl, Franz; Stamminger, Marc; Lohmann, Daniel (ACM Bd. 51, Nr. 3, 2015)
Rendering performance is an everlasting goal of computer graphics and significant driver for advances in both, hardware architecture and algorithms. Thereby, it has become possible to apply advanced computer graphics technology even in low-cost embedded appliances, such as car instruments. Yet, to come up with an efficient implementation, developers have to put enormous efforts into hardware/problem-specific tailoring, fine-tuning, and domain exploration, which requires profound expert knowledge. If a good solution has been found, there is a high probability that it does not work as well with other architectures or even the next hardware generation. Generative DSL-based approaches could mitigate these efforts and provide for an efficient exploration of algorithmic variants and hardware-specific tuning ideas. However, in vertically organized industries, such as automotive, suppliers are reluctant to introduce these techniques as they fear loss of control, high introduction costs, and additional constraints imposed by the OEM with respect to software and tool-chain certification. Moreover, suppliers do not want to share their generic solutions with the OEM, but only concrete instances. To this end, we propose a light-weight and incremental approach for meta programming of graphics applications. Our approach relies on an existing formulation of C-like languages that is amenable to meta programming, which we extend to become a lightweight language to combine algorithmic features. Our method provides a concise notation for meta programs and generates easily sharable output in the appropriate C-style target language.

PolarBearVidID: A Video-Based Re-Identification Benchmark Dataset for Polar Bears - Zürl, Matthias; Dirauf, Richard; Köferl, Franz; Steinlein, Nils; Sueskind, Jonas; Zanca, Dario; Brehm, Ingrid; von Fersen, Lorenzo; Eskofier, Björn (Animals, 2023)
Automated monitoring systems have become increasingly important for zoological institutions in the study of their animals’ behavior. One crucial processing step for such a system is the re-identification of individuals when using multiple cameras. Deep learning approaches have become the standard methodology for this task. Especially video-based methods promise to achieve a good performance in re-identification, as they can leverage the movement of an animal as an additional feature. This is especially important for applications in zoos, where one has to overcome specific challenges such as changing lighting conditions, occlusions or low image resolutions. However, large amounts of labeled data are needed to train such a deep learning model. We provide an extensively annotated dataset including 13 individual polar bears shown in 1431 sequences, which is an equivalent of 138,363 images. PolarBearVidID is the first video-based re-identification dataset for a non-human species to date. Unlike typical human benchmark re-identification datasets, the polar bears were filmed in a range of unconstrained poses and lighting conditions. Additionally, a video-based re-identification approach is trained and tested on this dataset. The results show that the animals can be identified with a rank-1 accuracy of 96.6%. We thereby show that the movement of individual animals is a characteristic feature and it can be utilized for re-identificatio
Keywords: deep learning; re-identification; computer vision; animal identification; animal welfare; automated behavior analysis; motion features; video-based method; dataset

Tracking the visitor: An optical indoor system for visitor research in museums - Köferl, Franz; Jami, Jitin; Li, Jindong; Zanca, Dario; Eskofier, Björn; Zürl, Matthias (AI in Museums: Reflections, Perspectives and Applications, 2023)
The analysis of the success of any exhibition depends on the visitor experience. The data required for analysis is usually painstakingly collected by hand. We propose a large-scale optical tracking pipeline to estimate visitor data such as visit trajectory, duration, and, potentially, other personal parameters like age, weight, and sex, yet remain ethically acceptable by obtaining visitor consent. We further show, in preliminary results, that the edge device has a localization error of 0.64 meters and an average precision of 0.2. With this work-in-progress, we intend to ensure a viable alternative to current data collection processes in museum research.
CRIS-Team Scopus Importer:2024-12-16