Data-Centric Benchmarking of Neural Network Architectures for the Univariate Time Series Forecasting Task - Schlieper, Philipp; Dombrowski, Mischa; Nguyen, An; Zanca, Dario; Eskofier, Björn (Forecasting, 2024)
Time series forecasting has witnessed a rapid proliferation of novel neural network approaches in recent times. However, performances in terms of benchmarking results are generally not consistent, and it is complicated to determine in which cases one approach fits better than another. Therefore, we propose adopting a data-centric perspective for benchmarking neural network architectures on time series forecasting by generating ad hoc synthetic datasets. In particular, we combine sinusoidal functions to synthesize univariate time series data for multi-input-multi-output prediction tasks. We compare the most popular architectures for time series, namely long short-term memory (LSTM) networks, convolutional neural networks (CNNs), and transformers, and directly connect their performance with different controlled data characteristics, such as the sequence length, noise and frequency, and delay length. Our findings suggest that transformers are the best architecture for dealing with different delay lengths. In contrast, for different noise and frequency levels and different sequence lengths, LSTM is the best-performing architecture by a significant amount. Based on our insights, we derive recommendations which allow machine learning (ML) practitioners to decide which architecture to apply, given the dataset’s characteristics.
CRIS-Team Scopus Importer:2024-10-04
Keywords: data synthesis; deep learning; model selection; neural networks; time series; univariate forecasting

Enhancing unsupervised outlier model selection: A study on ireos algorithms - Schlieper, Philipp; Luft, Hermann; Klede, Kai; Strohmeyer, Christoph; Eskofier, Björn; Zanca, Dario (ACM Transactions on Knowledge Discovery from Data, 2024)
Outlier detection stands as a critical cornerstone in the field of data mining, with a wide range of applications spanning from fraud detection to network security. However, real-world scenarios often lack labeled data for training, necessitating unsupervised outlier detection methods. This study centers on Unsupervised Outlier Model Selection (UOMS), with a specific focus on the family of Internal, Relative Evaluation of Outlier Solutions (IREOS) algorithms. IREOS measures outlier candidate separability by evaluating multiple maximum-margin classifiers and, while effective, it is constrained by its high computational demands. We investigate the impact of several different separation methods in UOMS in terms of ranking quality and runtime. Surprisingly, our findings indicate that different separability measures have minimal impact on IREOS' effectiveness. However, using linear separation methods within IREOS significantly reduces its computation time. These insights hold significance for real-world applications where efficient outlier detection is critical. In the context of this work, we provide the code for the IREOS algorithm and our separability techniques.
CRIS-Team Scopus Importer:2024-07-05
Keywords: anomaly detection; model selection; Outlier detection; unsupervised evaluation