Data Driven Relationship Discovery in Large Time Series Datasets (DARTS)
Third party funded individual grant
Acronym: DARTS
Start date : 01.04.2022
End date : 31.03.2025
Project details
Short description
This research project aims to develop different efficient methods that enable knowledge mining in large time series datasets to uncover relationships and descriptive patterns within the dataset. A key focus is the identification of relationships, which are uncovered using the technique of changepoint correlation that we are actively developing within the project. As the project aims to support or even automate a manual process, visualization, and demonstration of the techniques in question also play a major role in the development.
Scientific Abstract
Modern complex systems, such as power plants or other industrial structures, combined with the rise of IoT and Industry 4.0, produce thousands of time series measuring different aspects within these systems. As time series measure the state of these complex systems, the correct identification and integration of these time series are key to enabling advanced analytics and further optimization. As acquiring contextual information about each time series and their relations is currently a time-consuming and error-prone manual process, techniques to support or even automate this process are in high demand. While there are different available metadata formats, such as Brick, this metadata often is not available for all data sources and is not commonly used for all systems. Integrating time series at scale requires efficient algorithms and robust concepts that can deal with the heterogeneity and high volume of time series from different domains.
Additional Applications and Outcomes:
Changepoynt Python Package
Changepoint correlation heavily relies on suitable changepoint detection algorithms, many of which were implemented from research papers within a pip-installable package "changepoynt" (https://changepoynt.de). Changepoint detection, a critical task in time series analysis, identifies abrupt shifts or transitions in data patterns, offering insights into underlying phenomena. Developed with flexibility and scalability in mind, "changepoynt" integrates a range of state-of-the-art methods for changepoint detection, empowering researchers across domains to efficiently analyze and interpret their data.
CATCH: Contextual Anomaly Tracking with Changepoint Detection
Together with a research partner from the industry, we basically use the inverse of our idea of relationship discovery to detect contextual anomalies. The hypothesis of the project states that signals, which should have relations (e.g. Input-Output measurements of a dynamical system), behave anomalously if they stop showing simultaneous changes. In contrast to classical anomaly detection methods, change point anomaly (the comparison of multiple changepoint signals) is mainly targeted at contextual anomalies, where two signals are measuring the same component, and consequentially should change at similar times when the plant changes operational status. In case the signals change separately, a contextual anomaly occurs. While the methods are available in theory, the project is necessary to test the applicability, feasibility, and correct parametrization of the methods for selected use cases. A demonstrator for a two-dimensional case can be found under https://anomaly.changescore.de/ and for the multi-dimensional case under https://heatmap.changescore.de/.
