RAVEN: Reinforcement Learning for Generating Verifiable Run-time Requirement Enforcers for MPSoCs

Esper K, Spieck J, Sixdenier PL, Wildermann S, Teich J (2023)

Publication Language: English

Publication Type: Conference contribution, Original article

Publication year: 2023

Publisher: Schloss Dagstuhl -- Leibniz-Zentrum für Informatik

City/Town: Dagstuhl, Germany

Book Volume: 108

Pages Range: 7:1 - 7:16

Conference Proceedings Title: Fourth Workshop on Next Generation Real-Time Embedded Systems (NG-RES 2023)

Event location: Toulouse

ISBN: 978-3-95977-268-6

URI: https://drops.dagstuhl.de/opus/volltexte/2023/17738

DOI: 10.4230/OASIcs.NG-RES.2023.7

Open Access Link: https://drops.dagstuhl.de/opus/volltexte/2023/17738

Abstract

In embedded systems, applications frequently have to meet non-functional requirements regarding, e.g., real-time or energy consumption constraints, when executing on a given MPSoC target platform.
Feedback-based controllers have been proposed that react to transient environmental factors by adapting the DVFS settings or degree of parallelism following some predefined control strategy. However, it is, in general, not possible to give formal guarantees for the obtained controllers to satisfy a given set of non-functional requirements. Run-time requirement enforcement has emerged as a field of research for the enforcement of non-functional requirements at run-time, allowing to define and formally verify properties on respective control strategies specified by automata. However, techniques for the automatic generation of such controllers have not yet been established.
In this paper, we propose a technique using reinforcement learning to automatically generate verifiable feedback-based enforcers. For that, we train a control policy based on a representative input sequence at design time. The learned control strategy is then transformed into a verifiable enforcement automaton which constitutes our run-time control model that can handle unseen input data. As a case study, we apply the approach to generate controllers that are able to increase the probability of satisfying a given set of requirement verification goals compared to multiple state-of-the-art approaches, as can be verified by model checkers.

Authors with CRIS profile

Related research project(s)

How to cite

APA:

Esper, K., Spieck, J., Sixdenier, P.-L., Wildermann, S., & Teich, J. (2023). RAVEN: Reinforcement Learning for Generating Verifiable Run-time Requirement Enforcers for MPSoCs. In Fourth Workshop on Next Generation Real-Time Embedded Systems (NG-RES 2023) (pp. 7:1 - 7:16). Toulouse, FR: Dagstuhl, Germany: Schloss Dagstuhl -- Leibniz-Zentrum für Informatik.

MLA:

Esper, Khalil, et al. "RAVEN: Reinforcement Learning for Generating Verifiable Run-time Requirement Enforcers for MPSoCs." Proceedings of the Workshop on Next Generation Real-Time Embedded Systems Co-located with HiPEAC 2023, Toulouse Dagstuhl, Germany: Schloss Dagstuhl -- Leibniz-Zentrum für Informatik, 2023. 7:1 - 7:16.

BibTeX: Download