Multi-Agent Reinforcement Learning for the Energy Optimization of Cyber-Physical Production Systems

Bakakeu J, Kißkalt D, Franke J, Baer S, Klos HH, Peschke J (2020)

Publication Type: Conference contribution

Publication year: 2020

Journal

Canadian Conference on Electrical and Computer Engineering Institute of Electrical and Electronics Engineers

Publisher: Institute of Electrical and Electronics Engineers Inc.

Book Volume: 2020-August

Conference Proceedings Title: Canadian Conference on Electrical and Computer Engineering

Event location: London, ON CA

ISBN: 9781728154428

DOI: 10.1109/CCECE47787.2020.9255795

Abstract

The paper proposes an artificial intelligence-based solution for the efficient operation of a heterogeneous cluster of flexible manufacturing machines with energy generation and storage capabilities in an electricity micro-grid featuring high volatility of electricity prices. The problem of finding the optimal control policy is first formulated as a game-theoretic sequential decision-making problem under uncertainty, where at every time step the uncertainty is characterized by future weather-dependent energy prices, high demand fluctuation, as well as random unexpected disturbances on the factory floor. Because of the parallel interaction of the machines with the grid, the local viewpoints of an agent are non-stationary and non-Markovian. Therefore, traditional methods such as standard reinforcement learning approaches that learn a specialized policy for a single machine are not applicable. To address this problem, we propose a multi-agent actor-critic method that takes into account the policies of other participants to achieve explicit coordination between a large numbers of actors. We show the strength of our approach in mixed cooperative and competitive scenarios where different production machines were able to discover different coordination strategies in order to increase the energy efficiency of the whole factory floor.

Authors with CRIS profile

Romuald Jupiter Bakakeu Ngassam Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik Dominik Kißkalt Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik Jörg Franke Lehrstuhl für Fertigungsautomatisierung und Produktionssystematik

How to cite

APA:

Bakakeu, J., Kißkalt, D., Franke, J., Baer, S., Klos, H.H., & Peschke, J. (2020). Multi-Agent Reinforcement Learning for the Energy Optimization of Cyber-Physical Production Systems. In Canadian Conference on Electrical and Computer Engineering. London, ON, CA: Institute of Electrical and Electronics Engineers Inc..

MLA:

Bakakeu, Jupiter, et al. "Multi-Agent Reinforcement Learning for the Energy Optimization of Cyber-Physical Production Systems." Proceedings of the 2020 IEEE Canadian Conference on Electrical and Computer Engineering, CCECE 2020, London, ON Institute of Electrical and Electronics Engineers Inc., 2020.

BibTeX: Download