On-line Energy Optimization of Hybrid Production Systems Using Actor-Critic Reinforcement Learning

This paper presents a novel approach for energy optimization in large scale industrial production systems based on an actor-critic reinforcement learning (ACRL) framework. The objective of the on-line capable self-learning algorithm is the optimization of the energy consumption of the whole production process. Our central ACRL framework is realized by artificial neural network (ANN) function approximation using Gaussian radial-basis functions (RBF) for the critic and the actor, respectively, and gives the opportunity to cover not only a discrete but also continuous state and action space, which is necessary for hybrid systems where discrete and continuous actuator behavior is combined. For testing and validation purposes we develop a software model of our bulk good laboratory plant as application example for the developed ACRL algorithm. The model is based on mass-flow equations for a continuous bulk good supply whereas the energy consumption is modeled by functions dependent on the actuators behavior. The capability of our machine learning (ML) approach for energy optimization is underlined by simulation results for the task of supplying bulk good to a subsequent dosing section.