Investigation of a Model‐Based Deep Reinforcement Learning Controller Applied to an Air Separation Unit in a Production Environment
6
Citation
21
Reference
10
Related Paper
Citation Trend
Abstract:
Abstract The need for load flexibility and increased efficiency of energy‐intensive processes has become more and more important in recent years. Control of the process variables plays a decisive role in maximizing the efficiency of a plant. The widely used control models of linear model predictive controllers (LMPC) are only partly suitable for nonlinear processes. One possibility for improvement is machine learning. In this work, one approach for a purely data‐driven controller based on reinforcement learning is explored at an air separation plant (ASU) in productive use. The approach combines the model predictive controller with a data‐generated nonlinear control model. The resulting controller and its control performance are examined in more detail on an ASU in real operation and compared with the previous LMPC solution. During the tests, stable behavior of the new control concept could be observed for several weeks in productive operation.Keywords:
Model Predictive Control
Reinforcement Learning continues to show promise in solving problems in new ways. Recent publications have demonstrated how utilizing a reinforcement learning approach can lead to a superior policy for optimization. While previous works have demonstrated the ability to train without gradients, most recent works has focused on the simpler regression problems. This work will show how a Multi-Agent Reinforcement Learning approach can be used to optimize models in training without the need for the gradient of the loss function, and how this approach can benefit defense applications.
Presentation (obstetrics)
Cite
Citations (0)
This chapter introduces some typical predictive control algorithms based on the basic principles, with different model types, aiming at illustrating how the predictive control algorithm can be developed by concretizing these principles. Dynamic matrix control (DMC) is one of the most widely used predictive control algorithms in industrial processes. Generalized predictive control (GPC) is another kind of predictive control algorithms rising from the research area of adaptive control. The early predictive control algorithms generally adopted an input-output model, either nonparametric such as model predictive heuristic control or DMC, or parametric such as GPC. However, since the 1990s, the state space model has been widely adopted in theoretical research of model predictive control because such research needs to take the mature theoretical results of modern control theory as reference. The prediction model aims at providing the basis for solving the optimization problem.
Model Predictive Control
Predictive Analytics
Cite
Citations (0)
In multi-agent reinforcement learning, the state space grows exponentially in terms of the number of agents, which makes the training episode longer than before. It will take more time to make learning convergent. In order to improve the efficiency of the convergence, we propose an algorithm to find shortcuts from episode in multi-agent reinforcement learning to speed up convergence. The loops that indicate the ineffective paths in the episode are removed, but all the shortest state paths from each other state to the goal state within the original episode are kept, that means no loss of state space knowledge when remove these loops. So the length of episode is shortened to speed up the convergence. Since a large mount of episodes are included in learning process, the overall improvement accumulated from every episode's improvement will be considerable. The episode of multi-agent pursuit problem is used to illustrate the effectiveness of our algorithm. We believe this algorithm can be introduced into most other reinforcement learning approaches for speeding up convergence, because its improvement is made on episode, which is the most foundational learning unit of reinforcement learning.
Cite
Citations (1)
Learning a high-performance trade execution model via reinforcement learning (RL) requires interaction with the real dynamic market. However, the massive interactions required by direct RL would result in a significant training overhead. In this paper, we propose a cost-efficient reinforcement learning (RL) approach called Deep Dyna-Double Q-learning (D3Q), which integrates deep reinforcement learning and planning to reduce the training overhead while improving the trading performance. Specifically, D3Q includes a learnable market environment model, which approximates the market impact using real market experience, to enhance policy learning via the learned environment. Meanwhile, we propose a novel state-balanced exploration scheme to solve the exploration bias caused by the non-increasing residual inventory during the trade execution to accelerate model learning. As demonstrated by our extensive experiments, the proposed D3Q framework significantly increases sample efficiency and outperforms state-of-the-art methods on average trading cost as well.
Q-learning
Cite
Citations (1)
Q-learning
Spectrum management
Cite
Citations (13)
Due to model predictive control can better deal with the constraints problem of nonlinear systems and improve the dynamic performance of the controlled system, therefore, this technology has attracted much attention in the field of motor drive. This article first introduced the basic principle of model predictive control, continuous control set model predictive control and finite control set model predictive control. Secondly, summarized the research status of generalized predictive control, explicit model predictive control, model predictive current control, model predictive torque control and commonly used improved model predictive control in motor drive systems. Thirdly, prospected the future development trend based on the current research status of model predictive control in motor drive system. Finally, the advantages and disadvantages of the continuous control set model predictive control and the finite control set model predictive control were comprehensively compared, and the ways in which the two algorithms act on the motor drive system were summarized.
Model Predictive Control
Motor Control
Cite
Citations (1)
This paper investigates the usage of Discrete-time Linear Model Predictive Control in controlling a nonlinear Coupled Tanks System. Two different schemes of Model Predictive control are employed. To begin with, a basic Model Predictive Control based on Generalized Predictive Control is used and then a Model Predictive Control approach based on Laguerre functions. Simulation results have been included which demonstrate the performance of both controllers when used to control Single-Input Single-Output Coupled Tanks System and the performance when Laguerre based Model Predictive Control is applied to Multi-Input Multi-Output Coupled Tanks System.
Model Predictive Control
Laguerre polynomials
Nonlinear model
Cite
Citations (28)
Unmanned Aerial Vehicle (UAV) is increasingly becoming an important tool used for a variety of tasks. In addition, Reinforcement Learning (RL) is a popular research topic. In this paper, these two fields are combined together and we apply the reinforcement learning into the UAV field, promote the application of reinforcement learning in our real life. We design a reinforcement learning framework named ROS-RL, this framework is based on the physical simulation platform Gazebo and it can address the problem of UAV motion in continuous action space. We can connect our algorithms into this framework through ROS and train the agent to control the drone to complete some tasks. We realize the autonomous landing task of UAV using three different reinforcement learning algorithms in this framework. The experiment results show the effectiveness of algorithm in controlling UAV which flights in a simulation environment close to the real world.
Drone
Cite
Citations (4)
Introduction Common Linear Models Used in Model Predictive Control Prediction in Model Predictive Control Predictive Control-The Basic Algorithm Examples - Tuning Predictive Control and Numerical Conditioning Stability Guarantees and Optimising Performance Closed-Loop Paradigm Constraint Handling and Feasibility Issues in MPC Improving Robustness-The Constraint Free Case The Relationship Between Modelling and the Robustness of MPC Robustness of MPC During Constraint Handling and Invariant Sets Optimisation and Computational Efficiency in Predictive Control Predictive Functional Control Multirate Systems Modelling for Predictive Control Appendices Conclusion
Model Predictive Control
Robustness
Cite
Citations (1,034)
With the wide application of predictive control in industrial processes, the theoretical research on predictive control also achieves great development in recent years. In this paper, recent advance in research on predictive control performance is surveyed, which includes the stability, robustness, feasibility of predictive control systems, as well as nonlinear predictive control.
Model Predictive Control
Robustness
Cite
Citations (9)