Safe reinforcement learning for discrete-time fully cooperative games with partial state and control constraints using control barrier functions

In this paper, a novel safe reinforcement learning is proposed for fully cooperative games of discrete-time multi-player systems with partial state and control constraints. The fully cooperative game is a special case of the nonzero-sum games where all players cooperate to accomplish the common task. However, there are few works for fully cooperative game issues of discrete-time systems with partial state and control constraints. The issue is addressed by our algorithm based on the constrained value iteration framework using the measured data along the system trajectories, and the Nash equilibrium of the constrained fully cooperative game is achieved. Compared to previous methods for fully cooperative game issues, neither the accurate system dynamics nor the initial admissible control policies are required via the algorithm. Meanwhile, the discrete-time exponential control barrier functions are adopted to address the issue of state constraints. Moreover, the convergence of the proposed algorithm is proven in theory. Then, the system dynamics, the control policies and the value function are approximated by the three-layer neural networks, respectively. Finally, two experiments are presented to demonstrate the safety and effectiveness of the proposed algorithm.