Self-Tuning Attitude Control Design of Hypersonic Aircraft with Multi-Constraints Based on Soft-Constrained PI2 with Action Smooth

Nonlinear control has led to several great achievements for attitude control in virtual simulation environments, where the system model can be simplified to meet the application condition of those traditional nonlinear control algorithms. However, only a few of those algorithms can be applied to a practical Hypersonic Aircraft system, which suffers from uncertainties, multi-constraints, and various noises. To solve those problems effectively, in this paper, a novel reinforcement learning framework based on PI2 is proposed, wherein three kernel techniques are introduced. Firstly, a generalized path- integral-control approach is proposed to obtain the numerical solution of a stochastic dynamic model, wherein the calculation of the gradient and matrix inversion is avoided to ensure fast and reliable training convergence. Secondly, an action smooth method is used to strengthen the ability of the controller to resist noise. Thirdly, a novel RL algorithm combined with soft constrained techniques is illustrated to address the constrained nonlinear control problem. Several experiments are carried out and analyzed to demonstrate the outstanding performance of the CS-PI2.