Learning control of a bioreactor system using kernel-based heuristic dynamic programming

To solve the learning control problem of a bioreactor system, a novel framework of heuristic dynamic programming (HDP) with sparse kernel machines is presented, which integrates kernel methods into critic learning of HDP. As a class of adaptive critic designs (ACDs), HDP has been used to realize online learning control of dynamical systems, where neural networks are commonly employed to approximate the value functions or policies. However, there are still some difficulties in the design and implementation of HDP such as that the learning efficiency and convergence of HDP greatly rely on the empirical design of the critic and so on. In this paper, by using the sparse kernel machines, Kernel HDP (KHDP) is proposed and its performance is analyzed both theoretically and empirically. Due to the representation learning and nonlinear approximation ability of sparse kernel machines, KHDP can obtain better performance than previous HDP method with manually designed neural networks. Simulation results demonstrate the effectiveness of the proposed method.