A reinforcement learning-based scheme for direct adaptive optimal control of linear stochastic systems

Reinforcement learning where decision-making agents learn optimal policies through environmental interactions is an attractive paradigm for model-free, adaptive controller design. However, results for systems with continuous state and action variables are rare. In this paper, we present convergence results for optimal linear quadratic control of discrete-time linear stochastic systems. This work can be viewed as a generalization of a previous work on deterministic linear systems. Key differences between the algorithms for deterministic and stochastic systems are highlighted through examples. The usefulness of the algorithm is demonstrated through a nonlinear chemostat bioreactor case study. Copyright © 2009 John Wiley & Sons, Ltd.