Data-Based Cost-To-Go Design for Optimal Control

An optimal controller design technique is presented in this paper. The regular parametric optimization approach leads to a nonlinear optimization problem even for a linear system. The presented approach converts a nonlinear optimization to a linear one, thus eliminates typical problems associated with nonlinear optimization. The approach relies on direct identification of the cost-to-go function from input- output or input-state data. The optimal controller is then obtained by minimizing this cost-to-go function. The optimal controller can be designed directly from data obtained from a simulation code or from the physical system. The need to perform system identification first to obtain an explicit model of the system in standard form is therefore eliminated. The derived optimal controllers can be either in state feedback form or dynamic output feedback form. Comparison of the proposed data-based optimal controller to the model-based linear quadratic regulator (LQR) is shown in a model aircraft example. This paper is focused on linear systems. Extension of the technique to the nonlinear case is reported in a companion paper.