Feedback control techniques for gradient based learning

The well-known gradient descent based learning strategy is considered as a feedback control system. In order to improve the convergence performance of the gradient based learning, both a proportional+integral+derivative (PID) control based gradient descent learning and a fuzzy control plus gradient descent learning strategies are proposed. The convergence properties of the proposed methods are discussed. The comparison investigation concerns the gradient technique and the proposed approaches are finally illustrated.