Research on Overfitting of Deep Learning

Deep learning has been widely used in search engines, data mining, machine learning, natural language processing, multimedia learning, voice recognition, recommendation system, and other related fields. In this paper, a deep neural network based on multilayer perceptron and its optimization algorithm are studied. MNIST handwritten digital datasets were used to verify the reliability of the model and the optimization algorithm. The important correlation between the overfitting and the activation functions, network structures, training epochs, learning rates is verified. The study of overfitting is of great significance to reduce generalization error. This paper proposes an innovative activation function called: modified-sigmoid which is based on the well-known sigmoid function. This activation function can effectively improve the accuracy of the model and inhibit the overfitting problem.