Regression and Multiclass Classification Using Sparse Extreme Learning Machine via Smoothing Group L1/2 Regularizer

Extreme learning machine (ELM) is a simple feedforward neural network, and it has been extensively used in applications for its extremely fast learning speed and good generalization performance. Nevertheless, it is implemented normally under the empirical risk minimization scheme and the model trained by ELM is prone to overfitting. In addition, the ELM provides more nodes than it actually needs, this means that the network structure is not sparse enough. To solve above problems, two efficient algorithms for training ELM: Group $L_{1/2}$ regularization and smoothing Group $L_{1/2}$ regularization methods are proposed in this article. But, the basic group $L_{1/2}$ regularization is nondifferentiable at the origin and which causes oscillation. So, we modify the basic group $L_{1/2}$ regularization by smoothing it at the origin. Simulation results show that the ELM with smoothing group $L_{1/2}$ regularization can effectively prune redundant nodes and redundant weights of the surviving nodes, which has better performance than traditional ELM, the ELM with $L_{1}$ regularization method, and with group $L_{1/2}$ regularization method.