Nonlinear Online Learning — A Kernel SMF Approach

Principles of adaptive filtering and signal processing are useful tools in machine learning. Nonlinear adaptive filtering techniques, though often are analytically intractable, are more suitable for dealing with complex practical problems. This paper develops a nonlinear online learning algorithm with a kernel set-membership filtering (SMF) approach. One of the main features in the SMF framework is its data-dependent selective update of parameter estimates. Accordingly, the kernel SMF algorithm can not only selectively update its parameter estimates by making discerning use of the input data, but also selectively increase the dimension of the kernel expansions with a model sparsification criterion. This results in more sparse kernel expansions and less computation in the update of parameter estimates, making the proposed online learning algorithm more effective. Both analytical and numerical results are presented in this paper to corroborate the above statements.