Fully Adaptive Dictionary for Online Correntropy Kernel Learning Using Proximal Methods

Abstract We introduce a new sparse variant of the Correntropy Kernel Learning model, hereafter named Fully ADaptive Online Sparse CKL (FADOS-CKL), for online system identification in the presence of outliers. For this purpose, we develop a fully adaptive dictionary of support vectors (SVs) so that it can either grow (as most of the kernel models to date do) or shrink if some of the SVs become obsolete with time. For inclusion of SVs into the dictionary, existing strategies (ALD, Novelty, Surprise, and Coherence) have their performances compared in this paper, while for elimination of SVs we adopt a class of optimization techniques known as proximal methods. Dictionary updating in FADOS-CKL is carried out on-the-fly by the introduction of a recursive methodology based on the Sherman-Morrison-Woodbury formula to update the kernel matrix and its inverse with low computational complexity. Aiming at achieving the smallest predictive errors with the highest sparsity level, a comprehensive performance comparison involving the FADOS-CKL model and powerful alternatives is carried out using two large-scale benchmark datasets for different levels of outliers contamination. The results indicate an impressive balance between reduction in the dictionary size and the corresponding generalization capability of the proposed FADOS-CKL model over the existing alternatives.