Support Vector Machines

In this chapter we introduce basic concepts and ideas of the Support Vector Machines (SVM). In the first section we formulate the learning problem in a statistical framework. A special focus is put on the concept of consistency, which leads to the principle of structural risk minimization (SRM). Application of these ideas to classification problems brings us to the basic, linear formulation of the SVM, described in Sect. 30.3. We then introduce the so called “kernel trick” as a tool for building a non-linear SVM as well as applying an SVM to non-vectorial data (Sect. 30.4). The practical issues of implementation of the SVM training algorithms and the related optimization problems are the topic of Sect. 30.5. Extensions of the SVM algorithms for the problems of non-linear regression and novelty detection are presented in Sect. 30.6. A brief description of the most successful applications of the SVM is given in Sect. 30.7. Finally, in the last Sect. 30.8 we summarize the main ideas of the chapter.